In This Edition:
Stability, Speed and Things to Come
19 June 2014 —Sometimes, you just want to take a deep breath and relax. Parents are all too familiar with this desire. You’ve spent weeks trying to get your child over a nasty cold and now she seems to be doing better, so you decide that tonight will be okay to hire a babysitter, hop in the car and go see a movie. But halfway through the movie, your cell phone, whose volume has been shut off to respect the rules of modern cinema, begins to vibrate and before you know what hits you, you’re back in the car, on the way home, trying to stabilize the situation once again.
On that ride home, you want to get there as quickly as possible, without losing your license. How fast you can go is affected by many variables. But every once in a while, you discover a really good short cut while wandering seemingly aimlessly through back streets.
And when all is calm and quick once more, you can close your eyes and dream of better things to come.
One of the unfortunate consequences of good bug fixing work is that it can occasionally bring issues previously thought to be rare diversions from normalcy into the spotlight. While these problems are not what we would usually classify as show-stopping bugs, they tend to get their priorities elevated when they are observed with increased frequency and by more people.
All of which is a long way of saying that after solving the AquireEventFromQueue (AEFQ) bug and resuming forward motion toward the Helix of the future, a handful of recurring issues began appearing in RADE and Client, causing enough of a problem that we chose to pause long enough to address them. After a few very significant problems — the kind that caused crashes — were corrected, we realized our forward motion had stopped and being able to resume that forward motion meant another 6.2 maintenance release and all the testing and documentation work that entails.
The fruit of that labor is Helix 6.2.4, which is available today. It is a free maintenance upgrade to all owners of a Helix 6.2 product. The complete list of bug fixes and other improvements is found on the Helix 6.2.4 Release Notes page. The pricing remains the same, for those among you still mulling over a decision to upgrade.
An unforseeable speed bump
The more we operate in macOS, the more we learn about its quirks, the more we need to adjust our assumptions about how Helix should and does perform in macOS. Those assumptions are grounded in deep-rooted perceptions nurtured by three decades of pre-macOS experience.
When one must adapt to a totally new set of circumstances, feelings of loss generally permeate the experience. Some of the things we have lost in Helix — such as the ability to build popup menus with bright yellow 36-point Palatino bold italic type knocked out against a red background — have by now been long forgotten and are probably not missed by too many Helix users. But others are harder to simply kiss off, and of all the issues users have had to cope with in making their adjustment, speed has been the one missed the most.
Dogged by speed issues throughout its early history, Helix was finally able to put those issues behind it with the introduction of RAMJet in Helix 5.0. The process of properly setting up RAMJet — known today as the ‘collection buffer’ — was a bit cumbersome, but well worth the effort because the resulting performance was blindingly fast. So fast, that it laid down a yardstick against which every future incarnation of Helix would be measured.
While macOS is light years more cool than previous Mac operating systems, and while it is incredibly cool that Helix is finally running in macOS head to toe, macOS performance can be maddeningly frustrating at times, just as any OS can be. And when that happens, performance of the applications running in it can and do suffer.
There has been a seemingly endless cycle of frustration with Helix performance while getting all these products to macOS. Why, users always seem to ask, why are we wasting time and energy on whatever we have been doing when we should have been working to make Helix faster? Well, you can’t make a car that does not yet exist go faster than it does, or, for that matter, as fast or faster than any other car. Not only is it illogical; it’s impossible. Our process has been — and will remain — one in which functionality comes before performance. Once and only once you rid something of most of its perceptible imperfections, can you truly productively address issues of performance.
During the bug fixing that was slated for 6.2.4, a user alerted us to a performance problem observed in Client/Server. It appeared to them that performance was degrading with each additional client that logged in. With a full complement of users, running one of their daily tasks went from being a little slow to unbearably so, approaching several minutes to execute. On the surface, the problem appeared to be quite intractable and would demand a depth of intellectual penetration such that we were extremely hesitant to plumb so close to putting the maintenance release out. The last thing we wanted was another multi-month delay, as has happened before.
The first step in fixing any problem is to identify it, then isolate it as much as possible. This is a job that falls into Matt Strange’s realm, and when the sample supplied is in a complex collection, the repetitiousness of ‘remove another piece and test again’ can become a mind-numbing grind. And if the problem is in Client/Server, as opposed to RADE, the complexity of the process increases exponentially.
If it has never been properly drawn in The Latest Word before, now is the time to clearly draw the distinction between inspiration and creativity. Inspiration is a solo act. It’s that ‘aha moment’ that happens to someone seemingly out of the clear blue. “It’s really dark in here,” mused Thomas Edison one night when his supply of paraffin had been depleted. “I’ll bet that if I could find a way to light a tungsten filament, it could burn for hours!” Creativity, on the other hand, is when an inspiration is handed to a group of people with the skills to turn that inspiration to reality. The latter is much lauded over the former, but creativity is useless without that spark. You cannot create it. It just has to come.
Suddenly, in a pure moment of genius, it did. Matt realized that the problem had nothing at all to do with Client/Server and how many people were logged in. It had to do simply with how many windows were open, regardless of the Helix application. By duplicating the offending view ‘a whole bunch of times’ and opening them all in RADE, he was able to reproduce the problem perfectly. From there it was a few quick steps to reduce it to a very simple example.
At last we had the little rat cornered, but exactly how to ‘kill it’ was still a matter to be determined. Most of the ideas we floated amongst ourselves seemed too potentially disruptive to the stability we had achieved, but we took a run at it; at least we could say we tried.
Our engineers put their heads together, looked at what Matt had found, and not only was this specific performance bottleneck removed, but several others seemed to fall away with it. We could not be sure at first, but the performance differences we observed were, in some cases, astounding. Naturally, we needed to do some testing, and the testing bore out that everything was working properly with some processes actually running twice as fast as before and faster. With that sort of potential payoff, we decided to delay the release of 6.2.4 until we could test these changes internally and then give them to our beta testers for final approval.
What began with a simple inspiration resulted in a more creative solution than we could have anticipated at the outset. We took a little longer than we wanted to, but the effort will be rewarded many times over in increased productivity.
Things to Come
Finally, we come to what lies ahead. While we have already started down the road to the next Helix, we have now had four detours back through 6.2. After each one, we have rejoiced in the belief that we were finally ready to forge ahead into the unknown future, but each time, that joy was quickly tempered.
The first macOS RADE shipped a little more than a year ago. And the first new Client/Server, 6.2.1, based on that code has been out since this past November. Helix 6.2.2 with support for Mavericks debuted this past January, and 6.2.3 at the end of April.
A lot of frustration, punctuated by occasional moments of validation. And right in the middle of all that, we asked you to consider the future in the form of 247 potential new features for Helix. And you did. A decision will be made about each of them, that is, whether or not to include them in the upcoming version or defer them. Input from the latest edition of The Feature Game will be considered in those decisions. While a comprehensive review remains incomplete as of this writing, there are a few decisions we can share with you that are already fairly well etched in stone.
In what was really no surprise to us, far and away the most requested new feature was a web client that allows anyone to connect to a Helix collection from a web browser. This ultimately means that a user can create web pages right in Design Mode in their Helix collection and a person can log in and view, enter and edit information on those pages.
While having a Helix client for iOS or Android devices and Windows PCs were also highly requested features, the larger issue, for Helix, has always been to be able to communicate as far and as wide as possible. With a browser client, we can achieve greater penetration in a reasonable period of time than we ever could have if it required developing a Client application for all those other platforms. The one thing they have in common is that they have browsers. For us, it was a no-brainer. Once we have that capability, it will always improve, just as the web has done. And we can always develop iOS, Android and Windows apps later.
So, in evaluating the results of the survey, our first priority was to gather together all of the requested features that would have to be implemented in order to make a web client work. We were also pleased to see that several of the items that fared well in the survey were things that we have always known that Helix had to have, but could not be done until we had replaced Helix’s Open Transport Networking code with Sockets (6.2.2) and were ready to actually modify the basic collection structure. One example of this is encryption of Client/Server communications.
Giving Helix designers the ability to make a collection available to anyone using any kind of computer is clearly a giant step for Helix; being able to do that without reliance on any third-party software is completely in keeping with the Helix paradigm. But this doesn’t mean that the current crop of third-party ways to enter, extract and display data from a Helix application will cease to be useful in any way. Nor will this new way of connecting to Helix over the internet replace the existing Helix Client. But it will open a new world of possibilities and continue to level the playing field for Helix users.
While browsers and actual client applications are both observed through the same piece of glass, they are significantly different pieces of software, each with its own rules, features and limitations. The critical piece to take away from this news is that for the first time in its history, Helix will be able to reach across platforms in a single bound. The first iteration of the ‘client in a web browser’ won’t do everything the current Helix Client application does. Over time, as technical barriers are overcome, that interface will improve. For now, it is where the future begins.