In This Edition:
Helix RADE 6.2 Preliminary Release 8: Deleting the Undeletable
16 January 2012 — The good progress continues with the eighth Preliminary Release of Helix RADE 6.2 for macOS available today.
To those of you who signed up to the Europa Pioneer Plan via the monthly payment plan, your $20* charge appeared on your credit card on the 10th, and earlier today you should have received an email containing a link to the newest installer. Click that link to download the lastest update to your computer. (You can also get it by logging in to our web store and checking your current licenses.)
For those out there who haven’t joined us yet, remember that if you’re thinking of joining the monthly plan this month, the initial payment is now $160. The product you receive today benefits from the contributions of your fellow Helix users who have already joined. All subsequent monthly payments will be $20.
But if you are thinking of joining up now, why not sign up for the annual subscription instead? For only $40 more, you’ll get every update through May 31, 2012. There should be four more releases between now and then, so it’s like getting two months worth for free.
We also need to remind everybody that signed up for the Annual plan that the current plan runs through May 31, 2012, regardless of when you signed up. That $200 gets you “all of the progress made in the first year of the plan,” not “all of the progress made over the next year.”
The installation process remains the same: mount the installer disk image, drag the new Helix RADE onto the Applications folder and let go. If you are updating from an earlier version of Europa, double click the Install Update icon and your Europa RADE will be seamlessly updated to the new version, even if you have a collection open! (You will be prompted to save changes before closing an open collection.)
One last note regarding updates: if you are using the Interim Builds and would like to help us test an ‘automatic update’ process, contact Matt. He’s got something you might really like.
As always, the complete notes regarding what’s new in this release are found in the Preliminary Release 8 Release Notes. Here we want to highlight some of the more important advances.
AppleScript Project is Complete
This month’s release brings us to the completion of the AppleScript project. Of course, there will be the inevitable bug fixes and minor improvements, but as the ‘About Helix RADE’ screenshot on the right shows, every Helix icon is now fully scriptable. There is now no Helix collection construction that can not be created using AppleScript. One can now, if sufficiently motivated, live without Classic, with no asterisk pointing to a footnote that says ‘except for …’ anything.
We don't get to beat our chests that often. Getting the AppleScript project finished in eight months is one of those moments we need to savor. That we managed to complete this huge task in such a relatively short time is a testament to our Lead Engineer and Helix co-creator, Larry Atkin. Often working under less than optimal conditions (to put it mildly), it is Larry’s devotion to the cause of seeing Helix through this monumental transition that has made the rapid progress to this day possible.
And ‘where we are’ is in fact a very good place. Along the way to a complete AppleScript interface to both User Mode and Design Mode, we’ve been making modest improvements to the graphic interface and fixing some very long-standing bugs, some of which should excite you even if using AppleScript is not part of your plans for Helix.
We’ll get to those in a moment. But lest we give AppleScript short shrift today, let’s stop and celebrate the final pieces of the puzzle: The Abacus Expression Language (AEL) which was described in the December 10th The Latest Word is now write-enabled, so you can create any tile construction using a plain text ‘expression language.’ We’ve used this to create some very complex abaci, and it works great. For those of you who want to dig in to this new way of creating abaci, this technote contains the complete technical documentation that explains the language used to create an abacus expression.
To show you how it works at a simple level, we’ve added a new Make A Constant script to the sample scripts page. This script does exactly what it says: it makes any abacus constant you want. Need a ‘true’ flag or ‘0’ fixed point constant? This simple script shows how simple it is. (Actually, the script is more complicated than that, because it lets you create any constant; the script Make True Constant on that page make a constant for ‘true’ in less than 10 lines.)
Along with finishing that off, we wrapped up a few loose ends. We added more summarization flags to the abacus class, so you can now tell at a glance whether the abacus uses any of the tiles that can’t be indexed; improved the view code so that changes to the template no longer force the view to close and reopen; and made it so that changes to a user’s menu items while in User Mode now update the menus immediately.
Deleting the Undeletable
Back in the mid-twentieth century, man was at last hurtling into space. Only a few people could go, but everyone on earth could benefit from what was being learned on that frontier. In the Helix universe, that time is now. Things we have learned from working in macOS and AppleScript are now beginning to provide benefits for all Helix collections. Something that happened between PR7 and PR8 will illustrate.
A major, long-standing problem in Helix has been fixed, thanks in no small part to Helix’s new AppleScript capabilities. But this is not just about AppleScript, so if you’ve had it up to here with all the AppleScript talk, this news is right up your alley.
Often, when we are looking at customer collections to help them work through some thorny problem, we see icons labeled with some variation of “DO NOT DELETE.” Words, bullets, exclamation points and other myriad ASCII characters are used to flag these notorious icons which, when discarded, cause the collection to fail Update Collection’s structure check.
A large mythology has built up over the years about them, but the reason the problem exists is really not a mystery at all. A few years ago, we even published a technote describing the root cause of this problem. We pointed out that the damage was actually caused by something that had happened in the past — and that throwing the icon away simply reveals what was there all along. That technote concluded by saying that we would search for a solution to the problem. Of course, since then, many more pressing needs have kept us from pursuing it as we would have liked.
But when we started testing the AEL write capability, we immediately got a report of damage in our own techdb. Further investigation revealed that, because of the way AEL creates an abacus, the ‘latent damage’ described in that technote would rush right to the surface. Clearly it was time to address this longstanding bug.
As it turned out, there were actually two problems that needed to be addressed. The internal ‘backlink’ tables we referred to in TS1042 are reported with diagnostic code 30, but there’s a mirror image complement to every backlink table, which we call the ‘SUL’ table. When an SUL table goes ‘out of sync’ that is reported in Update Collection with diagnostic code 36. We’ve done a few collection repairs over the years to fix that particular error, but could never find out what was triggering them. We knew it wasn’t tied directly to deleting old icons, but without a reproducible case, we were at a loss to know how they were getting out of sync.
Helix RADE 6.2b8 (5767) finally puts this bug to rest. It does so by cleaning out all of the backlink and SUL tables when a collection is opened. Because of the speed of current Macs, even a PowerPC Mac, this happens in a matter of seconds, even in a large collection.
However, those tables are not also rebuilt when the collection is opened. Instead, rebuilding is deferred until the first time you open a view. We won’t go into technical reasons for this now; all you need to remember is that if you open a collection with Helix RADE 6.2 you must open at least one view before saving and quitting if you are going to reopen that collection with any Helix 6.0 or 6.1 product.
But don’t let that disuade you from using Europa! In fact, because any collection that has been in use for many years may have internal tables that are out of sync, it is our strong recommendation that all collections be opened with RADE 6.2b8 or later in order to rebuild those tables and eliminate the possibility of this bug being able to affect you in the future.
If you are already using Helix 6, but not yet in the Europa Pioneer program, all you have to do is get on board. Program members can take any collections they are using and do this with confidence.
Of course, if you are not yet using any Helix 6 product, doing this means upgrading, and this is a very compelling reason to make the leap now, even if you have no plans to use AppleScript. Work on the graphical user interface is at last front and center and, for the foreseeable future, Classic RADE 6.1 can work together with macOS RADE 6.2 on your collections, so you have a wide safety net,
Once you’ve done that, go ahead and delete those awful icons and do it with total confidence that the potential for seeing a ‘not corrected’ result has now been completely eliminated.
But Wait! There’s More!
As they say on TV, that’s not all!
We also fixed another bug that has afflicted Helix users for years. This one never resulted in collection damage, but it sure was annoying when it struck. Classic Helix users who work in Design Mode’s ‘View by Kind’ or ‘View by Name’ lists are occasionally afflicted with icons that simply will not behave.
These icons are ‘stuck’ in the selection, and can only be deselected by special tricks best described as ‘fiddling around’ with the icons in the relation.
In this case, the bug was triggered by pasting structure into a relation, and the result was that another internal table — the NSI table — would get out of sync with reality. We fixed that bug, but knowing that there are certainly collections out there where this harmless but annoying bug has already struck, we made it so that choosing the Select All Rows command rebuilds that relation’s NSI table. So if you discover an icon that simply won’t cooperate when you try to select it, deselect it, move it, etc., just press CMD-A (the shortcut for ‘Select All Rows’) followed by CMD-Opt-A (the shortcut for ‘Clear Selection’) and the internal table will be rebuilt. Poof! the bug is squashed.
And while we were in there, we took a moment to address one of our personal pet peeves. Have you even noticed that when you paste icons into a relation that is in one of the list views, the icons you pasted end up way down in the lower left corner of the icon field when you switch to View by Icon? They are now placed in the general vicinity of other icons, as seen when the window is opened in icon mode.
Sadly, Helix RADE doesn’t actually have ‘View by Icon’ mode yet, but you can see that we haven’t abandoned it. We’re preparing the code for the day we can restore this fan favorite.
And as long as we’re talking about ancient bug fixes, we discovered a bug in the way the code handles the result of a Styled Text tile in an abacus. This bug appears to be harmless, but eliminating it means one less thing that might trigger ‘flaky’ behavior in Helix.
All of these bug fixes bring a new level of confidence to Helix, such that we decided that RADE no longer need to display the no anomalies detected dialog after a save. The only notification comes now when inconsistencies are detected.
Act Now and We’ll Include…
Since Helix took its first steps into the macOS world five years ago, performance has been subject of debate. Although we readily admit that the early releases were quite slow compared to Classic, we have, since Helix 6.1.6, been satisfied that Helix is as fast as, or faster than, Classic Helix in most head-to-head tests.
Yes, there are some areas, such as views that use multiple subforms, where there is still work to do but all in all performance is pretty good. But that doesn’t mean we aren’t attentive to ways to make Helix faster, and when one such opportunity presented itself, we grabbed it.
The latest improvement has to do with lists. Previously, a list view would cache only what was necessary to display the currently visible pages. The result was that as soon as you scrolled to a new page, you had to wait a moment while the next page filled in. That is no different than how it always was in Classic. The problem is that a Helix list can contains hundreds of thousands of records, and loading the entire list into memory, while impossible in Classic Helix, simply isn’t practical even with today’s system’s gigabytes of memory.
But now, for the first time, Helix caches more than just the visible pages. By default, a list view now caches both one page before and one page after the currently visible pages, making the next or previous page appear instantaneously. Depending on how you work with lists, you may never see the ‘data pending’ checkerboard pattern again. In fact, for a list of three pages or less, you can now scroll through the list with absolutely no redrawing. That’s something not even Classic Helix could boast.
But we took it a step further, and added a new preference (HxListPageCacheSize) you can use to control the number of pages that are cached on your machine. If you’ve got a really fast machine, you might be able to cache a dozen or so pages without a terrible delay. We’ve updated the Editing Preferences page with information on the new preferences in Europa and updated the Helix Preference Editor (found on that page) for those who want assistance setting this preference.
But be warned: caching pages requires memory, and increasing the number of pages cached has an increasingly detrimental effect on performance. A more powerful computer can handle more pages, but even on a very fast Mac, values above 20 are essentially unusable. The chart on the right shows the results of our tests on an Intel Mac mini.
A “What’s Next” Teaser
If you can’t already tell, we’re quite happy with this release. We’ve reached a major milestone, fixed some longstanding bugs, and added some ‘fit and finish’ to the experience.
In fact, we’re so happy, we decided to update the Preliminary Release Warning dialog that appears when you launch Europa. For those of you who have checked the ‘Do not show this dialog again’ box and won’t see it, the new wording can be seen in the screenshot on the right.
Our plan was that as soon as we reached this milestone, we would turn our attention fully to the graphic interface, so those of you who aren’t on the AppleScript bandwagon can start to see some real progress in that direction. But you may not see much of that on February 10th because of a major opportunity that has come our way.
Exactly what that opportunity is, we won’t reveal just yet, but here’s a hint: along with all of the major improvements noted above, there are many minor improvements and bugs fixes this month. (See the release notes for the full list, and when you are there, make note of the one that reads: “All bugs reported in the Paste Structure dialog have been fixed.”) Now, why would we want to pay particular attention to that?
Beyond that, all we’ll say is that if you haven’t joined the Europa Pioneer Plan yet, you’re going to want to be in it next month.