This web page is designed to provide an honest look at what makes a good bug report. If you are plagued by a bug in Helix and you want to see it fixed as soon as possible (and who doesn’t?) you owe it to yourself to read this page.
Much of what is found here is stated in similar fashion in an article by Brett Simmons of Ranchero Software, the creator of NetNewsWire and MarsEdit. But Brett leaves a few things unsaid that I want to address here.
So you read that and think: why does any of that matter? Isn’t it enough that I reported it? Sadly, no. If you are reading this, you are probably aware of how far behind the OS X curve Helix is. Our efforts are all directed toward getting OS X Helix done. The only way we will consider stopping to fix a bug is if it is either a) a critical flaw or b) made so blindingly clear that fixing it won’t take that much time.
Of course, any bug we see we try to fix. (If we can’t fix it quickly, we document it in the Known Problems section of the Release Notes.) So, getting us to see the bug is your top priority. How do you do that?
When it comes to bug reports, this is pretty much how it works: when a bug report is read, the first reaction is to parse it for ‘coherence’ — is it clear and concise? Write as though you have 30 seconds or less to grab our attention and give us a compelling reason to look deeper into your report.
Using our requested channels is important. Describing the bug as directly as possible is also highly desirable. Writing in a non-adversarial tone helps. Brevity (where possible) is critical. Accessibility to the information is also important. Simple demonstrable examples are a huge bonus. And lastly, avoiding speculation is big plus too. Do all that and you’ve got a pretty good chance of getting an engineer to stop and look for a fix.
Let’s consider the desired characteristics of a bug report one by one:
- Channels. Our recommended procedure is that you start by logging in to our online Helix database at techdb.qsatoolworks.com and searching the Bug Report list to see if others have already reported this bug. You can participate in the discussion and may even find a workaround. If you don’t find your bug there, you can enter a new bug report, but we request that you first contact technical support and discuss the issue. We may have already identified the bug, or we may determine that your issue is something else altogether — maybe a confusing spec in Helix.
- Coherence. Can we make sense of what is written? Not using accurate terminology makes it harder to follow what you are trying to say. Because Helix is a visual language, many people have invented their own words for the elements within a collection. But that is your language, not ours. If you take the time to ‘translate’ into the terms we understand, you make it much easier for us to follow your description of the bug. Likewise, references to code names (as in: "I‘m running Pele on a Blackbird under Puma") should be avoided.
Another serious issue is pronouns. Avoid them. Today I read a non-Helix bug report that said, "When I transfer a file between my Mac and my PC, it crashes." OK, what crashes? It may be obvious to you which it you are referring to, but keep in mind that we are starting from square one in trying to understand your problem. - Accessibility: Getting a report as an attachment in a text file is no fun. techdb (see above) is our primary channel for attachments. Emailing an attachment puts it on my desk. Sending it via techdb puts it in the hands of everybody here at QSA. If an engineer is assigned to "work on bug# 7023" and everything he needs is found in the expected location, that’s a formula for quick action.
- Tone: This one is difficult, but when sending a bug report be mindful not to ruffle feathers unnecessarily. When you report a bug, you are essentially saying to an engineer ‘you screwed up’ so you should try to soften the blow as much as possible. Phrases like ‘This is unacceptable’ and ‘This is costing me money’ often carry the connotation that we’re just being negligent or callous. Right or wrong, a personal reaction ensues on our end which has the net effect of knocking your report down on the importance ladder a few rungs. The old adage ‘You can catch more flies with honey than you can with vinegar’ is still around because it is true.
- Directness: Give us the important facts right up front. Don’t wait until the 8th paragraph to say ‘This bug only happens after running the collection for hours.’ Having used no such qualifiers up to that point, leads us to believe you have found a straightforward logic bug. Bugs that appear only after extended use are in a wholly different category and require much more one-on-one work with you to help up identify the source. Leading us through a complex series of steps and then saying, in effect, you probably won’t see the bug anyway is frustrating.
- Brevity: Short and to the point is best. Your personal Helix history is usually irrelevant. The exception to this rule is when you can point to a specific version where the bug first appeared. Using a "1-2-3" outline of the steps to reproduce is always better than a narrative rendition.
- Speculation: Unless you have demonstrable skills in programming, don’t offer suggestions as to where in the code the problem lies. (But if you do have programming skills, please contact us! We might give you an opportunity to work in the code and fix these bugs yourself!) History has proven that speculation of this sort is always wide of the mark. At best it merely distracts from the engineer’s ability to focus on the true source of the problem.
- Demonstrability: If you’ve got a well-documented sample that shows the bug, your bug immediately moves up the priority ladder two or three rungs. This does not mean you can just send us your existing collection with a ‘the bug is in here’ (somewhere) note. At the very least it means adding text labels (or comments, in the case of icons) that include everything that was also said outside of the collection. It may be logical to assume that the engineer can always refer to the other source material but even if they can, it isn’t the most efficient way to work. Bug swatting is a demanding task that requires significant focus. Every possible distraction needs to be eliminated. By putting the documentation right in the collection — right on the template itself even — you keep the focus directly on your bug. If you send a sample collection, the tighter the example, the better. By removing irrelevant rectangles from the template it becomes easier to see the bug, and to avoid chasing red herrings. Keeping the engineer focused on the code always makes for faster bug fixing.
It’s not that we don’t want to fix bugs. It’s just that the task of moving Helix to OS X is immense and chasing down old bugs is hard work. (If it were not so, they would have been fixed long ago!) We have to choose which tasks we work on very carefully, so the fact remains: easier you make it for us to see your bug, the better the chances are that we will be able to see it and fix it.
Don’t let any of that scare you away from contacting us with your bug reports. If you want to just send an email that says ‘Hey, I think I found a problem…,’ that is fine. We take all reports seriously, but we can only do so much, and some bugs simply have to go on the back burner. The purpose of this page is to give you an insight on how to get your bug on the fast track to resolution.