OSS bug tracking systems all SUCK

How hard is it to write a bug tracking system that doesn’t suck? Quite hard it seems …

I spent ten minutes trying to work out how to enter a bug in a Flyspray-based BTS. And I gave up. I gave up entering a bug! Isn’t that, like, the most important single task that BTSes should enable me to do?

Trac is a pile of crap too. The enter a bug button is hidden away somewhere (it’s called “New ticket”), and the main interface seems to be arranged around producing arbitrarily chosen “reports”, none of which I have ever found to be in the slightest bit useful.

Mantis … urrrgghhh. OCaml upstream use this to track bugs and it’s horrible.

And .. Bugzilla. Oh Bugzilla. When I get my own starship, I hope it’ll be simpler to use (and faster) than Bugzilla.

I would encourage everyone to try out (proprietary) FogBUGZ for an example of a usable BTS. It’s proprietary software, but you can learn from it. You can learn that (a) most of your users won’t be BTS experts, and (b) you should encourage as much feedback from all your users as possible. That means doing usability tests and concentrating mainly on making it as easy to use as possible. It’s the most important thing for BTS authors to do. Clue: if your users cannot work out how to enter a bug, then you should go back to the drawing board.

27 Comments

Filed under Uncategorized

27 responses to “OSS bug tracking systems all SUCK

  1. The default set of reports in Trac is not expected to be useful, but just to serve as examples of what reports you can set up, the site admin is expected to set up the reports (IMHO a classic case of “dear user, please write the program for me” soft coding, but whatever). Sadly, most projects never bother customizing their reports (which should serve as a hint to the Trac project that they need to focus on sane defaults more than customizability). For a basic set of Trac reports that makes sense for most projects, see e.g. https://www.calcforge.org/trac/calcforgelp/report (try clicking on “SQL Query” for each report to see how I set them up).

  2. Richard, what can you say about redmine? From my PoV it’s much MUCH better ticket tracking solution than horrible beasts like bugzilla/trac/etc.

  3. I think that most OSS BTS around think about power-users and forget about simple reporters.

    I must had some other BTS:
    – debbugs: fully mail oriented, quite hard to use for beginners
    – roundup: probably the easiest, but quite close to trac

    I also worked with Serena Teamtrack (proprietary), and it is at least as complicated as bugzilla.

    Most of the time people writing BTS try to make it as flexible as possible because bugs workflow is not (yet) well defined. It depends on how you handle bugs and many other factors: do you have someone that dispatch bugs for you, are you working alone or not…

    I think, if someone is willing to create a BTS, then he must begin by avoiding flexibility and makes a single workflow mandatory. He can then create something easy to use.

  4. Oli

    Isn’t Launchpad open source now? I’ve no idea how easy it is to set up, but it does have a much cleaner interface than some of the older projects that refuse to reform.

    The silly thing is, all these projects just need one extra person: a user-experience/front-end web designer-come-developer. There are literally thousands of people they could induct (or even pay) to get a much better interface.

  5. I think, Richard can only respond than redmine SUCK (more or less), cause it also OSS🙂

  6. Ed Avis

    Have you looked at roundup? A Fedora package is available.

    • rich

      The problem is that I don’t have a choice over the BTS that <some random project> uses. So while I could install this for my projects, it’s all the other projects I report bugs to that I really care about.

  7. Not OSS either, but I had the impression the one on google code [1] was also a step in the right direction. Especially the use of labels which avoid huge forms with useless fields but still allows you to define your own bug organization structure. But I didn’t end using it, so I don’t know how it turns out in practice.

    Mantis, failing to look up (without being logged in) an issue I did enter myself says everything I think.

    Oh and you didn’t mention gforge. But this is beyond hope, the whole system is bloated by a bureaucracy that 99.999% projects don’t need.

    [1] http://code.google.com/p/support/wiki/IssueTracker

    • rich

      Yeah, I use google code’s BTS for my ocaml bitstring project, and it’s not too bad, mainly because it’s very simple.

      Still nowhere near as good as FogBUGZ I regret to say.

      GForge is just a whole bucket of suck. The horrible BTS is the least of their problems.

  8. Matt

    Rich, I’ve adapted RT http://bestpractical.com/rt/ to use for bug tracking with great success, albeit not ready out of the box. I believe the Perl project and/or CPAN bug tracking use it as well.

  9. Ray

    I’m quite fond of another commercial solution — JIRA, which we use at work.

    Redmine did look promising however.

  10. Debian’s BTS is easily the worst of the lot, since it provides no direct interaction whatsoever, forcing you to check the man page for the correct email command format for even the simplest changes to a bug. I recently had to send three or four emails to successfully retitle a bug, because Gmail insists on wrapping long lines and the bug daemon couldn’t deal with it.

    Launchpad was GPL’s last year and it’s easily the best OSS bug tracker I’ve seen (though I’ve yet to see it anywhere but launchpad.net). Obviously it doesn’t have the polish of a commercial tool…

    • foo

      Firstly, as an MUA, GMail = FAIL.

      For me, debbugs is a dream. I hate dealing with bug trackers other than Debian’s.

      Definitely agree about Launchpad, probably the best of the FLOSS clicky-clicky bug trackers.

    • Charles

      I strongly agree about Debian’s BTS. Other OS’es BTSes vary from good (Launchpad) to okay (Bugzilla), but Debian stands alone as being nearly unusable through a web browser. Having to dig through documentation to figure out the format for email just to add a tag to a ticket is insanity…

  11. The state of bug trackers quite a few years ago was even worse, which is why I wrote Anthill (http://freshmeat.net/projects/anthill) quite a few years ago (started it 10 years ago I think). I moved on to other things and Anthill is pretty much unusable now due to it being written for PHP3 IIRC, but it was pretty simplistic and easy to use back then.

  12. I don’t think Bugzilla is terrible, really, once you get used to it. It does have significant issues when used for a distribution, though, as it was designed to be used for a single software project. It and Launchpad are the two I cringe least upon seeing. Trac tends to be a nightmare, I agree.

    I think something that was a combination of the best ideas in BZ and Launchpad – and, of course, 10x faster – would be great. Go write it!🙂

    • rich

      Well it’s very slow, but the significant problem is the one you allude to just there: I don’t think Bugzilla is terrible, really, once you get used to it. You mustn’t discourage feedback about your project by having an unusable BTS. It must be simple and effortless to submit a bug.

      Many of those bugs submitted will be spam/crap, but that is our problem as package maintainers, not the problem of the volunteers testing and giving feedback.

  13. Your comment doesn’t seem reasoned enough to be taken into account, but I’d like to join your rant to say: https://bugzilla.redhat.com/ is slow as hell!

    Really, I don’t know if it’s a Buzgilla problem, but filling a bug in Fedora is PITA!

    I’ve used Mantis and it’s not that bad. I wouldn’t say Trac it’s a general BTS, because it’s focused on development and developers.

    • It’s due to the size of the component list. That’s not a problem with bugzilla itself (bugzilla itself is quite snappy, actually). I’m not an admin on there, so can’t even guess, but next time you’re in there, look at the product list. It’s huge. Then look at the component list for a given product (let’s say Fedora or RHEL). They also are huge. And I mean _really_ huge. Probably on the order of thousands of components per product (not all products, of course, but at least for RHEL and Fedora).

      So you’re looking at, overall, a few thousand components, maybe close to a hundred products (completely off the top of my head)… yeah, I’m not surprised that the Red Hat bugzilla is slow. In fact, I’m impressed that it isn’t _slower_, all things considered. =)

  14. You might want to try http://www.pivotaltracker.com/ .

    I did not try it, but I’ve seen it advertised as a BTS which purpose IS simplicity. Well, actually it’s not exactly a BTS but it is capable of tracking bugs :p

  15. Agile Maggie

    I am a big fan of hosted bug trackers that also have code repos, wikis, etc. It’s all setup and is regularly backed up so I can spend my time coding. Assembla is my fave.

  16. Bwah ha ha!

    I found your site through a quick search to see if the OSS world has sorted this out yet. Every year or so, I go through the same process.

    …Guess not.

    Thanks for a giggle anyway😉

  17. I wrote an article about this same frustration at “Bug Trackers: Do They Really All Suck?” (O’ReillyNet) http://lwn.net/Articles/163500
    They are are applications that are used by many kinds of people – developers, QA, project managers, the public, and this pulls them in many directions.

    I find that the tool I consult on (Atlassian’s JIRA) is flexible enough to make most people happy (80%-90%). It’s still good value for money ($10 to a few thousand for most companies). It’s not OSS but you get the source code at all license levels.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s