Should Software Reviewers be Filing Bugs?

Before I joined Novell, I spent years writing software reviews — mostly, but not exclusively, about open source and free software projects. In that time, I ran into hundreds if not thousands of bugs. I might have filed 20 bugs in that time, and that’s being generous. A couple of reasons for that:

• When I first started reviewing software, I really wasn‚Äôt familiar with the process for filing bugs.
• Even when I became familiar with filing bugs, it was a different process for each project, and if it required creating a login or too many steps, I simply didn‚Äôt have the time to follow up.
• Sometimes the bugs were already known, so no point in filing a new bug. This was usually the case with really egregious bugs.
• Occasionally I would email the developers directly and assumed they were filing bugs‚Ķ at any rate, they were aware of the problem, and sometimes it would get fixed during the process of the review.
• I reviewed a lot of software and I didn‚Äôt get paid to file bugs, I got paid to turn in reviews. Taking time to file bugs (even 30 minutes) was time that I wasn‚Äôt earning.

But the big question to me is whether or not a software reviewer has a responsibility to actually file bugs. Occasionally I would get comments or emails asking/complaining “well, did you file a bug?” — sometimes as a “hey, help us out and file the bug” and sometimes as “how dare you ding us for something in a review instead of filing a bug?!”

As a part of a project, I often wish people would take the time to file bugs — lots of people will take the time to comment extensively on posts on news.opensuse.org about bugs they think they’ve found, but won’t bother to actually file the bugs. It might feel good to vent that way, and perhaps if someone else has the problem or is feeling kind, they’ll file the bug for the commenter, but usually that bug goes unreported (officially) and the same person wonders six months later “why hasn’t my issue been addressed?”

I’m not really sure of the answer. It’s clearly not what reviewers are being paid for, and I know a lot of people still doing reviews for a living — not sure they have the bandwidth to spend a few hours per week filing bugs in addition to testing software and writing about it. But it’s also good community citizenship to file bugs if you’re using the software. (Even better to go ahead and send a patch if you’re a developer, but that’s another story…)

Anyway, just something I’m pondering with my morning coffee…