QA 101 - How to Write a Bug Report
The purpose of a bug report is to enable an engineer to fix the bug. In companies with no dedicated testing/QA resources, bug reports get generated by many different people. Product managers may encounter a bug when validating a new release. Customer support may be talking to a user who has encountered a bug. Bugs may get discovered by other engineers.
Career QA engineers develop the art of intuiting what is causing a bug, and hone the craft of writing the most actionable possible bug reports. The rest of us have to muddle though.
Your Friendly Neighborhood Engineer
Fixing a bug is typically only a small portion of bug squashing time. If you spent 100 units of time on a bug, 90 units will be spent trying to make the bug happen in your development environment, 9 units will be spent finding where the problem is in the code, and 1 unit will be spent fixing it. This is of course a generalization; in a small number of cases once you find the code that’s responsible for the bug, you realize that it will be a huge amount of work to fix.
Engineers are inundated by bugs. Any software project of any size will eventually end up with hundreds, maybe thousands of known bugs. The ones that get fixed tend to be reproducible and prioritized.
Reproducible
Being able to make a bug happen on demand, and writing down the steps to make it happen, is called having a reproducible. It’s the difference between saying “a user said something happened to them”, or “something happened to me” and “I can make this happen” or “I can show you”. Understand that for the engineer, the most important thing is being able to make it happen, ideally on their own environment.
The quality of a bug report from worst to best goes:
- It’s broken
- This particular thing is broken
- This particular thing is broken, but only when I do this.
- This particular thing is broken, but only when I do this. I expect this to happen instead.
- This particular thing is broken, but only when I do this. I expect this to happen instead. It also happens on staging.
The zen of bug reporting is whittling the reproducible case down to the simplest set of steps that still trigger the issue, while not omitting anything critical. Maybe this bug does not happen for all users. Maybe it’s just new users. Maybe it’s just new users with a certain status. If you get that specific, it’s very likely that the engineer will know what the problem is off the top of their head. It’s also possible that you can come up with a work-around on your own!
What the engineer really needs is to make the same thing happen in their development environment. The challenge is that data is different in production and staging. But if you can reproduce the same bug in both production and staging, which also have different data, they you can probably explain how to reproduce it in development.
Not every bug can be reproduced. Or rather, very occasionally, the step of steps to reproduce is so esoteric that we cannot reasonably divine them. For example, this bug only happens on Tuesdays to users in Chile, when there are at least 100 other users in the system. But I can’t stress enough that this is NOT the typical case. 99% of the time, you should be able to reproduce a bug. After all, if you cannot reproduce a bug, how will we know whether we have fixed it?
Information to Include
- Facts (make sure they are right)
- Speculation (this is OK, but label it as not a fact)
- What you have already tried that did not work
- Error messages
- User names, record IDs
- Screen shots (not Word documents, please!)
- Stack traces
Pro-tip
If you’re testing a web-app, you should know how to view the development console. This can often reveal stack traces or at least errors messages when stuff goes wrong. This is bug report gold!
In Chrome:
- Go to View -> Developer -> Developer Tools
- Open the “Console” tab
- Reproduce the bug. You may see a red error like this:
