Prioritisation in Quality Assurance — How We Do It

Let’s say that a QA Engineer has found five bugs while testing one feature. How to continue? What to tackle first? Sometimes it can be overwhelming to add a few extra bugs to the pile of features you have to deliver – especially when you’re working in sprints or there’s a release just around the corner. 

Luckily, different levels of prioritisation can help you decide what to tackle first.

5 types of prioritisation at COBE

Defining different levels of priority might be a challenge for a newly set up QA team since the internet is full of different, and often contradicting information. When we did our research, practically every page we’ve opened had its own definitions and explanations. That’s when we decided to develop our own categorisation, explained in the following text.

By analysing the literature and information found online, these two terms came up: priority and severity. The definitions online state that:

• Severity stands for the impact a bug has on the apps’ functionality 
• Priority defines the bug-fixing order

In our team, we use the combination of both and call it prioritisation. 

All bugs found in our projects are split into 5 categories: Trivial, Minor, Major, Critical, and Blocker. 

The definition of each type is based on the way a bug impacts the functionality of the app. Let’s plunge in:

1. TRIVIAL

Definition: A trivial bug doesn’t affect the functionality, progress, or the user’s usage of the app.

Example: The Login screen contains the Login button, but the text on the button is misspelt as ‘Logun’. 

Explanation: Yes, this is definitely a bug. But it doesn’t affect the functionality of the app. The users can still log in to the app which is the main purpose of this feature. This misspelling doesn’t affect the users’ understanding of the usage of the button.

This type of bug is the lowest on the priority list. It’s going to be fixed if there’s enough time, or in the polishing phase of the app.

2. MINOR

Definition: A minor bug has the potential to affect the progress of the app development. A part of a feature is malfunctioning or is completely unable to function. There’s a simple workaround to avoid this behaviour.

Example: There's a disabled button in the application, and a tooltip with an explanation of why the button is disabled is missing.

Explanation: As a user, you’re able to use the app without any problems. It won’t be a trivial bug as it can be pretty annoying if the user doesn’t understand why the button is disabled. 

These bugs should be resolved only after all other, more serious, bugs are fixed.

3. MAJOR

Definition: There’s a workaround but it’s hard to go through it. There’s also a possibility that this bug will affect the progress.

Example: One of the features in the app is deleting an item on the list. The item can be deleted from the list, but it can also be deleted from the item’s details page. Now, let’s say that deleting it from the list isn’t working.

Explanation: This is a major bug because one of the features isn’t working at all so it’s affecting the progress. But, because there’s more than one way of deleting the item, the user has a workaround and they can still perform the action.

They aren’t showstoppers, because there’s a way for a user to finish an action, but these bugs should be fixed promptly as they aren't harmless.

4. CRITICAL

Definition: This bug affects the usage of the app and there’s no workaround. It’s a showstopper, which means the functionality cannot be delivered unless the bug is resolved.

Example: There’s a contact form in the app. The user is able to send their data in order to apply for a job. After fulfilling the whole form, he clicked on the submit button, but the form wasn’t sent to the employer. 

Explanation: The main function of this feature isn’t working. The user can fill out everything (that’s why it’s not a blocker) but isn’t aware that the form has never been sent.

If the bug has the label “critical”, it means that something is broken. Usually, the feature is unusable, so this should be fixed immediately.

5. BLOCKER

Definition: The entire app or some of its features are broken and aren’t usable. The users or testers are blocked.

Example: One of the features in the application is ‘Edit profile’ where the user can change their data. However, when they tap on the edit button, nothing happens.

Explanation: As it’s written in the definition, we’re blocked. In the edit profile section, there can be more than one feature. For example, a user can edit their name, email, or image. All of these features cannot be tested because there’s no way for a user to access them.

Same as a critical bug, a blocker should be fixed immediately.

Please note! There are circumstances where the same bug can move up or down on the priority list. For example, the trivial bug we mentioned earlier  (with the misspelt word “Logun”) is only trivial during the production phase. But if the team is one day away from a release point, this bug becomes more important as it's not good practice to go live with a mistake in the first version of the app.

Two more factors affecting priority

It’s all about context. If you're building an app where the most important thing is the look and feel of the app, then the UI bugs will have the highest priority. If that’s the case, a one-pixel difference, which would usually be a trivial bug, can move up on the priority list and become major. In other cases, it will stay trivial. The other factor that affects the priority of a bug is the personal opinion of the tester who’s reporting the bug. The difference between a trivial and a minor bug, or a critical and a blocker is not that big. The noted examples above are pretty straightforward, but sometimes the difference won't be that obvious and the priority will depend on the opinion of the person reporting it.

We didn’t forget about severity, but…

The definition online states that severity stands for the impact that the bug has on the functionality of the app, and the priority defines the bug-fixing order. In our team, we use the combination of the two and call it prioritisation. Our projects are mostly done using Scrum or Kanban. We divide the implementation into two weeks of iterations, with a mini-release after each of them.

Therefore, we’re dealing with one set of bugs for one iteration. We go in with the mindset that most of the bugs need to be fixed anyway. But first, the team needs to deal with the bugs that have the highest prioritisation, and if there’s time, they’re going to fix the rest. At this point, labeling both severity and priority would be an overhead.

How do you handle priority in your company? Do you have the same categories as we do? Feel free to share your experience, I'd like to hear from you!