OpenGov Test Army: Taking Quality Engineering Beyond Research and Technology
The process of bug bash is a very common software testing practice where stakeholders get together to test and ensure the product meets the requirements. At OpenGov Inc, we do a bug-bash with small subset of users for every release. We found an opportunity to tune this popular process to better suit our quality engineering needs. If you answered – yes to any of the questions listed in the illustration below, you will find this blog useful.
In Q3 of 2019, we decided to improvise our bug bash approach. Given the number of rich set of features scheduled to be released, we recognized the need for a structured, large-scale, organized and systematic bug bash. To achieve this, we rolled out an an initiative called “OpenGov–Test Army” and invited OpenGov employees from various groups to participate in testing these new features. This process did not replace or minimize our standard quality engineering practices. This initiative complemented our existing software test pyramid. A corollary benefit that we wanted to achieve with this was to organically train our “front line staff”, such as in tech support, customer success, and sales, on the new features and functionality.
To align on this process, we introduced a few new terms:
Test Army: Volunteers from various departments in the company who choose to be involved in this initiative.
Squad: The test army volunteers naturally fall into groups based on their areas of expertise. Each of these groups forms a squad.
Swarm: Everyone in a squad focuses on testing the same feature-set at the same time on the same test environment—working towards the common goal of ensuring a better customer experience. This process of collective testing is called swarming. The action performed by a squad is called “swarm”.
Wave: Collective representation of all the processes involved in organizing swarms is called a “wave”. One wave of the test army could involve more than one squad swarming.
Squad Owners: Usually the first line of support for the test army participants. Product managers, quality leads, and tech leads are prime candidates to be designated as squad owners. Squad owners should possess a clear understanding of the squad’s features, and hence, they are expected to help with questions that might arise during swarming.
Swarm Document: A document used for collaborating with each other during swarming. This document is used extensively during post wave assessment.
We were successful in planning and executing 5 waves (one swarm/week). We used the following template to capture all of the information. Since we were able to provide visibility into all planned waves ahead of time, participants were able to register for specific waves. We published two checklists—one for regression and another for new features that were being swarmed. These checklists were intended to provide a head start to our participants so that they could get a productive head start to this process.
The results of this initiative have been very impactful. We uncovered a number of bugs in each wave that were fixed before release.
We also generated a number of knowledge base articles to empower our customer facing teams. Engineering and customer-facing team collaboration has become the organic new norm. Teams are naturally communicating with each other in all roles.
This collaboration across the organizations continues to feed into OpenGov’s #CustomerFirst and #OneTeam culture.
If you would like to read more from OpenGov’s engineering team, read the rest of our Technology blog posts.
Interested in contributing to OpenGov’s Engineering culture of innovation, leading-edge technology adoption, and quality? Check out our current job openings.
Category: Technology