Strategies: Building QA in your organization

Let me level with you: there are a surprising number of reputable software companies out there that lack well-defined QA practices. Some dev shops may even admit to skipping out on formal QA, altogether. The reality is: if you're saying to yourself, "Everything we ship gets tested thoroughly - I sleep well at night." you're in the minority…and I salute you.

So where does that leave everyone else - people looking to attack the problem & elevate their QA game? I've discussed the subject with enough stakeholders at various companies that I thought I'd share a few key topics to consider, as you begin making meaningful progress.

Let's start out with this scenario: Let's say your company has been around a few years but lacks structured QA practices (not to say nothing gets tested, it's just ad-hoc & uncoordinated). Your team does follow Agile development conventions, though, so they're already in the habit of defining work by release cycles.

Good news - there's hope!

Here are some points to discuss internally. I recommend getting a variety stakeholders from all corners of the organization to weigh in. The divergent perspectives they offer will expose planning gaps & also act as an insurance policy toward arriving at a solution that works for everyone.

Goals (Get everyone to agree on these)

1. The need for QA should be understood and supported at all levels of the organization.
   (If not, tackle this before proceeding further.)

2. We must document everything we build, so we can transparently determine ways to test it.

3. We must document everything we test, and trace our tests back to the development goals.

4. We must document everything that breaks, and trace bug fixes back to the codebase.

5. We need management software (ideally an ALM* platform) to tie together points 2 - 4.

6. Testing is a collective exercise. Everyone can & should contribute.

7. We should not try to automate all the testing (in practice, this never works!).

*ALM stands for Application Lifecycle Management. Think of this as an integrated software platform to manage your product development info, code, testing, and releases: a one-stop-shop for all your product development needs.

Questions (To guide you in designing solutions)

These Q's are meant to asses maturity across a variety of dimensions. After each question, ask the follow up: "Can this be done better?"

1. Can our testing tools create clear mappings between tests and development work?

2. Do our testing tools allow us to create individual tests, as well as suites of tests, easily?

3. Do our testing tools allow us to manage both automated & manual tests?

4. How easy is it to distribute the testing workload? (across both people & machines)

5. How easy is it to aggregate and view test results?

6. Do the testing tools provide a UX that's truly relevant to all user roles?

7. How good are our analysis & reporting capabilities?

8. Do we have the capacity to identify gaps in test coverage and plug holes with each test cycle?

Repeat these questions for all products you deliver. Most companies have multiple products & code bases, and the answers for each can be very different. Use the answers to the previous questions to help you arrive at answers to the following:

1. What is the business value we place on improving the current state QA in our system?
   In $ / year. This is our QA budget.

2. What fraction of this budget are we comfortable applying towards technology?

3. What fraction of this budget are we comfortable applying towards personnel resources?

Roadmap (You implement QA like you eat an elephant)

1. Utilize an Application Lifecycle Management suite.

2. Have a workflow for tracking bugs by product & release.

3. Create a product-to-test coverage map.

4. Build out infrastructure to run automated software tests.

5. Define an automated* smoke test of core functionality that would create a showstopper, if broken. Place a cap on the runtime of this test of < 5 mins.

6. Create 100% automated test coverage for all new things you build in each sprint.

7. During sprint planning sessions, budget time to progressively increase automated test coverage of existing product functionality .

*Automated testing can mean many different things (Unit / API / UI testing, for starters). While detailing the flavors and how best to leverage them is beyond the scope of this post, for now lets just say: pick a lane. Ensure everything new you build achieves "coverage" by at least a respectable effort at a Unit/API/UI test case.

The Takeaway

My mantra is: "superior product quality is a competitive advantage." Good QA practices will make everyone's lives better by reducing the "self-inflicted" business wounds related to poor coordination, bottlenecks, process inefficiencies, not to mention embarrassing customer-facing gaffes. An experienced QA specialist will be able to tie together product complexities & organizational goals and customize a solution that fits your tech stack.

Note: This is part of a series on Enterprise IT maturity. Also see my post on Building DevOps in your organization.