QA Checklists help drive developer excellence
Q: The role of QA is less to test out the bugs than figure out how bugs are introduced into the product and reduce that opportunity. How does QA contribute to removing those obstacles?
A colleague of mine once emphasized “you can’t test Quality in!” For some, it is sensible to move forward with decisions based on the notion that a longer work day for testers or throwing more offshore resources at a project at the tail end of a rush of development would somehow make the software better.
Yes, it is important to establish and govern a defect management system, manage test cases, and archive and communicate the nature of the test execution results. Yes, the ROI can be quite positive when an organized automated test framework, regression methodology, and business-process or product-feature-based test analysis is performed. In addition to these reactive methods, however, software development lifecycles require defect prevention techniques, especially as applied through Agile or XP philosophies which deliver working software more often. Defect prevention techniques are not a replacement for effects management, but rather are applied in concert.
When examining what a group is doing for defect prevention, I begin with a strategic checklist:
- Do we have clear roles and responsibilities of our positions? Are we able to map our job title to our role on the team? QA can often question and refine this at the VP/Dir level.
- Are there opportunities to define ownership for the strategic value of building code, architecting technical infrastructure, production support, new functionality testing, regression testing, defect process management, UAT process management, work ticket management, and work prioritization as mapped to a tactical project plan as mapped to strategic company goals? QA can often question and refine this at the VP, Dir, ProjM, and ProdM level.
- Do we understand what Test-Driven Development is? Are we doing it? Why? QA can work with the head of App Mgmt to define/refine/implement this. Ultimately development must own and lead this effort. QA can audit it.
- Do we have a consistent unit test strategy integrated into our build process? Unit testing keep development honest to their own standards. The tests provide the measurement of the standards. Development must innovate and support this.
- Do we have a deployment checklist for a smoketest based on which QA either accepts or rejects builds from development? Operations, Support, and development must co-author this. QA’s job is to spearhead and inspire the effort and audit the results by ensuring each step is followed –and works – in a staging or pre-production integration environment.
- Are we using source control and a well-thought-out branching strategy?
- Do we have test, branch, n+1, and staging environments established that can/will maintain parity with our deployment intensity?
- Do we have script- and repository-based deployment systems set up that automated code, configuration, data schema, and upgrade rollouts and rollbacks?
- Do we have strong application development technical leadership?
- Do we know why we are developing software the way we are and what the critical success factors (CSFs) are to make our software development lifecycle (SDLC) succeed?
- Have we followed Occam’s Razor to reduce our meetings and deliverables to the minimum required set in order to consistently deliver excellent software?
- Do we know what portions of the system is and is not covered by tests? Do we understand the nature and depth of these tests? Are we sure the tests that pass are testing the product correctly and effectively?
- Are QA tests, results, and automation code under revision control?
- Are automated QA tests and the automation infrastructure code reviewed?
Executing on this checklist is a delicate balance between getting your day to day work done and working in strategic adjustments that can build on the momentum the team has already built. Building change into an existing process that requires putting the brakes on should be strongly evaluated (often with at least two layers of management) before applying. The same goes for staff position elimination and hiring. Always take steps to motivate and audit your staff’s ability to produce before making hiring or firing changes – I find this evaluative process generally takes at least 2-3 months, especially for a new manager.
QA’s role in defect prevention is one of assertive leadership and mindful audit of current procedures. By applying experience and a checklist-style approach to evaluating and inspiring change, the front-loaded quality rating of software can be vastly optimized so that defects are minimized before the software is built.