Go confidently knowing what good quality practices are
QA “Best Practices” are often thrown around by people who want them, need them, have them available to sell, or thoughtfully implement them based on a smart needs assessment. The title of this post is the goal.
There seem to be QA best practices that, in my experience of dropping in and cleaning up shop for various clients, are consistently tough. These best practices fall into four buckets: strategic process, writing tests, running tests, and entering and managing bugs.
QA Strategic Process
1. Monitor Jira (or whatever management software you choose) on a daily basis to be aware of changes in what is being built
2. Have a clear workflow that serves Dev, QA, and Product. I find something minimalist works better than complicated: “Committed”, “In Dev”, “In Test”, “Done”. Try to avoid having a status “Blocked”. Unblock yourself or escalate to the responsible party which creates a work ticket for them to track and for you to follow.
3. Work to build a reasonable understanding of how the SUT (system under test) works and how the current sprint will affect the system
4. Work to understand the tickets and efficient, valuable ways of testing them
5. Think in terms of functional positive/negative and non-functional baseline/scalability/load/
6. When possible, test on an iterative build that contains a full merge of all work ready for QA within the sprint, not a branch that contains only the change made.
7. Expected value is quantifying the value a decision will make above its anticipated cost. Probability is the reasonable chance of something happening. Impact is the depth of force of an event. Risk is the product of probability x impact. Think and communicate in terms of expected value and risk, not available time or people & technology resources.
1. Author tests into a central library that map to description of what the product needs to do
2. Seek to build MECE (mutually exclusive, collectively exhaustive) suites of tests that provide valuable certification of correct form and function
3. Suites contain tests that target common functions. Sets contain tests grouped for execution together, usually with a common personal responsible for ensuring the testing is completed.
4. Unit tests are automated code that runs with the build which exercises functional checks against methods written by the people who created the methods
5. QA tests are automated and manual checks that are run after build delivery which verify function, validate product integrity and user experience, and are written by professionals trained in critical thinking, root cause analysis, and technical communication.
6. A test has a name, description, unique ID, a test type, steps, and expected results. When a test is in a set, it has a pass value (Not run, Passed, Failed) and execution timestamp.
1. Create sets of tests for execution based on who is responsible for testing, what is being built, and what must be protected
2. Test sets should reflect the timelines on the test plan, responsible parties, and reflect the size of the test effort
3. Understand when you need stories delivered in order to test them on time and drive to get builds from dev to give you the time you need to test
4. Run tests in a central library that maps results to tests to product requirements and which keeps a record of every test execution
5. Figure out and communicate early the people, process, and technology needs you need to achieve your testing goals for the sprint
1. When testing and you have found strange behavior, stop. Dig around the bug to fully describe its shape and nature. Think about your configuration and limitations that may be causing the bug. Look into logs to trace the problem as deep as possible. Talk to the developer responsible about what it might be. Only if after doing all this the bug still looks like a bug, enter it in Jira.
2. Think about entering a bug like pulling the emergency stop cord on a manufacturing line. Your bug will require that product stops to read and review, and the developer stops what they are doing to read and review. Certainly stop manufacturing if you feel it should be stopped. But, understand there is an opportunity cost each time you do so. Be mindful.
3. If you enter a bug, name it well, perhaps using this format: “<this thing> <behaves like this> <under these conditions>”
4. When you write a bug description, include the test environment, the build ID, the TestID that found the bug, the versions of related information (e.g., browsers, applications), the experimental conditions including URLs or file paths, the behavior observed vs. expected, any questions that need to be answered, and how an acceptable fix would be measured and verified (if known.) Include a screen grab if it helps.
5. Differentiate between a bug (unexpected behavior as a result of correctly implemented stories) versus defects (coding mistakes) versus enhancements (future-looking opinions about product operation).
6. Be in sync on what priority is. I propose the following simple model using numbers:
* P0 – Blocked from testing, drop everything and fix it right now
* P1 – Broken code that must be fixed in this sprint
* P2 – Broken product that will be entered at the top of the backlog
* P3 – Request that may be a priority one day maybe/not
7. Recognize that using the above prioritization schema, most bugs raised in a sprint should be closed in the same sprint or next sprint.
8. If you find bugs or defects in a ticket In QA, do not enter a bug. Make a note on the ticket after discussing with the developer and send the ticket back in Jira to In Development.
9. Care about driving your bugs to closure. Be aware of what state they are in. Keep on top of them.
10. Use capitalization and abbreviations (e.g. FB=Facebook) in bug names. Use capitalization, proper grammar, and spell words out fully in the description.
11. Seek to independently verify functional correctness and validate that the nature of the changes make the product work better.