How to Build a Software QA Process in Two Minutes

There are a variety of software development processes, opinions on the best way to build software, and differing points of view on how to even define both the different kinds and the various layers of software (social-enabled, web-enabled, client-server).

QA Managers, Project Managers, and Product organizations are faced with establishing and enforcing Quality guidelines to ensure they deliver a solid product.

How do you build a QA process in the middle of all of this mess?

There are many varieties of QA processes that a QA engineer or analyst can implement.  Most of them fall into three general categories:

  • Simple (linear – A therefore B) QA processes,
  • Complex (non-linear – A and B and C therefore D and E and F if G or H or J) QA processes, and
  • Complicated (A therefore B… most of the time) QA processes.

You might be tempted to think that the small companies implement the simple processes, and larger companies put in more well-stocked solutions.

In my experience, these different-sized processes for testing and preventing defects in software do not necessarily correlate to the size of the team or size of the company.  Both startups and multinational behemoths can have a bare cupboards or a fully-stocked Titanic of a QA process: both can succeed fantastically, and both can fail fantastically.  So perhaps the size and shape of the process isn’t linked to the budget of the firm.  Perhaps the three general categories of QA process matter less than the reasons why the process was implemented.

If you ask an online QA bulletin board forum what the best QA process is, you will get something like 1 unique answer per post (not counting snarks, me-tooers, and WTFs).  You might compile 20 or 30 responses in a week, many of them insisting on the right way or the ridiculousness of the idea of another suggestion.  [Helpful hint: do not check the “Receive responses to this post by email”]

Why is there so much dissent even within the QA ranks as to what the best QA process is?  It’s because it’s a trick question: the answer to what is the question isn’t “what is the best QA process I should implement,” the better question to ask yourself is “how do I build a QA process for this team?”

Begin in the meta layer: begin thinking about thinking about your process.  Consider your management structure, your budget, your timelines, the size of your product, and how often you’ll be expected to deliver working software.

Think about how your approach might best integrate into the preferences and environment of your group, your company, and your own career path.  Consider the opinions and personalities on your team.  In order to test software and put in practices to prevent defects, at the most fundamental level you’ll be expected to answer one question: Is this tested?

Your QA process has a primary mission composed of two parts: answer the question “is this tested” for your stakeholders and be generating defects for your development team. Your QA process is broken if it’s not executing on both of these actions.

There is probably a fancy word describing the “2 minute, 2 week, 2 year” solution approach that I’m going to reference here; a colleague of mine uses this process to describe his approach to ascertaining software architectural solutions.  I use this approach to answer the question of how to set up a QA process.

2 Minute Solution

Your solution must have the ability to deliver a 2 minute testing response.  In 2 minutes, what can you do?  The most manual effort will be required here.  You’ll want views into your presentation, application logic, caching, and data layers of your application.  You’ll want views into your network topology.  You’ll want views into your customer portals – quite literally, the browsers they use on the platforms (Mac/PC) they use, or the specific hardware configurations your hybrid client-server solution supports.

  • Put those to-dos on your list of people to talk to and diagrams to understand so that you can lay out a step-by-step checklist of what you can check and how.
  • What systems do you need logins for?
  • What corporate policies must you adapt to in order to view the data you need to see?
  • Who do you need to get to know better in order to win their trust for permissions on their system?
  • What do you know you don’t know?
  • What don’t you know that you don’t know?
  • What don’t you need to know right now?
  • What do you believe is blocking you?

The 2 minute solution is your think-on-your-feet instant service response to any organization with a question about the quality of the software.  Think about what simple reports you will want to have on hand or be able to run very quickly.  These will probably be short checklists for each product.  Can you ping the box, how much disk IO and CPU is the box using, what is the transaction time between data input and calculation output, how much memory is the database using to perform a complex select statement, etc.

Create these checklists and bounce them off your architects and lead developers – they usually like to see that a QA manager or analyst is taking the initiative to learn the systems.  Their propensity to help you is usually generous.

2 Week Solution

You’ll use your 2 Minute Solution in your 2 Week Solution.  The 2 Week Solution is often not exactly two calendar weeks, but a metaphor to get you into the headspace of thinking about what you can do with a little planning ahead and listening to your successes and failures with your 2 Minute Solution.

The easiest implementation of the 2 Week metaphor is test automation.  The most straightforward implementation of a Q&D (quick and dirty, often implying a Graceful Hack as opposed to something resembling a TV Dinner) test solution is a straightforward open-source implementation of three basic tenets of software QA: a defect tracking system, an automated testing framework, and a performance testing system.

All of these can be set up in one week.  If you’re working for a major software vendor such as HP or IBM, you’ll have instant access to their Tier-1 QA tools such as Quality Center, QuickTest Professional, LoadRunner, Performance Center, or the Rational suite of QA tools.  Your 2 Week metaphor must result in your keen evaluation, installation, and basic configuration of these tools so that you can get some basic QA systems up and working.  They may be taken down in a matter of weeks because of politics, a purchase, management decision, or a change in product strategy.  This doesn’t matter – what matters most is that you show the initiative to show up to the game with the tools you need to play the game and that you use them.

Defect tracking: Pivotal Tracker or Bugzilla can be set up in one afternoon for defect tracking.  There are other tools available you can install on your own test machine, local box, or virtual machine (if you’re so lucky).  Talk with your system administrator if it’s okay if you install your own Linux system and virtual machines on your own box.

Automated framework: Download and install Firefox, Firebug, and the Selenium IDE for a basic automated web testing framework to get you started with a basic regression suite.  You can have 10-20 test scenarios recorded and played back in a browser in an afternoon.  Winrunner or QuickTest Professional may be available to you – if they are, find out who is using them.  Print out the help pages from these applications and read them over lunch or at home to ramp up on the local syntax of the tool and the test parameterization methods.  Have lunch with the test engineers who are using these tools if possible.  Hear their stories and frustrations and learn from them out of the gate.

Performance framework: If you can’t get Load Runner and you don’t have a Performance Center installation, install JMeter or an open-source performance tool in a language you’re comfortable with.  PyLot is a Python tool that provides some basic functionality.  The mechanize and multi-mechanize toolkits are available for Perl, Ruby, Java, C#, and Python.  Google these packages and try to get a simple POC up and running in the language of your product.  Remember, if you can adapt your tools to the language being used in your company, you’ll have resources around you who can help immediately.  Google is great for code snippets but there is no substitute for on the ground help with the syntax of a programming language.

Web link integrity: If you are testing a web application, the easiest win you can install here is Xenu’s Link Sleuth.  You’ll need a win32 or win64 operating system to run it, but it’ll be worth it.  This tool can check for broken links across your entire web site crawling as deep as you specify.  You can manipulate the output to view oversized pages, SEO violations of multiple URLs with the same content, and reveal your slowest loading pages on your origin servers.  You can reveal not only the broken pages but the click path that leads to them. Tune Link Sleuth to scan 20 threads at a time 3 levels deep: this is the setting most PCs can handle.  Increase the threads per second to play with your machine’s capability to keep up (you’ll begin to see timeouts – hit CTRL-R upon completion to retry.)

Set manageable goals for your 2 Week Solution and measure your own ability to hit them.  If you hit failure, ask yourself what you were missing that you simply didn’t know you were missing when you set the goal.  Figure out how to fill in that gap and set a new goal.  Be transparent with your failures to your line manager if he or she is open to that sort of discussion.  Maintain a bright attitude, always pursuing a “what are our options here” or a “this is my next step, what do you think” line of thinking with your line manager and colleagues.  Avoid blame at every turn: it will only weaken your approach and morale.

2 Year Solution

You’ll use your 2 week solution in your 2 Year Solution.  As you’ve probably figured out, this is also not an exact week by week outline of every activity you will carry out for 104 calendar weeks, but rather a brain expansion of being able to identify your team’s purpose and mission statement.  Think about where your QA team is delivering today and where you want them to be delivering at in 2 years.  Think about what exists today in your business and technology delivery environment that you do not like and want to change in 2 years.

Continue to identify deltas in:

  • staffing (QA, product, project, support, development)
  • attitudes (your own, your leadership, your team)
  • leadership styles (What is working? What isn’t?  What training may really benefit everyone?)
  • decision-making approaches
  • team working environment
  • deployment process
  • product management
  • defect management
  • business prioritization processes
  • defect triage
  • web technology
  • back end data services
  • APIs.

These are all places where QA has an opportunity to lead and guide.  Brainstorm these areas out in a checklist, and discuss with your line manager to ensure that your direction matches the direction of the company and of your technology group.  Match this QA plan to your own career goals and career evaluation so that you have a clear growth path for yourself.

Ensure your oxygen mask is fully snug, and then you’ll be able to help grow an amazing and effective QA team with your new QA process!  And all that starting with a 2 minute exercise.

1 comment

  1. Eric thanks for the article. I’ve been working on putting together a better QA process at our shop and one thing that really changed our overall process was not allowing the fixer of a bug to be the tester. That second set of eyes makes a big difference in the resolution of the bug.

Leave a Reply

Your email address will not be published. Required fields are marked *