QA – Creating a high performing QA function

Two years back I was asked to put together a presentation on how to create a high performing QA function for a large financial company who were considering an Agile transformation.

The presentation touches upon all aspects of Agile; process and practices. It covers continuous integration, continuous delivery, test automation, Scrum process, defect management, etc…

Though the presentation below does contain some specifics to the actual company, it does have a lot of reusable content that you may find useful when evaluating your own company QA function needs within your Agile implementation.



Recently I had the pleasure of working with Gareth Waterhouse, he has an indepth knowledge of testing within an Agile environment as well as a very strong knowledge of test automation and I would highly recommend visiting his blog for everything about QA within an Agile development team.

Quick, Intuitive, Simple Scrum Tools

Scrum WallI have had lots of people ask me which software I use when running their Scrum ; TFS, OnTime, Jira, etc. All have their merits, but my personal favorite, is a wall with different colour post-it notes and some black tap.

My problem with software tools are that they over complicate the process, not having a wall also means a lack of instant visibility to the PM and stakeholders, something I promote (I like the business walking up to the wall and seeing the flow of work. Having the wall also self-promotes Scrum.)

To supplement my Scrum wall I use a simple Excel spreadsheet which generates my burndown chart and tracks progress. I will post up my template tomorrow spreadsheet tomorrow.

Below is my current SCRUM wall, I use different colour cards to acknowledge different task types (Green – Product Backlog, Red – Developer Task, Yellow – Tester Task, Orange – Impediment);

Agile test curve

This post is an update to my original post Testing Curve. I’ve had a few emails asking for a more detailed graphic and some advise on avoiding the unhealthy test curve.

Agile Test Curve

Don’t allow test debt to build up in your Sprints. Start testing as early as possible. Tips on how to avoid the unhealthy test tidal wave;

As a Team

  • Have smaller size stories in your Sprint backlog. Never, ever play an Epic.
  • Use BDD to capture all test scenarios, this will help the tester test the right area’s as well as the edge cases.
  • Automate as much of your testing as possible, reduce the testers manual efforts.

As a Developer

  • Manage your WIP; complete a story before working on another so that you release a story for testing sooner rather than later.
  • Minimise bounce back in Sprint. Run unit tests to ensure that the testers are testing on test ready code.

As a Tester

  • Prepare your test environments at the beginning
  • Look at the team task board to see what is going to be ready next for testing and make sure all test cases are in place.

Pragmatism + Agile = an excuse for bad Agile?

Over the last few years I have heard a lot of people use two words more and more often together; ‘Pragmatic Agile’.

One of the most abused words in software development is the word ‘Agile’, the second most abused word is ‘pragmatic’, couple these together and you have a train crash ‘pragmatic Agile’ (this even trumps ‘Lean’ Six Sigma).

Often I see individuals or teams who apply ‘pragmatic’ views to Agile using it to justify their own reasons to hack the bits out they don’t like, often using a ‘pragmatic’ approach to leverage in traditional waterfall bits (upfront design, etc…)

Every time I’ve heard some one talk about pragmatic Agile its more often than not to justify the failures in their Agile model. It’s a great strap-line, particularly when you are trying to convince a business to go on an Agile transformation journey (a simple way to silence the skeptics in the room). Be weary of statements such as ‘we don’t do Sprint planning as we’ve taken a pragmatic view that we don’t need to breakdown our releases’, etc…

Selling Agile concepts is difficult; you are telling people that the way they have been delivering software is not perfect, this is a difficult pill to swallow. So all too often when going through the early stages of a transformation you will find that the ‘compromise’ is that the business won’t have to abandon certain ‘comfortable’ current practices (poor practices or not). You will find that some of the ‘pragmatic’ historic bad processes / practices have been merged together with the newly introduced Agile processes / practices (because everyone cannot see the problems with the existing process / practices as they we’re comfortable with them).

All to often a business will pick up on the Agile ‘buzz’ terminology and overlay it on top of their current delivery model.

A good example here at was a previous employer that believed the team had to produce a Sprint plan which consisted of all of Sprints being defined and backlog assigned before work began. Sprint planning is not a mechanism of defining all of the Sprints with the backlog distributed across the Sprints from beginning to end of the entire delivery life-cycle. Sprint plans will last only the duration of 1 or 2 Sprints (2 – 4 weeks) and are for the team to track how they are progressing within this small time-boxed window. Trying to create a Sprint plan that is from beginning to end of the entire delivery is waterfall, as you are planning every piece of work upfront and just putting ‘Sprints’ on top without getting any benefit from having Sprint plans. Why do we still have gantt chart end-to-end plans still? Because it was a comfortable process from the traditional model!

This is a recipe for disaster when adopting Agile; as it will appear that Agile is too blame when an existing pre-Agile ‘bad’ practices / processes fails (which it always was always going to do, but not the skeptics, they have an excuse that it as an Agile failing).

How do you quickly identify if your Agile model is struggling due to ‘pragmatism’ being applied? The moment you hear teams or management making up excuses and blaming it on Agile (EDD = Excuse Driven Development).

Is applying genuine pragmatism to Agile a bad thing? No, as long as it doesn’t become an excuse for not being genuinely Agile and if you find yourself sacrificing too many of the Agile principles or beliefs, ask the honest question, ‘do we really, really want to do this?’. Pragmatic Agile is when you take the best bits of one Agile methodology (i.e. continuous integration, pair programming, etc… from XP) and apply those practices to another Agile methodology (i.e. Scrum).

What a Scrum Master is and what they are not…

Over the years and the companies I have worked with, the most misunderstood role within the Agile sphere has been that of the Scrum Master. Out of the various companies I have worked for, only 2 have had a good grasp on what the role is.

To help clear up the definition of what a Scrum Master is, I thought I would list out what a Scrum Master is to the team and what they should not be;

What is a Scrum Master;

  • Facilitator / enabler
  • A referee of the Agile processes / practices
  • An impediment remover
  • A change agent / champion
  • Protector of the team
  • Motivator
  • Metrics owner
  • Coach – Its not just Agile Coaches who can coach. I am both; an Agile Coach and Scrum Master, I can do both and I feel that a good SM / AC are fore-filling both roles

What a Scrum Master is not;

  • Part of the team – This one is often a bone of contention. A Scrum Master is a slave for the team, if you hold them accountable for the team not meeting their commitment you will drive all the wrong behaviors. Your Scrum Master won’t want to take a kicking for a developer not cutting code or check-in code that broke the build on your CI environment, so they’ll start forgetting about being a Scrum Master and start cutting code, writing unit tests, etc… (I’m lucky that I was a developer, remember most Scrum Masters are NOT techies)
  • A team member wearing the ‘extra’ hat of being a Scrum Master – This applies pressure on the person if they have two conflicting needs; for example a developer needs to finish some code so the team meets their commitment but at the daily update a major impediment that also jeopardises the commitment needs resolving but the developer cannot manage to do both before the end of the Sprint.
  • Wearing both PO and SM hats – Self validation jeopardises the integrity of the SM role
  • Determining the priority stack – Being a product expert is a full time role
  • Responsible for the distribution of the backlog amongst the team – The team are responsible for selecting the backlog to play. If your Scrum Master distributes the work this dilutes the teams collective responsibility of delivery.
  • A project manager, development manager, team manager, delivery manager, release manager etc… - Though there have been times when I have had to lead, a Scrum Master should not be looked to as the ‘leader’ of the pack; if you notice the team look at the Scrum Master when giving their daily updates, try sitting behind them or deliberately not attending the daily Scrum every now and then.
  • Responsible for delivery – Often the one that messes with Senior Managers the most. Scrum Masters are there to assist the delivery, they are not the person who cuts or tests code, therefore they are not in control of the teams destiny. Scrum Masters need to be clear to the team that they will help how ever they can, but if the team fails to deliver its on their own heads be it.

The Testing Curve

Following on from my post, Testing in Scrum / Sprint, I wanted to share with you a diagram I have used many times to help explain why testing must start as soon as possible within a SPRINT and why it is important that task cards are testable as a self entity (Unit test).
It is incredibly important that teams do not have ‘Epic’ task card’s included within their SPRINT, as this possibly hinders the ability of a tester to start testing early, as the ‘Epic’ wouldn’t become testable until development is complete, hence delaying the test phase.

Note: I include an initial ‘peak’ to allow for testers writing up their test scripts

Agile Testing Curve

The unhealthy testing line has a tidal wave at the end of the SPRINT, unlike the nice calm waters of the healthy testing curve. Tidal waves should be avoided at all cost, as you run a greater risk of failing a SPRINT; the tester may not be able to complete all the testing within the time period and more than likely the QA will be compromised due to the pressure on the testers to pass a release as shippable.

If you see a very calm testing period at the beginning / middle of your SPRINT, be careful this may be the calm before the storm!

Agile is not an arms race

Over the past few years there appears to be an Arm’s Race within the Agile arena. Many people are preaching about advance techniques that will not only help improve your Agile model, but will save your Agile model (your marriage, your job, etc…).

One exec I worked with at a previous company was driven by identifying the ‘next big thing’ in Agile; so that he could preach about this from his podium, not because it was good for the teams or that his teams were doing the basics well, instead it was purely for the PR stunt; ‘We have successfully rolled out BDD across my teams, we have reduced test time by 50% because of ATDD’ etc…

I’m all for utilising any of these techniques as long as the team is actually doing the basics well. I worked with a team who had successfully implemented BDD using Cucumber, but they would often deliver nothing in a Sprint due to playing Epic stories (hats off to the chaps that did implement Cucumber, as the language they had to implement it with was not easy). Another team very interested in ATDD and BDD would frequently over committing (admittingly this has stopped and they are a great team). Is it really a good thing that these teams are trying to go more bleeding edge? I’d prefer they get their house in order before attempting to introduce new practices (unless the practices get’s them out of the sh*t).

My advice, instead of looking for the bragging rights of who is practicing the latest and greatest techniques, ensure you are doing the basics well, then through Pragmatic Continuous Improvement introduce more ‘advanced’ techniques. Mastering the basics will give you far more reward than running with a broken baseline and implementing advance techniques.

People often forget that Agile’s beauty is its simplicity, something I fear is often lost as more and more ‘old skol’ managers start using it. Like my previous post when I asked companies that claimed to be Agile but who were not (band wagon jumping) to stop, I also ask that companies who have implemented Agile stop trying to push on with new technique’s until the have mastered the basics.

Difference between a user story and a story

A user story defines the requirements of the business and is written by the PM along with a BA (if available / possibly the same person), the user story is then broken down into individual stories that can be delivered against, which in turn is broken down into individual tasks.

Sounds simple, but in practice it is often not achieved. If you don’t break down User Stories you’re always working against Epic scale stories. It is critical to break down user stories into multiple stories so that each story is deliverable within an iteration (or if using Kanban ensures it can fit into the swim lanes).
It is important to break down stories into tasks so that you have a clear vision of what needs to be achieved to deliver the story. Having tasks also allows us to understand during an iteration how much time is remaining. Story estimations are finger in the air stuff, in particular if you are using relative points as your measurement, task hour / day measurements helps us to understand the actual remaining time against the original story estimation. Long term this will help improve your team’s ability to better estimate both user stories and stories.

On a side note, even though acceptance criteria is only a must have for user stories and stories, I also encourage teams to have acceptance against a task. Doing this increases QA throughout the whole of the process.

 

ITIL, SLA’s and Agile

Three years ago I sat down for two days with a chap who used to be the IT Director for the Pru, one of the questions he asked was if we had SLA agreements (or as I prefer I(nternal) S(ervice) L(evel) A(greement) for SLA’s for departments within the same business) and I replied ‘kind of’, in one sentence he taught me a valuable lesson when he then responded ‘there’s no such thing; either you don’t or you do’, luckily I’m a quick learner so when the session continued with another question, ‘what percentage of delivery against SLA’s should you as a business be promising?’, I replied 100%.

If you go into a car garage and buy a new car you wouldn’t be happy if the salesmen said that the warranty was only valid 70-80% of the time if your engine blew up, so why do so many IT businesses guarantee only 80-99% SLA’s, why not aim for 100%.
All to often IT departments say that it isn’t possible to achieve 100% SLA’s, or they build in silly clauses. Why is it so difficult for IT departments to strive for 100% SLA’s, the Japanese train system is measured on failure by seconds (unlike the UK train companies which are on deemed to be late or early if they arrive or leave 10 minutes either side of the stated time!) and even based on these ultra tight they achieve 99+% against their SLA’s to millions of daily commuters.

I have spoken with many Agile experts over the years and it surprises me how many are unaware of ITIL, though my bread and butter (as well as my beliefs) is that Agile is the best way to develop software, I am a great believer in using the right parts from PRINCE2 and ITIL, in the same way we build our Agile model up using the best parts of Scrum, XP, Kanban, etc…. we should be open to using other non-Agile principles and judge them on their own merits, if it works for your business then why not embody  it and build it into your Agile model.