QA – Creating a high performing QA function

Two years back I was asked to put together a presentation on how to create a high performing QA function for a large financial company who were considering an Agile transformation.

The presentation touches upon all aspects of Agile; process and practices. It covers continuous integration, continuous delivery, test automation, Scrum process, defect management, etc…

Though the presentation below does contain some specifics to the actual company, it does have a lot of reusable content that you may find useful when evaluating your own company QA function needs within your Agile implementation.



Recently I had the pleasure of working with Gareth Waterhouse, he has an indepth knowledge of testing within an Agile environment as well as a very strong knowledge of test automation and I would highly recommend visiting his blog for everything about QA within an Agile development team.

Agile ALM

Do you have an Agile Application Lifecycle Management plan?

Agile ALM

Agile ALM’s are good when taking waterfall centric teams on the Agile journey (ALM’s are not Agile specific, arguably they are waterfall in their nature). If you are converting a waterfall team, draw up a ALM, they’ll be able to relate to this very easily and quickly. I have a PowerPoint slide that has the high-level view above and then a transition smart graphic that covers each one in greater detail. Use each section to your advantage, highlighting Agile practices / benefits against each of the above.

 

Agile test curve

This post is an update to my original post Testing Curve. I’ve had a few emails asking for a more detailed graphic and some advise on avoiding the unhealthy test curve.

Agile Test Curve

Don’t allow test debt to build up in your Sprints. Start testing as early as possible. Tips on how to avoid the unhealthy test tidal wave;

As a Team

  • Have smaller size stories in your Sprint backlog. Never, ever play an Epic.
  • Use BDD to capture all test scenarios, this will help the tester test the right area’s as well as the edge cases.
  • Automate as much of your testing as possible, reduce the testers manual efforts.

As a Developer

  • Manage your WIP; complete a story before working on another so that you release a story for testing sooner rather than later.
  • Minimise bounce back in Sprint. Run unit tests to ensure that the testers are testing on test ready code.

As a Tester

  • Prepare your test environments at the beginning
  • Look at the team task board to see what is going to be ready next for testing and make sure all test cases are in place.

Double funnel look at Kanban

Double FunnelThough many have drawn up double funnel models, a few days back whilst talking with Dan Brown (aka Kanban Dan) about constraints and Kanban, we modified his double funnel diagram by adding in a vice with the initial applying forces of Command & Control and as these forces are applied the blame force grows exponentially.

So what does this double funnel represent?

The incoming funnel is the constraint the business places on the delivery (fixed scope, time and cost – even worse they may dictate the actual solution as apposed to presenting IT with the problem), the middle part is the teams attempting to run in an Agile fashion (iterations, test driven, etc…) and the outgoing funnel is the release train (rigid schedules, manual releases, over importance placed on documentation, etc…).

What affect does this have? Ultimately it festers a blame culture (without naming companies, I have seen this in two companies I have worked for). The business places unrealistic expectations on the IT function and the limited release mechanism limits their ability to get features to market (this creating an image of a slow IT function).
The business will complain that IT cannot meet its expectations and the release team will be Teflon and point the finger back at the delivery team (and often because they are doing butchered Agile they’ll say its the teams responsibility even though they place release constraints on the team; responsibility but side stepping accountability by the release team).

Initially the blame towards the delivery team will be small, then over time it will grow exponential until the point that before a team has even started working on a product / project there will be an unhealthy negativity towards them (if you hear rhetoric like; ‘well IT always let us down’, ‘we should consider outsourcing because they’ll be quicker and cheaper’, ‘because of your past failed deployment, your out of release credits’, etc….). This will ultimately lead to a high turn over of IT staff as they will feel undervalued.

How to avoid funnels? Read Theory of constraints and Kanban, both adopt production smoothing techniques (WIP, etc…), invest in continuous deployment / delivery, work on improving business collaboration, etc…

Have a go at drawing up your own double funnels, here is a template you can use (PSD format), double funnels aren’t just limited to requirements and release, there’s a number of ‘double funnels’ in a business, a good start would be to capture all of these and present them back to those creating the constraints and looking at ways to remove the bottlenecks.

Team Kaizens

Kaizen (改善), Japanese for “improvement”, or “change for the better” refers to philosophy or practices that focus upon continuous improvement

Strangely I’ve never blogged about something I’ve implemented in virtually every Agile environment I’ve worked in recently, something I call Team Kaizen’s. Kaizen’s are often associated to Kanban, but there’s no reason why you cannot use Kaizen’s in Scrum, XP, DSDM, etc… (most already have some form of CI method method).

Often I use Team Kaizen’s to replace traditional retrospectives (Team Kaizen’s can be implemented irrelevant of which Agile methodology you use).

Why delay waiting until the end of a time boxed period to raise possible improvements? Why wait to discuss these improvements? Why delay working on making improvements?

Why not encourage people to raise improvements at any given moment in time, and if feasible (I’ll clarify later), discuss and act upon a raised Kaizen.

Team Kaizen’s

Next to the teams physical task board, I mark out an area for the team to put their Kaizen’s. When one of the team thinks of an improvement (related to the project or team), they write down a brief note and put it up on the wall. Depending on the team there is two ways to deal with Kaizen’s raised.

  • 1) Daily update- Team review’s each new Kaizen after each team member has given their update. The team will then bin any Kaizen they collectively do not feel has any value add. For the remaining value add Kaizen’s, depending on your Agile methodology, you can;
    • Scrum – Flag it for conversation at the next Sprint planning session. At Sprint planning the team selects 1 or 2 of the flagged Kaizen’s and includes them as deliverables within your Sprint. Like Stories, Epic Kaizen’s are not allowed, if it cannot be completed within your time boxed period (1-4 weeks) then it needs decomposing.
    • Kanban – Associate a class of service and put into the backlog
  • 2) Agreed cadence – Review the Kaizens at an agreed cadence (end of Sprint or at an agreed reflection point in Kanban). The team would discuss all of the Kaizens on the board and select 2 to work on over the period of the next month. Again, like Scrum epic’s, the only rule I would put in place would be that the Kaizen/s chosen can be delivered before the next review period, otherwise you could end up with a Super Tanker Kaizen and that is more than likely outside of the scope of the project or team.

Optional extras

You can tweak when you carryout your Kaizen review, so additional triggers could be used exclusively or in addition;

  • Team Kaizen area is overflowing
  • Team has no more backlog

Replacing the retrospective or retrospective-lite

You could do team Kaizens and continue your full fat retrospectives, but with a little Lean thinking you could either reduce the time of your retrospectives greatly or do away with them completely.

One of my previous teams had a 15min retrospective at the Kaizen wall every 4 weeks (Kaban team), they would carry out their Kaizen review, discuss and select two Kaizens to work on and briefly discuss AOB. Done.

This worked well for the team as they were driven by getting deliverables out of the door, though some teams I work with have a more ‘emotional’ angle to their retrospective (like group therapy), in which case they still may want to have 10-20mins to talk about how they feel, draw pictures, etc… its not personally something I like, as I’d rather focus on identifying area’s for improvement, generate an action list based on these areas and get out of the room (teams who moan about to many meetings or don’t value / get anything from retrospectives often like Team Kaizens as they are focussed, constant and mitigates the need for retrospectives)

* Warning – If the team does do away with retrospectives, the Scrum Master needs to communicate that any sensitive Kaizen’s or other issues that may normally get brought up in a retrospective should be emailed to him and an ad-hoc meeting arrange to discuss.

Agile / Six Sigma / PRINCE2

This is a bottle top;

Coke Top

If your goal is to manufacture a million identical units with as few defects as possible, ie per 1m units I want to have only 4 defective units. This is were Six Sigma works, this is manufacturing identical units many times over.

This is a train station;

Train Station

This requires a lot of upfront design, this cannot be emergent. This is when PRINCE2 works, lots of upfront design and considerations for building begins. It is often built with scale considered, but limited extensibility. If extensions are needed this often invokes a new ‘project’

This is software;

Code

This is uniquely different every time. This is were you use Agile.

Software isn’t a constant repeated process nor does it need everything being designed upfront. Software is emergent, embraces the demand of change and is unique each time its cut.

For too many years people talk about factories and software, I’ve never liked the term factory, it makes things sound far to robotic which stifles innovation and most successful factories produce identical products each cycle (just ask a manufacturing company if they could do something bespoke per unit, and they tell you that the cost of tooling / re-tooling would make it cost prohibitive.)

Pragmatism + Agile = an excuse for bad Agile?

Over the last few years I have heard a lot of people use two words more and more often together; ‘Pragmatic Agile’.

One of the most abused words in software development is the word ‘Agile’, the second most abused word is ‘pragmatic’, couple these together and you have a train crash ‘pragmatic Agile’ (this even trumps ‘Lean’ Six Sigma).

Often I see individuals or teams who apply ‘pragmatic’ views to Agile using it to justify their own reasons to hack the bits out they don’t like, often using a ‘pragmatic’ approach to leverage in traditional waterfall bits (upfront design, etc…)

Every time I’ve heard some one talk about pragmatic Agile its more often than not to justify the failures in their Agile model. It’s a great strap-line, particularly when you are trying to convince a business to go on an Agile transformation journey (a simple way to silence the skeptics in the room). Be weary of statements such as ‘we don’t do Sprint planning as we’ve taken a pragmatic view that we don’t need to breakdown our releases’, etc…

Selling Agile concepts is difficult; you are telling people that the way they have been delivering software is not perfect, this is a difficult pill to swallow. So all too often when going through the early stages of a transformation you will find that the ‘compromise’ is that the business won’t have to abandon certain ‘comfortable’ current practices (poor practices or not). You will find that some of the ‘pragmatic’ historic bad processes / practices have been merged together with the newly introduced Agile processes / practices (because everyone cannot see the problems with the existing process / practices as they we’re comfortable with them).

All to often a business will pick up on the Agile ‘buzz’ terminology and overlay it on top of their current delivery model.

A good example here at was a previous employer that believed the team had to produce a Sprint plan which consisted of all of Sprints being defined and backlog assigned before work began. Sprint planning is not a mechanism of defining all of the Sprints with the backlog distributed across the Sprints from beginning to end of the entire delivery life-cycle. Sprint plans will last only the duration of 1 or 2 Sprints (2 – 4 weeks) and are for the team to track how they are progressing within this small time-boxed window. Trying to create a Sprint plan that is from beginning to end of the entire delivery is waterfall, as you are planning every piece of work upfront and just putting ‘Sprints’ on top without getting any benefit from having Sprint plans. Why do we still have gantt chart end-to-end plans still? Because it was a comfortable process from the traditional model!

This is a recipe for disaster when adopting Agile; as it will appear that Agile is too blame when an existing pre-Agile ‘bad’ practices / processes fails (which it always was always going to do, but not the skeptics, they have an excuse that it as an Agile failing).

How do you quickly identify if your Agile model is struggling due to ‘pragmatism’ being applied? The moment you hear teams or management making up excuses and blaming it on Agile (EDD = Excuse Driven Development).

Is applying genuine pragmatism to Agile a bad thing? No, as long as it doesn’t become an excuse for not being genuinely Agile and if you find yourself sacrificing too many of the Agile principles or beliefs, ask the honest question, ‘do we really, really want to do this?’. Pragmatic Agile is when you take the best bits of one Agile methodology (i.e. continuous integration, pair programming, etc… from XP) and apply those practices to another Agile methodology (i.e. Scrum).

What a Scrum Master is and what they are not…

Over the years and the companies I have worked with, the most misunderstood role within the Agile sphere has been that of the Scrum Master. Out of the various companies I have worked for, only 2 have had a good grasp on what the role is.

To help clear up the definition of what a Scrum Master is, I thought I would list out what a Scrum Master is to the team and what they should not be;

What is a Scrum Master;

  • Facilitator / enabler
  • A referee of the Agile processes / practices
  • An impediment remover
  • A change agent / champion
  • Protector of the team
  • Motivator
  • Metrics owner
  • Coach – Its not just Agile Coaches who can coach. I am both; an Agile Coach and Scrum Master, I can do both and I feel that a good SM / AC are fore-filling both roles

What a Scrum Master is not;

  • Part of the team – This one is often a bone of contention. A Scrum Master is a slave for the team, if you hold them accountable for the team not meeting their commitment you will drive all the wrong behaviors. Your Scrum Master won’t want to take a kicking for a developer not cutting code or check-in code that broke the build on your CI environment, so they’ll start forgetting about being a Scrum Master and start cutting code, writing unit tests, etc… (I’m lucky that I was a developer, remember most Scrum Masters are NOT techies)
  • A team member wearing the ‘extra’ hat of being a Scrum Master – This applies pressure on the person if they have two conflicting needs; for example a developer needs to finish some code so the team meets their commitment but at the daily update a major impediment that also jeopardises the commitment needs resolving but the developer cannot manage to do both before the end of the Sprint.
  • Wearing both PO and SM hats – Self validation jeopardises the integrity of the SM role
  • Determining the priority stack – Being a product expert is a full time role
  • Responsible for the distribution of the backlog amongst the team – The team are responsible for selecting the backlog to play. If your Scrum Master distributes the work this dilutes the teams collective responsibility of delivery.
  • A project manager, development manager, team manager, delivery manager, release manager etc… - Though there have been times when I have had to lead, a Scrum Master should not be looked to as the ‘leader’ of the pack; if you notice the team look at the Scrum Master when giving their daily updates, try sitting behind them or deliberately not attending the daily Scrum every now and then.
  • Responsible for delivery – Often the one that messes with Senior Managers the most. Scrum Masters are there to assist the delivery, they are not the person who cuts or tests code, therefore they are not in control of the teams destiny. Scrum Masters need to be clear to the team that they will help how ever they can, but if the team fails to deliver its on their own heads be it.

Minimal Viable Product / Minimal Marketable Feature Set

If you don’t have backlog with either Minimal Viable Product or Minimal Marketable Feature Set make it your top priority!

I don’t believe in hard fast rules within Agile, but defining a MVP or MMFS is something I insist on for all of my teams / backlog.

What is a MVP / MMFS?

When you create a backlog of stories you often find that every story needs to be delivered before the product can be delivered. Putting on my lean start up hat, start thinking about getting something to market as soon as possible so that your business can start to realise some of the ROI benefit. It isn’t the complete functionality of the product, but its enough to sell to your customer and then incrementally improve. For example mobile phone software has been doing this for years; early iOS and Windows Mobile 7 didn’t ship with copy and paste, simple functionality, but more importantly the OS’s allowed you to make phone calls. It wasn’t until later releases of these two OS’s that you could copy and paste.
There is a balancing act; getting your product to market too soon could be damaging to the customer experience if its buggy or you choose to deliver the wrong functionality early (and MVP / MMFS is no excuse for poorly tested products or for not engaging with your customer). Equally if you ‘over cook’ your MVP / MMFS you could find that you could have gotten to market fair sooner than you did and therefore have missed out on a greater ROI.

Take a look at the screen shot below. This shows some backlog where a MVP / MMFS has been defined and a line drawn to divide what is in and out of MVP / MMFS.

 

Is MVP / MMFS the same thing?

They can be. Often teams have either a MVP or MMFS, I like to have both, but it is dependent upon your backlog shape. For example I have a MVP defined for the Product Backlog, and a MMFS defined at a release level (I always have a preference for having releases that are shaped on delivering a feature). This gives you flexibility at both a high level and more detailed level.

How do I prioritise my Backlogs so I can draw a MVP / MMFS line?

I always use MoSCoW as a measurement for determining the stories which are core and which ones would be added value but not critical. Be careful when asking the business what is a MUST, as often they will say everything is a MUST, limit the amount of MUST’s (if you get really clever you can limit the amount of each category based on Story Sizes, but that’s another blog post).

Recently I was asked by the business if we could realise some of the benefit earlier rather than wait for the entire product to be delivered, ironically the same person who posed the question the following day told me nothing was negotiable! If nothing is negotiable then you cannot have a MVP or MMFS, its not feasible.

The Testing Curve

Following on from my post, Testing in Scrum / Sprint, I wanted to share with you a diagram I have used many times to help explain why testing must start as soon as possible within a SPRINT and why it is important that task cards are testable as a self entity (Unit test).
It is incredibly important that teams do not have ‘Epic’ task card’s included within their SPRINT, as this possibly hinders the ability of a tester to start testing early, as the ‘Epic’ wouldn’t become testable until development is complete, hence delaying the test phase.

Note: I include an initial ‘peak’ to allow for testers writing up their test scripts

Agile Testing Curve

The unhealthy testing line has a tidal wave at the end of the SPRINT, unlike the nice calm waters of the healthy testing curve. Tidal waves should be avoided at all cost, as you run a greater risk of failing a SPRINT; the tester may not be able to complete all the testing within the time period and more than likely the QA will be compromised due to the pressure on the testers to pass a release as shippable.

If you see a very calm testing period at the beginning / middle of your SPRINT, be careful this may be the calm before the storm!

Agile is not an arms race

Over the past few years there appears to be an Arm’s Race within the Agile arena. Many people are preaching about advance techniques that will not only help improve your Agile model, but will save your Agile model (your marriage, your job, etc…).

One exec I worked with at a previous company was driven by identifying the ‘next big thing’ in Agile; so that he could preach about this from his podium, not because it was good for the teams or that his teams were doing the basics well, instead it was purely for the PR stunt; ‘We have successfully rolled out BDD across my teams, we have reduced test time by 50% because of ATDD’ etc…

I’m all for utilising any of these techniques as long as the team is actually doing the basics well. I worked with a team who had successfully implemented BDD using Cucumber, but they would often deliver nothing in a Sprint due to playing Epic stories (hats off to the chaps that did implement Cucumber, as the language they had to implement it with was not easy). Another team very interested in ATDD and BDD would frequently over committing (admittingly this has stopped and they are a great team). Is it really a good thing that these teams are trying to go more bleeding edge? I’d prefer they get their house in order before attempting to introduce new practices (unless the practices get’s them out of the sh*t).

My advice, instead of looking for the bragging rights of who is practicing the latest and greatest techniques, ensure you are doing the basics well, then through Pragmatic Continuous Improvement introduce more ‘advanced’ techniques. Mastering the basics will give you far more reward than running with a broken baseline and implementing advance techniques.

People often forget that Agile’s beauty is its simplicity, something I fear is often lost as more and more ‘old skol’ managers start using it. Like my previous post when I asked companies that claimed to be Agile but who were not (band wagon jumping) to stop, I also ask that companies who have implemented Agile stop trying to push on with new technique’s until the have mastered the basics.