QA – Creating a high performing QA function

Two years back I was asked to put together a presentation on how to create a high performing QA function for a large financial company who were considering an Agile transformation.

The presentation touches upon all aspects of Agile; process and practices. It covers continuous integration, continuous delivery, test automation, Scrum process, defect management, etc…

Though the presentation below does contain some specifics to the actual company, it does have a lot of reusable content that you may find useful when evaluating your own company QA function needs within your Agile implementation.



Recently I had the pleasure of working with Gareth Waterhouse, he has an indepth knowledge of testing within an Agile environment as well as a very strong knowledge of test automation and I would highly recommend visiting his blog for everything about QA within an Agile development team.

On to Dunhelm, left Asos…

After 6 months of Asos, I move onto pastures new. I hoped to find a team like the one I had at Figleaves, which was an incredibly tall order, but I was lucky to find one that rivaled it.

I will miss team Drago and my beloved London, but I’m looking forward to having an easy commute for the first time in 4-5 years and the challenge that Dunhelm will provide.

Hats off to you team Drago.

Quick, Intuitive, Simple Scrum Tools

Scrum WallI have had lots of people ask me which software I use when running their Scrum ; TFS, OnTime, Jira, etc. All have their merits, but my personal favorite, is a wall with different colour post-it notes and some black tap.

My problem with software tools are that they over complicate the process, not having a wall also means a lack of instant visibility to the PM and stakeholders, something I promote (I like the business walking up to the wall and seeing the flow of work. Having the wall also self-promotes Scrum.)

To supplement my Scrum wall I use a simple Excel spreadsheet which generates my burndown chart and tracks progress. I will post up my template tomorrow spreadsheet tomorrow.

Below is my current SCRUM wall, I use different colour cards to acknowledge different task types (Green – Product Backlog, Red – Developer Task, Yellow – Tester Task, Orange – Impediment);

QA – Reap the QA rewards by reducing defects

Reducing software defects has huge rewards for a business. I recently put together a slide on the commercial benefit of reducing defects through sound QA practices, to quantify this as a cost to the business I put together a simple demonstration on the cost to the business.

Base

  • £58 p/d
  • 10 incoming defects p/w
  • 2 days per defect to fix

Defect Cost

If you reduce the total incoming defects by 40% in the first year you’ll save just over £22k, but it is completely feasible to achieve a 99% reduction in your incoming defects and your saving would be £55k p/a.

Any managers reading this will be over the moon, but the developers / QA and release managers will be scratching their heads @ how to achieve this. Defects only occur if you allow them to escape in the first place, look at your testing strategy, automate as much of your testing as possible, achieve continuous integration and aim for continuous delivery. Achieving a 99% reduction would be easier in some environments than others; i.e. a web application is going to be far easier to drastically reduce the incoming defects than a legacy installed application on the desktop (as you may not have the capability to upgrade / update desktop applications as frequently).

Agile ALM

Do you have an Agile Application Lifecycle Management plan?

Agile ALM

Agile ALM’s are good when taking waterfall centric teams on the Agile journey (ALM’s are not Agile specific, arguably they are waterfall in their nature). If you are converting a waterfall team, draw up a ALM, they’ll be able to relate to this very easily and quickly. I have a PowerPoint slide that has the high-level view above and then a transition smart graphic that covers each one in greater detail. Use each section to your advantage, highlighting Agile practices / benefits against each of the above.

 

Agile test curve

This post is an update to my original post Testing Curve. I’ve had a few emails asking for a more detailed graphic and some advise on avoiding the unhealthy test curve.

Agile Test Curve

Don’t allow test debt to build up in your Sprints. Start testing as early as possible. Tips on how to avoid the unhealthy test tidal wave;

As a Team

  • Have smaller size stories in your Sprint backlog. Never, ever play an Epic.
  • Use BDD to capture all test scenarios, this will help the tester test the right area’s as well as the edge cases.
  • Automate as much of your testing as possible, reduce the testers manual efforts.

As a Developer

  • Manage your WIP; complete a story before working on another so that you release a story for testing sooner rather than later.
  • Minimise bounce back in Sprint. Run unit tests to ensure that the testers are testing on test ready code.

As a Tester

  • Prepare your test environments at the beginning
  • Look at the team task board to see what is going to be ready next for testing and make sure all test cases are in place.

Blogging about football

Many people have asked how and why I choose my career; many of the techniques and my approach comes from sport (player, coach and spectator).

Anyone who knows me knows of my love of football (soccer for my American friends) and that I put in a lot of effort ferrying the little fella around to games.

In honor of my father, who I wish was here to coach my son, I’m going to start using my blog to also post about coaching techniques he taught me whilst we ran a team in deep south London many, many years ago.

Nokia Lumia 920

So after waiting for the Orange system to update the EE system, I finally recieved my new mobile yesterday, a Nokia Lumia 920.

Early impressions are very good, I’m still kicking myself for the terrible HTC Titan I’ve been carrying around for the last year. I did have a choice of a iPhone 5, but I’m a big fan of mobile windows, its massively better than the dull, un-interactive iOS and is a damn sight better than the beta-looking mess of Andriod. Though it should be noted I have an iPad for playing games, as the Win mobile app store isn’t exactly blessed with many triple A games or apps.

The Phone is very well built, Nokia apps are very cool, great screen, wireless charging works great and but best of all, and the reason I moved to ee’s 4G network, is that in the remote village I live in no one gets a reception, other than my other-halves ee 4G iPhone and a 3 Mifi dongle, as I’m writing this not only did I get a reception, but more notably I got a genuine 4G signal. Happy days from a remote village in Leicestershire.

Update

Very impressed with not only my phone, by the EE 4G network, after attaining 10Mbp/s in my remote village, I achieved 21Mbp/s at the Asos office (foot of Camden) in London today;

Speed Test Results

Double funnel look at Kanban

Double FunnelThough many have drawn up double funnel models, a few days back whilst talking with Dan Brown (aka Kanban Dan) about constraints and Kanban, we modified his double funnel diagram by adding in a vice with the initial applying forces of Command & Control and as these forces are applied the blame force grows exponentially.

So what does this double funnel represent?

The incoming funnel is the constraint the business places on the delivery (fixed scope, time and cost – even worse they may dictate the actual solution as apposed to presenting IT with the problem), the middle part is the teams attempting to run in an Agile fashion (iterations, test driven, etc…) and the outgoing funnel is the release train (rigid schedules, manual releases, over importance placed on documentation, etc…).

What affect does this have? Ultimately it festers a blame culture (without naming companies, I have seen this in two companies I have worked for). The business places unrealistic expectations on the IT function and the limited release mechanism limits their ability to get features to market (this creating an image of a slow IT function).
The business will complain that IT cannot meet its expectations and the release team will be Teflon and point the finger back at the delivery team (and often because they are doing butchered Agile they’ll say its the teams responsibility even though they place release constraints on the team; responsibility but side stepping accountability by the release team).

Initially the blame towards the delivery team will be small, then over time it will grow exponential until the point that before a team has even started working on a product / project there will be an unhealthy negativity towards them (if you hear rhetoric like; ‘well IT always let us down’, ‘we should consider outsourcing because they’ll be quicker and cheaper’, ‘because of your past failed deployment, your out of release credits’, etc….). This will ultimately lead to a high turn over of IT staff as they will feel undervalued.

How to avoid funnels? Read Theory of constraints and Kanban, both adopt production smoothing techniques (WIP, etc…), invest in continuous deployment / delivery, work on improving business collaboration, etc…

Have a go at drawing up your own double funnels, here is a template you can use (PSD format), double funnels aren’t just limited to requirements and release, there’s a number of ‘double funnels’ in a business, a good start would be to capture all of these and present them back to those creating the constraints and looking at ways to remove the bottlenecks.

Team Kaizens

Kaizen (改善), Japanese for “improvement”, or “change for the better” refers to philosophy or practices that focus upon continuous improvement

Strangely I’ve never blogged about something I’ve implemented in virtually every Agile environment I’ve worked in recently, something I call Team Kaizen’s. Kaizen’s are often associated to Kanban, but there’s no reason why you cannot use Kaizen’s in Scrum, XP, DSDM, etc… (most already have some form of CI method method).

Often I use Team Kaizen’s to replace traditional retrospectives (Team Kaizen’s can be implemented irrelevant of which Agile methodology you use).

Why delay waiting until the end of a time boxed period to raise possible improvements? Why wait to discuss these improvements? Why delay working on making improvements?

Why not encourage people to raise improvements at any given moment in time, and if feasible (I’ll clarify later), discuss and act upon a raised Kaizen.

Team Kaizen’s

Next to the teams physical task board, I mark out an area for the team to put their Kaizen’s. When one of the team thinks of an improvement (related to the project or team), they write down a brief note and put it up on the wall. Depending on the team there is two ways to deal with Kaizen’s raised.

  • 1) Daily update- Team review’s each new Kaizen after each team member has given their update. The team will then bin any Kaizen they collectively do not feel has any value add. For the remaining value add Kaizen’s, depending on your Agile methodology, you can;
    • Scrum – Flag it for conversation at the next Sprint planning session. At Sprint planning the team selects 1 or 2 of the flagged Kaizen’s and includes them as deliverables within your Sprint. Like Stories, Epic Kaizen’s are not allowed, if it cannot be completed within your time boxed period (1-4 weeks) then it needs decomposing.
    • Kanban – Associate a class of service and put into the backlog
  • 2) Agreed cadence – Review the Kaizens at an agreed cadence (end of Sprint or at an agreed reflection point in Kanban). The team would discuss all of the Kaizens on the board and select 2 to work on over the period of the next month. Again, like Scrum epic’s, the only rule I would put in place would be that the Kaizen/s chosen can be delivered before the next review period, otherwise you could end up with a Super Tanker Kaizen and that is more than likely outside of the scope of the project or team.

Optional extras

You can tweak when you carryout your Kaizen review, so additional triggers could be used exclusively or in addition;

  • Team Kaizen area is overflowing
  • Team has no more backlog

Replacing the retrospective or retrospective-lite

You could do team Kaizens and continue your full fat retrospectives, but with a little Lean thinking you could either reduce the time of your retrospectives greatly or do away with them completely.

One of my previous teams had a 15min retrospective at the Kaizen wall every 4 weeks (Kaban team), they would carry out their Kaizen review, discuss and select two Kaizens to work on and briefly discuss AOB. Done.

This worked well for the team as they were driven by getting deliverables out of the door, though some teams I work with have a more ‘emotional’ angle to their retrospective (like group therapy), in which case they still may want to have 10-20mins to talk about how they feel, draw pictures, etc… its not personally something I like, as I’d rather focus on identifying area’s for improvement, generate an action list based on these areas and get out of the room (teams who moan about to many meetings or don’t value / get anything from retrospectives often like Team Kaizens as they are focussed, constant and mitigates the need for retrospectives)

* Warning – If the team does do away with retrospectives, the Scrum Master needs to communicate that any sensitive Kaizen’s or other issues that may normally get brought up in a retrospective should be emailed to him and an ad-hoc meeting arrange to discuss.

Pragmatism + Agile = an excuse for bad Agile?

Over the last few years I have heard a lot of people use two words more and more often together; ‘Pragmatic Agile’.

One of the most abused words in software development is the word ‘Agile’, the second most abused word is ‘pragmatic’, couple these together and you have a train crash ‘pragmatic Agile’ (this even trumps ‘Lean’ Six Sigma).

Often I see individuals or teams who apply ‘pragmatic’ views to Agile using it to justify their own reasons to hack the bits out they don’t like, often using a ‘pragmatic’ approach to leverage in traditional waterfall bits (upfront design, etc…)

Every time I’ve heard some one talk about pragmatic Agile its more often than not to justify the failures in their Agile model. It’s a great strap-line, particularly when you are trying to convince a business to go on an Agile transformation journey (a simple way to silence the skeptics in the room). Be weary of statements such as ‘we don’t do Sprint planning as we’ve taken a pragmatic view that we don’t need to breakdown our releases’, etc…

Selling Agile concepts is difficult; you are telling people that the way they have been delivering software is not perfect, this is a difficult pill to swallow. So all too often when going through the early stages of a transformation you will find that the ‘compromise’ is that the business won’t have to abandon certain ‘comfortable’ current practices (poor practices or not). You will find that some of the ‘pragmatic’ historic bad processes / practices have been merged together with the newly introduced Agile processes / practices (because everyone cannot see the problems with the existing process / practices as they we’re comfortable with them).

All to often a business will pick up on the Agile ‘buzz’ terminology and overlay it on top of their current delivery model.

A good example here at was a previous employer that believed the team had to produce a Sprint plan which consisted of all of Sprints being defined and backlog assigned before work began. Sprint planning is not a mechanism of defining all of the Sprints with the backlog distributed across the Sprints from beginning to end of the entire delivery life-cycle. Sprint plans will last only the duration of 1 or 2 Sprints (2 – 4 weeks) and are for the team to track how they are progressing within this small time-boxed window. Trying to create a Sprint plan that is from beginning to end of the entire delivery is waterfall, as you are planning every piece of work upfront and just putting ‘Sprints’ on top without getting any benefit from having Sprint plans. Why do we still have gantt chart end-to-end plans still? Because it was a comfortable process from the traditional model!

This is a recipe for disaster when adopting Agile; as it will appear that Agile is too blame when an existing pre-Agile ‘bad’ practices / processes fails (which it always was always going to do, but not the skeptics, they have an excuse that it as an Agile failing).

How do you quickly identify if your Agile model is struggling due to ‘pragmatism’ being applied? The moment you hear teams or management making up excuses and blaming it on Agile (EDD = Excuse Driven Development).

Is applying genuine pragmatism to Agile a bad thing? No, as long as it doesn’t become an excuse for not being genuinely Agile and if you find yourself sacrificing too many of the Agile principles or beliefs, ask the honest question, ‘do we really, really want to do this?’. Pragmatic Agile is when you take the best bits of one Agile methodology (i.e. continuous integration, pair programming, etc… from XP) and apply those practices to another Agile methodology (i.e. Scrum).