My experiences in adopting agile practices and becoming a Certified Scrum Master

During the last 4 years as Production Director at Blueprint Web Technologies, I’ve been involved in a large number of software development projects, many hugely successful, others providing valuable lessons from the school of hard knocks. Recently, I was fortunate enough to meet the world renowned agile expert, Mike Cohn, during a Scrum Alliance Certified Scrum Master course I attended in London. Chatting with Mike, I began to reflect back on some of those “lightbulb” moments over the past 4 years and I decided to jot these down in the hope that they might help some other people to answer the question of just “what is agile?”

What is agile?

Developing good software that meets business requirements is hard. Developing great software that exceeds expectations and is slick, empowering and enjoyable to use, is even harder. Industry expert Martin Fowler talks about what he calls the “Yawning crevasse of doom” between intent and implementation. People have great ideas about how their software might work, but between the idea (the intent) and the finished software (the implementation) there is a plethora of possible pitfalls that awaits the unwitting. Perhaps unsurprisingly, then, there is a great interest both among smart business people and software developers in devising better ways to organise projects and manage the software creation process. Scrum is one such methodology and is one of the several flavours of an approach known as “agile” software development. Agile is an umbrella term that encompasses other processes, such as Extreme Programming (XP), DSDM, Kanban and more.

Agile methods focus on producing results through frequent iterative cycles of plan > do > review > deliver. This contrasts starkly with the earlier “waterfall” approaches to project management, where projects were divided into long linear sequential phases with all planning done up front and no testing being carried out until after development (the doing) had been completed. With waterfall, if it turns out during testing that something was wrong with plan, there is little option but go all the way back to the beginning and start the entire project over again from scratch.

One of the big, recurring, issues that I have found is that ideas and intentions about software are usually incomplete and, on closer inspection, frequently contradictory and changing. Although challenging, this problem is not by any means insurmountable, but in order to get past it, we need to recognise and acknowledge it and adjust our approach accordingly. There is no point in getting into the kind of arguments that follow patterns such as:

Developer: “This requirement was not in the brief”  
Client: “Surely you should have guessed we needed it”

But how should we best approach a problem where the requirements are incomplete, contradictory and changing? How can we provide answers if we don’t even know what the questions are?

What’s the problem with software development?

In their 1990 book “Wicked Problems, Righteous Solutions”, DeGrace and Stahl introduced the concept of “wicked problems” to software development. Wicked problems are difficult to solve because of incomplete, contradictory and changing requirements. Difficult, but not impossible - it all depends on the approach. Clearly, if requirements are incomplete and changing, then adopting a linear, waterfall, approach is likely to lead us head first into that yawning crevasse of doom, with budgets blown and reputations in tatters.

In contrast, if we adopt an iterative, agile and collaborative approach, we will have the opportunity to frequently test and refine our development plan as we go along, continually adjusting our course away from the edge of doom. An agile approach also aligns neatly with popular ideas around lean startup, minimum viable products and service design.

Simple right? So why isn’t everyone doing things the agile way? Why do organisations find it so difficult to adopt genuinely agile approaches.

Obstacles to going agile

In my experience, perhaps the biggest obstacle to adopting agile methods is found in one of the Twelve Principles of Agile Software, namely that:

Business people and developers must work together daily throughout the project.

I think that this requirement of daily communication between business people and developers often simply does not sit well with the structures, processes and priorities that are incumbent in many organisations. For many people, it is all too easy to fall back into the waterfall way of doing things, because this is the way it’s always been done - and it’s too much effort to change.

Another major challenge of the Agile Manifesto is the call for “Customer collaboration over contract negotiation.” With an agile approach, how do we know the scope of what will ultimately be delivered? How can our lawyers draft a contract to take account of this uncertainty? Both sides need to reach an understanding on this challenge and devise a workable solution.

In my experience, it takes a lot of effort from the business side to start working with developers in an agile way and to make a real commitment to this way of doing things in the long term. However, when it happens and when teams embrace this way of working, the impact can be genuinely transformational; resulting in greater efficiency, lower costs and, above all, radically better software.

Opportunities for a holistic approach

One of my favourite aspects of Scrum is that it takes a holistic view of the problem of software development. Scrum isn’t just for developers or just for business people; it’s a way of working that brings the two together. I can’t emphasise this point enough. The Scrum Alliance offers certification routes for both developers and business people. Where Certified Scrum Master is aimed at developers, the Certified Product Owner is aimed at business people. There are huge opportunities for projects where Certified Scrum Masters can work in close collaboration with Certified Product Owners.

It’s a great time to be working in software development and, at Blueprint, I’m lucky enough to be involved in some great projects for a range of fantastic clients. As a relatively new and small company, we are fortunate enough to have evolved around an agile core from day one. This is great, but it’s also particularly rewarding to work with clients who are new to agile and are transitioning away from old fashioned waterfall approaches – seeing the change taking place is brilliant.

I'm always interested to hear other people’s experiences with agile software development projects, so please feel free to get in touch if you have anything to share.