Get to know our Experts: Agile Practice lead, Dr. Kevin Thompson

Please give us a brief background of yourself for our readers.

I’ve always been fascinated by science and mathematics, and computer programming came to me as a very natural extension of these interests. After getting my B.S. in Physics from Santa Clara University, and my Ph.D. in Physics from Princeton University, I performed research in Astrophysics and Computational Fluid Dynamics at the NASA Ames Space Sciences Research division in Moffett Field, CA. At that time, CFD work entailed writing huge programs in Fortran on the fastest supercomputers in the world. It was fun, in a lot of ways, but it wasn’t a great way to make a living in the long term, and writing Fortran code (dare I say it?) palled eventually.

I migrated into commercial software development. Over several years, I moved from individual developer, to team lead and designer, and part-time software project management. I wrote code in Smalltalk, Object Pascal, C, and Java for companies such as OnlineFocus, IBM, K2 Technologies, and Macrovision. When writing Java code (dare I say it?) palled eventually, I made the transition to full-time software project management at a company called StarCite.

I first encountered the Scrum framework while wearing dual hats as a project manager and PMO Manager. We conducted a couple of pilot programs with Scrum and found that it provided far better visibility and predictability than the Waterfall-style process we’d been using, so I began working with other projects to roll out Scrum across the company.

Finally, when the day-to-day work of being a Project Manager and Scrum Master (dare I say it?) palled eventually, I decided to move full-time into Agile consulting and training . This decision happened about the same time (2009) as the opportunity to join cPrime to do exactly those things.

Five years later, I remain fascinated by the subtleties and practicalities of the Agile world, which has not palled at all.

Why do you think more industries and companies are moving toward Agile development?

The reasons vary from one company to the next, but there are a few very common reasons.

One motivation is to reduce time-to-market. We see this a lot at large companies, where eighteen-month deployments are not uncommon.

A second motivation is to improve the company’s ability to change direction as business needs evolve.

A third motivation is to escape from either paralysis or chaos into something that works and gets products out the door. This is especially true for small to mid-size companies, which have grown beyond a level where “cowboy coding” can even vaguely work. They need a process that works in a reliable and repeatable fashion, and which supports their growth over time.

What are some of the key challenges you’ve seen organizations have to overcome during an organization-level Agile transformation?

The number-one answer is always, “Culture.” We’re asking companies to change some very fundamental behaviors across a large number of people. Honestly, if they weren’t hurting so badly they would never make such sweeping changes, but they usually are hurting that badly, and are desperate for improvement.

I usually say that there are three things that have to be present for an Agile transformation to be successful.

The first one, which is overwhelmingly important, is executive support at the top. Without this support, meaningful change is unlikely. “Support” means more than some kind of vague and distant approval. Executives and managers must learn what the changes mean, and why they are important. After this, they must make the changes happen by ensuring that the transformation is planned and executed properly, that budgeting for the new world is provided, and that people get the training and tools they need. Finally, they need to live by the new model, and back up the new roles and authorities the process entails.

The second requirement is deep knowledge about the Agile world. Someone has to understand the to-be state, and provide that knowledge to the organization.

The third requirement is knowledge about how to conduct an Agile transformation, which is distinct from knowledge about Agile processes.

In the end, all problems can be solved with sufficient determination. So the ultimate challenge is this: Can the organization muster the will to succeed?

What are some metrics you like to use to measure the success of Agile transformation?

There are many metrics to measure the proficiency of Agile teams, but few metrics for measuring the effectiveness of an Agile transformation. The latter is a problem because organizations typically do not capture information about how they are doing before they transform, in a way that would enable determination of how much better an Agile process is serving them. Still, there are some things we can look at.

Product Quality: This is the easiest thing to measure. “Escaped defects” (bugs found by customers) typically go down in product releases after effective Agile transformations. The reason they go down is because quality is maintained at a high level throughout the development process.

Time to Market: For companies where time to market has been a problem, improvements in time to market due to Agile transformation are easy to measure.

Adaptability: This is harder to quantify, but easy to see in practice. Can the business (executives, product managers) shift direction more quickly and easily than used to be the case? The answer should be an unequivocal “Yes.”

What are some common mistakes organizations make when trying to move to Agile development?

The biggest mistake is trying to do this without help. Failure rates are very high when none of the people in the organization have Agile experience.

Another big mistake is assuming that “Agile is something engineers do.” This isn’t true. The entire sequence of activities from making strategic business decisions to shipping products is affected. Trying to insert an Agile process for writing software into a larger process that does not otherwise change will not work. The most effective organizations are the ones that reshape Portfolio and Program Management, as well as the mechanics of building products, to work along Agile lines.

In terms of people, process, technology – what are some key advice you like to prescribe to organizations during an Agile transformation?

The first piece of advice is to get help from someone who knows how to make Agile transformations successful.

The second is to take care in planning and executing the transformation. For any but very small organizations, it isn’t sufficient to train a few people and go. Each organization has different needs and issues, and it is important to identify these and plan the transformation in a way that will yield success. It is very easy to underestimate the scope and impact of an Agile transformation, and failing to plan (as always) means planning to fail.

The third is to internalize the knowledge that you can’t make changes by leaving everything the same. I often hear, “We don’t do things that way here,” as if that were the end of a conversation. I often say, several times in a row, and as diplomatically as I can, that, “If you want to achieve the benefits, you really do have to make these changes.” That can be a hard thing for people to hear, but it’s true.

Finally, and as part of the third item, it’s important to provide funding for test automation, code coverage, and continuous-integration tools, along with sufficient environments for development, testing, staging, and so forth. It is critical (as in non-negotiable) that the organization must have the tools and environments required to build and test a new capability end-to-end in just a few days’ time, and be able to test a new feature at the moment the feature is ready for testing. While no one expects perfection in these areas at the start of a transformation, the organization must summon the will to make these things happen, or the pain will go on for a very long time.