Most of us are familiar with waterfall, a simple concept for developing software that starts with requirements, then moves to design, code, and, finally, testing, with the entire project deployed as one piece at the end. Agile and DevOps disrupt those ideas, suggesting that requirements, design, code, testing, and deployment are activities, not phases. These activities weave in and out of each other all the time.
Waterfall is a bunch of translation activities, and those translations are redundant—that is, the design is a translation of the requirements into the language of architecture, the code is another, and a formal test process is a third.
And with each translation, there’s the opportunity to introduce error.
The Telephone Game
You may remember the childhood game of telephone. All the children sit in a circle, with the leader passing on a message, such as “Take the Christmas tree to the living room and decorate it.” Each child whispers what they heard into the ear of the child next to them. What comes out at the end will have nothing to do with Christmas, a tree, or decorations—the message gets misheard and garbled with almost every repetition, and the end result is usually laughable.
This information loss happens when we translate and substitute symbols. What is effective error-handling for the customer becomes efficient exception-handling for the programmer, which could mean anything to the tester.
The fix to the telephone game is to actually talk to each other. But this talking is messy. It involves misunderstanding, confusion, and sometimes arguments over the definitions of words. It can be uncomfortable and look like progress is stalled.
The idea of working together as an actual team, instead of handing things off along an assembly line, is sort of weird. It is different from the ideal of the “modern” corporate world, the hummy machine. It can make people uncomfortable because it involves developing an entirely new set of skills around communication and collaboration.
But without talking to each other, the confusion and progress stalls happen anyway; they are just delayed. We find the problems at the end of the process and have to do the work over. The problems compound and the fixes become more expensive.
A Hidden Bonus of DevOps
The great bonus of DevOps is that, when done right, it collapses the telephone game. Instead of translation activities, you get developers, testers, operations, and product owners in the same room, sharing skills to get the work done.
Except when you don’t.
Some companies take a different approach, forming a new, distinct “DevOps team.” On paper, this isn’t a terrible idea: Create a small group of architects who enable self-service to the teams, making it possible for the teams to own the entire process, from concept to production support.
I’ve even seen it work out that way—once.
More often, the DevOps team creates yet another dependency outside of the team, yet another handoff, and yet another round of the telephone game.
Lessons Learned
Two years ago I said that you can’t buy DevOps. At the time, companies were creating an entirely new job description, the “DevOps ninja rock star,” and trying to staff a team with them. Not only did the new team add to the telephone game, but people with the skills for the new technologies simply did not exist yet.
As a result, some people lied on their resume to get hired in these brand-new roles, and a small fraction of the development world collected oversized salaries as the late adopters of DevOps scrambled the hire them. This also created a second tier of technical staff: the “mere production workers” who stayed on the team, working on the old technology, waiting for the DevOps ninja rock stars to create the self-service, cloud-based, continuous continuity infrastructure they would need to deliver software.
Again, it sounds good.
But in the meantime, skills are stagnating on the technical team, specialization is actually increasing, and the telephone game is getting worse. Quality professionals—or at least professionals who care about quality—know that “the telephone game is getting worse” means more and bigger bugs that are discovered later and are harder to fix.
Let’s talk about a different way.
The Culture Clash
What I’m really writing about here is a difference in culture. If people are short-term assets you can use and throw away, then the only thing to do is separate them by “use,” hire DevOps ninja rock stars to “do DevOps,” and keep the technical team working.
If, however, people can actually learn and grow, then you can invest in skills development. Blur the lines between dev, test, and operations and allow everyone to work together, thinking of the workplace as one big experimental lab. Let them purchase books, visit websites, and experiment with new technologies. If your company has a dozen technical teams, let a dozen DevOps flowers bloom, then allow them to compare notes in communities of practice.
Like actually talking together, this approach will be full of problems. We’ll be following the advice of Miss Frizzle from The Magic School Bus: taking risks, making mistakes, and getting messy.
And make no mistake, it will be a mess. But I expect that it will also be an improvement.
User Comments
Nice article!
I'd like to work around the telephone game by improving communication around exactly what the SUT is doing (or, since I see this from a quality management perspective, what the functional quality is in highly trustworthy detail). To do this, make your automation driving the SUT for quality pay attention to what it's doing and what the SUT does in response - make it document itself at runtime in a way that supports all that information. That's what I do in the two most basic patterns of MetaAutomation.