A healthy agile project has several typical rhythms such as releases, iterations, stand-up meetings, builds kicked off by continuous integration, and the red-green-red test cycle of a developer. These rhythms have healthy ranges (such as a stand up meeting lasting less than 15 minutes) and characteristics (such as that same stand up meeting not containing design discussions). When they fall out of these ranges or do not display the appropriate characteristics, they indicate that something is wrong with the agile process.
A healthy agile project has several typical rhythms such as releases, iterations, stand-up meetings, builds kicked off by continuous integration, and the red-green-red test cycle of a developer. These rhythms have healthy ranges (such as a stand up meeting lasting less than 15 minutes) and characteristics (such as that same stand up meeting not containing design discussions). When they fall out of these ranges or do not display the appropriate characteristics, they indicate that something is wrong with the agile process.
As a doctor takes a patient's pulse or blood pressure reading to determine if the patient is ill, we take readings of our rhythms to let us know if something is wrong with our agile process. The rhythms, when off, sometimes need to be addressed directly, and in other cases point us towards areas of the process where we need to dig more deeply to find and correct the root cause of the problem.
Before we dig into each rhythm individually let us ask ourselves a question: why are rhythms important to agile methods? Think about it: agile methodologies are all about agility, which is response to change. Before we can even respond to change, we have to know that the change has occurred. How do we know that a change occurred? Feedback. Feedback at all levels of the development cycle. Each one of the rhythms we discuss is a form of feedback. If a rhythm is off then the information that we use to be "agile" is incorrect. When the rhythm is off it is a symptom of a malfunctioning agile process.
In this article, we define and categorize the rhythms of {sidebar id=1} agile processes. Then we will analyze each rhythm separately in order to highlight its valid ranges and characteristics.
The Rythms - An Overview
In true agile tradition, we will start with defining rhythms related to delivering value to the customer. The most basic rhythm that addresses customer value directly is the Release. When the time between releases is larger than one to three months (as it frequently is) then this is usually mitigated by several Customer Review Cycles. These Customer Review Cycles may be as frequent as every development Iteration but many times occur less often. The final cycle that is directly related to customer value is the Story which is the individual piece of functionality that has value to the customer. These three rhythms - Release, Customer Review Cycle, and Story- are directly related to customer value and solving the ‘right problem' (see Figure 1).
There is another set of rhythms related to delivering the ‘problem right'. These rhythms are driven by the complexity of the problem at hand. Thus they split the problem into manageable pieces. These rhythms should be very familiar to the development team. They are the Iteration, (Daily) Stand-Up meeting, Story Tasks which directly correspond to a cycle of Continuous Integration, and the Red-Green-Red cycle of test-first development.
Release
The release cycle is the most directly related to customer value. It is also the one cycle that is almost always beyond the decision-making scope of the development team. Many release cycles are over a year long (for those of us on agile projects in non-agile organizations), which means that direct feedback about the utility of the product happens very infrequently. Multiple customer review cycles are used to mitigate this issue by giving the team more frequent feedback.
Customer Review Cycle
The customer review cycle is often confused with an iteration in agile methods. XP, for example, with an onsite customer mandates that a customer review should be done at the end of every iteration. In reality we are not all so lucky to have a real customer review every iteration. In many cases we fall back to our own iteration reviews and fool ourselves into thinking that this is a true customer review. It is not. A customer review must involve a customer.
What are healthy ranges for the customer review cycle? Of course ‘it depends' on the particular project and context. However, we have found that anywhere from one to three months is healthy. We start to feel uneasy at the later stages of this boundary. When a customer review cycle is too long then there is a danger that the software being built will not address the real problems of the customer. The solution to this problem is to have more frequent customer reviews.
Stories, Functional Tests, and the Burn-down Chart
User stories are the basic unit of functionality and are used to create functional tests. A burn-down chart shows the stories that have been successfully completed and tested and those that are still remaining to be developed.
There are several characteristics that are to be expected in a healthy agile process:
- User stories should be built within one Iteration.
- User stories should describe business functionality, not technical requirements.
- The burn-down chart should be steadily decreasing.
When a user story spans more than one iteration this indicates a problem. Writing smaller user stories is a possible solution to this particular problem and is discussed in the lesson's learned paper Recognizing and Responding to "Bad Smells" in Extreme Programming .
If the burn-down chart oscillates and does not steadily decrease then another set of problems may be present. When stories are based on technical issues instead of functional requirements then as the design of the software evolves new technical stories are created. This will force the burn-down chart to oscillate and the measure of true progress is skewed as shown in Figure 2. Ron Jeffries discusses a metric called Running Functional Tests, which is very related to the burn-down chart.
Ken Schwaber and Mike Beedle describe several Scrum Signatures in Agile Software Development With Scrum which are common occurrences to different teams. They recommend that the Scrum Master ought to be aware of the team's signature and take a closer look upon deviation from their typical signature.
IterationSo what is a healthy iteration length? An iteration should be as short as possible while allowing a useful amount of function to be created, tested, and delivered. Iterations should be long enough to produce useful business functionality. Iterations allow the team to manage complexity. We have worked teams that were very successful with iterations of one to four weeks long.
An iteration may be too short if the stories or use case scenarios built have no useful value to the customer and are only ‘pieces' on the way to completion. This means that real story length has crossed the boundary of an iteration. An iteration may also be too short if it is exceedingly difficult to ‘quarter the chicken' (the chicken being a user story).
Nevertheless an iteration that is too long may point to a different set of problems.
- Stories may become epics with too much functionality. Although the lengthy stories are the problem, it sometimes helps to reduce the iteration length to force stories of smaller size.
- Feedback cycles, which ultimately correspond to an iteration length, are longer. For example, a four week iteration means that we get feedback at this rate. So if our team takes three cycles to learn from a mistake, that is a learning cycle of twelve weeks. As a result, we, the team, are less agile.
- Developers' estimates are inaccurate and thus they are likely to miss task completion with longer iterations. The most common solution here is to change velocity, but remember that velocity can only be adjusted every cycle (which is our iteration length).
Daily Stand-Up Meeting
Daily stand up meetings are the control for a healthy iteration. They give the team members an opportunity to get a complete picture of the current state of the project within the iteration. They also provide the chance to catch impediments early and have them removed in order to assure successful completion of the iteration goals. With the exception of the initial stand up meetings (not beyond the first iteration of a project) these meetings should be short and to the point. They should be reports of progress made and impediments encountered or removed.
So when do you know you have a problem? Firstly, the most notable indication is a lengthy meeting - more than 15 minutes should get you wondering, past 30 minutes should have you worried, a full hour and the stand up meeting is broken. Secondly, we find that many people revert to standard meetings where design is discussed or planning for the future. This is not the time or the place for these things to be discussed. Thirdly, if the chickens are talking or leading the meeting then the value of the meeting as a status and control meeting is compromised[1].
Story Task - Continuous Integration Cycle
The continuous integration cycle is very dependent on the check-in rate of the developers. A continuous integration build needs to be fast enough that a build can be run before too many developers check in. If the build is not fast enough and breaks then several developers may have checked in since the last good build. Of course, each one of them assumes that he or she did not break the build. They will therefore not fix the build since their belief is that one of the other five developers broke it. The build will therefore remain broken.
This is the beginning of a long, slippery and dangerous slope. Broken builds and broken tests should never be tolerated. They not only let small problems, indicated by a simple broken test, to become larger problems by having more tests break but they also desensitize the development team to broken tests. Moving from 0 broken tests to 1 broken test is much more traumatic than moving from 100 broken tests to 101 broken tests.
Broken builds and broken tests usually point to one of two possible problems: either the build is too slow, or the team's discipline with respect to failing tests needs to be improved.
Red-Green-Red Loop
The red-green-red test cycle is only present during test-first development. When it is present it should be extremely short, on the order of a couple of minutes to 15 minutes on average. If tests cycles are consistently taking 30 minutes or more to write and execute then this is a possible indication that the development steps being taken are not granular enough or that the system under test has an unhealthy degree of coupling. We have seen (and sometimes participated in) test cycles of 4-12 hours - these definitely indicate that there is a problem. In many cases unit tests have grown into system integration tests and trying to juggle an inordinate amount of objects to setup the system ‘just-right' so it can be tested leads to this problem.
Conclusion
We have presented several well-known rhythms of agile development processes and hopefully conveyed to the reader that these rhythms are excellent indicators of the health of his/her particular agile development process. We have not gone into great lengths of how to address problems. A prescriptive solution to each problem really does depend on the context. Our hope is that the reader can take away a few useful diagnostic tools. Once a rhythm is found to be out of range then due diligence should be done with the team to find and correct the source(s) of the problem.
[1] This is based on the naming of Scrum Meeting participants from the joke: "A chicken and a pig are together when the chicken says, "Let's start a restaurant!" The pig thinks it over and says "What would we call this restaurant?" The chicken says "Ham ‘n Eggs!" The pig says "No, thanks. I'd be committed, but you'd only be involved!".
About the Authors
Amr Elssamadisy is currently a Principal Consultant at Valtech (www.valtech.com). He considers himself a developer but has also worked for consulting companies since 1999, so maybe an outgoing, people-oriented, developer is a better description. He has been working professionally as a software developer, architect, manager, consultant, etc... for over 12 years helping build software systems in C++, J2EE, and .NET. His first agile development project was a large project XP effort in 1999 where he had a chance to work and learn from some of the best in the field. Since then he has lead, participated, and guided teams in several large and small agile development projects in both the .NET and J2EE worlds.
Ashley Johnson is VP of Business Planning and Strategy for Valtech Skill Development, an organization focused on guiding clients to successfully adopt modern development processes and technologies. In this role, his time is split between refining new service offerings and consulting with senior IT management to facilitate organizational optimization.
Ashley is passionate about applying lean and agile techniques, and using simple metrics to improve visibility and eliminate waste. He has taught, coached, and consulted with thousands of individuals and is an acclaimed presenter. Ashley began developing hardware and embedded software in the early 1980's when he couldn't understand why people would pay him to have so much fun. Under pressure to get a life, he learned to enjoy backpacking, skiing, scuba diving, and radio control soaring. These hobbies are now eclipsed by his wonderful children.
