Scaling Agile Development via Architecture

Every system has an architecture, even systems developed using agile methodologies. Whether you attempt to define that architecture up front in detail or whether it emerges over time is up to you.

My experience is that most agile teams follow a strategy somewhere between these two extremes, one that involves investing a bit of time up front to think through the "big issues" but which addresses the details on a just-in-time (JIT) basis.

That strategy, combined with proving your architectural ideas as soon as possible through working code, results in a very agile and low-risk approach to system architecture. This article summarizes a collection of strategies for addressing architectural concerns on agile projects and discusses how such strategies can be applied to scale agile methods to large development efforts.

Agile Architecture Strategies
Many of the architectural strategies listed below describe trade-offs; the implication is that you must tailor an approach which meets your unique situation. These strategies are:

1. Model with others. Modeling, architectural or otherwise, is best performed by two or more people working together collaboratively. By working collaboratively you will create a higher quality product, will develop a shared vision, and will learn from one another.
2. Focus on collaboration over documentation. {sidebar id=1} quot;Agile architectsquot; are active members of development teams who also write software–they are not simply people who document their vision and hand it off to developers. This greatly increases the chance that the system reflects the desired architecture and that the architectural vision reflects the realities faced by the team. Note that there may still be some documentation, but that's a secondary effort.
3. Prove it with code. Everything looks good on a whiteboard, or in a PowerPoint slide deck, or in a modeling tool. You have no idea if your architecture actually works until you try to implement and test it. The implication is that you should prove that your architecture works, something that Extreme Programming (XP) calls spiking and the Rational Unified Process (RUP) calls architectural prototyping. Sometimes you will discover that your original approach doesn't work or how your approach actually works (instead of how you thought it would work). Proving your architecture with code helps to reduce risk to your project because you quickly discover whether your approach is feasible; the longer you go without coding the greater at risk you are.
4. Keep it simple. Agile software developers model, but we often do it in ways which are very different than traditionalists. We create sketches, we capture requirements in index cards, we create abstract user interface models using paper and sticky notes, we write acceptance tests which we consider to be requirements, and we write unit tests for our detailed design efforts.
5. Use the simplest tools. The vast majority of agile developers create free form diagrams, often simple sketches on a whiteboard which overview how we intend to build the system. These diagrams evolve over time. Teams lucky enough to have dedicated whiteboard space will evolve their sketches appropriately as the project progresses. Sometimes the diagrams are captured and cleaned up using a drawing tool or a software-based modeling tool, and we may even generate new code from the models or use the tools to visualize existing code. Sometimes the simplest tool for the situation at hand is a whiteboard; sometimes you need a tool that is a bit more complex.
6. Think through the big issues up front. Figure 1 depicts the Agile Model Driven Development (AMDD) lifecycle. It indicates that initial architectural modeling is typically performed during quot;iteration 0,quot; the initial inception phase of your project. For short projects (perhaps several weeks in length) you may do this work in the first few hours and for longer projects (perhaps on the order of twelve or more months) you may decide to invest a few days to do the initial architecture modeling.
7. Think through the details just in time. As you can see in Figure 1, you will often choose to quot;model stormquot; focused issues on a JIT basis. These model storming sessions are typically impromptu events: one project team member will ask another to model with him, typically lasting for five to ten minutes (itis rare to model storm for more than thirty minutes).The people get together, gather around a shared modeling tool (e.g., a whiteboard), explore the issue until they're satisfied that they understand it, then they continue on (often coding). It is desirable to take a JIT approach to identifying the details because the requirements are going to change over time anyway, you can ask better questions based on your greater domain knowledge later in the lifecycle, and your stakeholders provide better answers because they've seen working software delivered on a regular basis. Frankly, there's no rush - if you have the ability to model something today you will have that same ability tomorrow.
8. Allow good architectures to emerge over time. Although you can start with an architectural vision, or an architectural metaphor if you like, the fact is that the details will emerge as your system evolves to meet the changing needs of your stakeholders.
9. Travel light. Don't create a 50-page document when a five-page one will do. Don't create a five-page document when a diagram will do. Don#39;t create a diagram when a metaphor will do. Remember Agile Modeling's They Ain't Gonna Read It (TAGRI) advice: close collaboration with others is a much more effective way to communicate your ideas than handing them documentation.
10. Have a few overview diagrams. Just because you're traveling light doesn#39;t mean that there isn#39;t any documentation at all. I typically strive to create one or more navigation diagrams, diagrams that present an overview of the "landscape" of the system. Just like a road map overviews the organization of a town, your navigation diagram(s) overviews the organization of your system.
11. Be flexible. The type of navigation diagram(s) that you create depends on the nature of the system that you are building. No one set of architectural views is right for every project. Instead, the nature of the project will help to define the types of views that you should consider creating.
12. Display models publicly. Your architectural diagrams should be prominently displayed where your team can see them and, better yet, update them. For a co-located team your best approach is to simply have dedicated whiteboard space for the team. Distributed teams find that a Wiki with snapshots of diagrams and point-form text works well.
13. Take a requirements-driven approach. Your architecture must be based on actual requirements put forth by your stakeholders, otherwise you are ";hacking in the large."

 Architecture Enables Scale
Although many software development projects are done by teams of less than ten people, some are much larger. Worse yet, your team may be spread out amongst multiple locations, time zones, and even organizations. To deal with the inherent complexities of managing such a team you will organize it into sub-teams, which in turn may be recursively organized into more sub-teams as required. These sub-teams will typically be responsible for developing one or more subsystems, and they must ensure that their work integrates easily with the other subsystems. This motivates you to have a core architecture team which identifies, supports, and evolves a common architecture across the various sub-teams.

At the beginning of a large project you should identify your most experienced developers and abstract thinkers, as well as a few people that you want to see get some architectural experience, and invite them to be members of the core architecture team. You want to do this for two reasons. First, you want good people. Second, each of the development sub-teams will include at least one member of the core architecture team. This increases the chances that each sub-team understands and follows the overall architecture and that the core architecture team will not ignore portions of the system. Furthermore, it ensures that each sub-team has some senior people on it with an understanding of the architectural vision. Frankly, this is something you would do on any large project, agile or not. Agilists are very practical, we're more than happy to adopt good ideas from the traditional community.

The core architecture team is responsible for identifying the initial architecture then bringing it to the rest of the project team for feedback and subsequent evolution. Architects will typically do this during iteration 0 of the project, which for larger projects can prove to be several weeks in length. Their goal is to identify the subsystems–critical information required to identify not only the potential sub-teams but the skills required on those sub-teams. At this point in time you also want to identify and agree to the interfaces to these subsystems. The interfaces will change over time, so be prepared to refactor your code accordingly, but the goal is to enable the sub-teams to focus on their portion of the system without having to worry about what the other teams are doing. This strategy is something that Grady Booch calls quot;managing to the seams.

To avoid an ivory tower architecture, the members of the core architecture team take active roles on the development sub-teams, communicating the architecture to other members of the sub-teams and working with them to prove portions of the architecture via concrete experiments. From the point of view of the development sub-teams, the architect acts as both an architectural consultant and as an active member of the sub-team. In other words, the architect is another member of the team who gets his hands dirty coding.

The core architecture team will find that members need to get together occasionally to evolve the architecture as the project progresses, negotiating changes and updating any architectural model(s) as appropriate. These working sessions will be frequent at the beginning of a project and will be needed less and less as the architecture solidifies. It will be common for members of the development sub-teams, who may not be members of the core architecture team, to be involved because they may have some specialized knowledge or experience to share with the team. The best meetings are short, no more than an hour in length, and are often held in a room with lots of whiteboard space. Everyone should come prepared to the working sessions, willing to present and discuss their issues as well as to work together as a team to quickly come to resolutions.

You may even find that you need to scale to an even more difficult situation: some projects are part of a larger program which may comprise several dozen other projects which are implemented over a ten to fifteen year timeframe. Your core architecture team and the models which they create will help to promote a shared, evolving vision throughout the lifetime of such an effort.

Parting Thoughts
You'd be hard pressed to find an agile team that doesn't model on a regular basis, and this includes architectural modeling as well. This article overviewed a collection of architectural strategies for agile software development teams. I invite you to join the Agile Modeling mailing list, www.agilemodeling.com/feedback.htm, so that we can continue to discuss agile approaches to architecture, requirements, and design as a community. I also invite you to get involved with the Eclipse Process Framework (EPF) at www.eclipse.org/epf/ where we are developing both an open source editing tool and open source process materials describing agile development methods and techniques. There is a lot of interesting work going on around agile architecture techniques and I'd love to see you get involved. I'd like to thank Per Kroll and Jim Ruehlin for their feedback regarding this article.

Suggested Resources
The Agile Data Site, www.agiledata.org, contains several articles describing agile approaches to enterprise architecture.

The Agile Modeling Site, www.agilemodeling.com, contains a large number of articles about agile modeling techniques, agile requirements, and agile architecture.

IBM developerWorks Architecture Forums and Community, offers a wide range of resources for web services, SOA, J2EE, UML, and architectural techniques.