An Architect in the Agile World

For the last twenty years, I’ve been building software with all kinds of different people and in all kinds of different companies. Since then, things have changed in almost every way imaginable. Even more interesting than the changes in workplace culture has been the diversity in processes. I’ve watched companies struggle for years to make the process of delivering the complex entity that is a software product in a way that is consistently reproducible, of high quality, and is produced as cheaply as possible. Most of us have watched these cycles go back and forth, even when much of the churn and turmoil is caused by the company itself as it tries to balance costs, time-to-market, and quality. Then in 2001, the agile process began to emerge and roles began to change. So, in the modern agile world, is there still room for the architect? If there is, how has that role changed in the last twelve years?

What Has Been the Traditional Role of the Architect?
There was a time when companies were active in the waterfall model of development or embraced rapid application development (RAD) where the architects were king. People got the image of the technical brains behind a product handing down designs from on high. At the time, there weren’t as many people well versed in engineering with the depth needed to make good, broad design decisions. The reality was most often much less glamorous, but there was some truth in the analogy. Products often lived and died because of the decisions of a few individuals, peer reviews were less rigorous, and few points of failure created stress on both sides of the equation. Many products also failed in spectacular fashion because of the wrong person being in the architecture role.

The Difficulty of Being a Generalist in Today’s Technical World
So, how have things changed since then? For one, the depth of talent is much deeper than it was twelve or thirteen years ago. As the industry has taken off and software has become an ubiquitous part of all of our lives, more people have grown up with an inherited exposure and background in computing. Young engineers enter the field these days, despite the questionable quality of our education system, having grown up interacting with software systems countless times a day. A natural understanding of how systems are assembled has evolved from this exposure. As people have access to the tools needed to build software in almost every home, tinkering at all ages has become more commonplace.

One might be tempted to think that the problem of architecture is then solved by the evolution of our daily exposure and understanding. Sadly, that isn’t the case. Instead, new problems have arisen to take their place. Systems around us have grown more complex with more moving parts, interactions with other systems, and competing technologies. This makes it almost impossible for any single individual to be a master in all design disciplines, such as user interface, server design, threading, database systems, Windows, Linux, IOS, mobile architecture, etc. Why is this significant to the role of an agile team architect? It’s vital to keep in mind that the expertise each individual brings to the team is important.

Despite the best intentions of the process itself, it is not always the case that engineers are interchangeable for all tasks. This is very dependent on the task at hand and how much depth is needed. What this highlights is the need to be able to spread critical design work out amongst your team in any such project. While someone in the architecture role may have excellent broad depth and even deep depth in many disciplines, the expertise of the whole team will be needed at various stages of the project.

The Architect As Mentor and Road-Block Destroyer
Does this mean that the architect is then obsolete in today’s agile project team? I believe the answer is a clear “no.” In fact, in an agile project the importance of an architect is even more critical than it has been in the past. So, how does an architect solve some of the problems we’ve just touched upon within an agile team?

First, the architect must be a peer, like every other member of the team. As with any successful agile process, there should not be a hierarchical structure within the team where some folks are reporting to others. This can undermine the honest flow of ideas and feedback and make some reluctant to speak up or share thoughts. This helps build the all-important trust aspect that is critical to the successful functioning of small agile teams.

Similar to the role of the ScrumMaster removing obstacles, this puts the architect in the position to help remove technical road blocks that the team may encounter. Even more importantly, while in this position, they can also ensure that these obstacles are removed in a way that is consistent with the overall design of a feature and the architecture of the product. Why is this so critical? Because there are many technical paths to a solution in an engineering project. In this approach, they can be solved in a way that mitigates two dangerous problems that can arise in such agile projects:

1. First, it ensures that the solution doesn’t cause design incrementalism and remains consistent with the long-term goals of the project. That is to say that each small piece of the system or each solution to a problem risks following its own isolated design pattern or approach; that of which only a segment of the team is following.
2. Secondly, because of the first mitigation, the solution is less likely to suffer from the need for downstream revision. This removes possible unforeseen work late in the project cycle that can derail the release, and also helps to ensure that the engineer doesn’t need to redo work and has more confidence in the long-term success of their work.

All of this solidifies the architect’s role as a mentor and helper, encouraging rather than blindly handing down decisions from on high, as might have been the case in the past.

Why the ScrumMaster Should Be neither the Architect nor the Manager
I’m often involved in a broad array of companies in which the manager of the development team is also the architect as well as the ScrumMaster. The managers of these groups are most often convinced that this method of organizing the development team is working very well. At the companies I’ve had opportunities to dig into further with I’ve almost always discovered that this isn’t the case, and the team frequently suffers from a general sense of powerlessness. Design decisions are taken out of the hands of those doing the work, their manager consistently plays the role of the smartest person in the room, and the team members aren’t comfortable speaking up. When issues are brought up in retrospectives they tend to be disregarded, as the ScrumMaster cannot act as an advocate to solve management problems. Since the feedback loop is fundamentally broken these managers create a closed loop of feedback in which they believe the team is functioning perfectly. The end result is that the manager continues to drift further away from reality (as the team sees it) until a sudden implosion occurs and the manager is taken off guard.

The roles of the ScrumMaster and the architect are critical, both of which should be active peers on the team helping remove road blocks. There should be no hesitation in the team members feeling empowered to bring problems to the folks in these roles or speaking up during the retrospective process to ensure that continuous improvement is being achieved.

It is my continued and strong belief that senior technical peers on Scrum teams should serve as the product architects. They should never be part of the reporting structure, but clearly in a position where the big picture design is part of their direct responsibility and planned into the user stories themselves. Their job is largely to ensure success of the product and trust within the team. However, when done properly, the entire team perceives them as helping each and every team member be more successful; this is good for the individual, the team, the product,