As Borland evolved over the last 25 years, acquiring companies and shifting business strategies, the delivery organization had become a collection of teams with different cultures, processes, release cycles and levels of performance. The cost structure of the organization wasn't aligned with the strategic objectives of the company, and the teams were struggling to consistently meet delivery goals.
In 2006, Borland's leadership made the decision to transition this development organization—more than 350 engineers, working on a broad portfolio of development projects from 13 different locations around the world—to a more Agile approach as part of an effort to vastly improve performance, be more responsive to customers and improve quality.
In February of that year, Borland piloted an Agile project on a totally new product to determine whether the methodology could help achieve three key goals: reduce delivery time, improve overall product quality, and ensure the product met the market need by engaging customers in the development process. The project was extremely successful; the team released the product ten days ahead of schedule and with more features than originally requested.
In expanding its use of Agile throughout the organization, Borland decided to use a stepwise, iterative approach to transformation based on geography—converting one location at a time. Like most organizations considering an Agile transformation, Borland faced the challenge of making a major process shift while still executing an aggressive product roadmap. Our business is to deliver innovative software products to the market, and market pressures, customer expectations and business needs made it impossible for us to "take a break" to do a process overhaul. Thus, we faced a situation where we were changing the tires while the car was still in motion. Today, Borland is in "the thick" of transformation: with approximately 70 percent of teams utilizing Agile methodologies—and reaping tremendous benefits.
In this article, I will share some of the experiences from our Agile journey, highlighting decisions we made, challenges we faced and lessons we learned along the way.
Building and Empowering the Self-Managing Team
When you have a team of people somewhat new to Agile, it can be difficult to keep them aggressively moving forward on a business goal while also staying true to Agile principles. The reality is that people tend to revert back to behavior that has personally served them well in the past. At the start of our transition, we made a concerted effort to ensure that both teams and management kept their eyes on the long term goal—the business outcome we were striving for—which was not to successfully implement Agile, but to improve productivity and team performance. Thus, when formulating our "base" team structure, we made the following decisions:
Recruit a full-time Agile expert
We felt it important to have someone who could mentor new Scrum Masters while coaching the teams and evangelizing the Scrum as we expanded it across the broader organization.
Combine the Scrum Master and Development Manager/Director Roles
In some cases, we found it necessary to deviate a bit from the traditional rules of Agile—one such example is our decision to give some of our managers a dual role: manager and also Scrum Master. We were able to do this only by hiring "enabling" management types: individuals who were less authoritative in their management style and much stronger in leadership skills. Managers who were less risk averse were able to put on their more enabling "Scrum Master Hats" and help their teams succeed by removing obstacles and empowering them to make decisions and mistakes.
Keep an eye on traditional "functional leads"
In a traditional team you have technical leads for various functional areas. Functional areas are usually very well defined and everyone on the team understands who's driving what and where the direction should be coming from. In a well-formed Agile team, the lines between technical leaders start to blur and that can be stressful for those leaders accustomed to such well-defined roles. In our case, we had some "technical leaders" that had been designated by management—rather than appointed by the team. As the dynamics within the teams started to evolve and Agile technical "go-to" people started to surface there became a blurring of roles between these emerging natural leaders and the traditional, "titled" technical leads. As we continued our migration, we kept a look out for functional leads that could be experiencing frustration with the move to Agile. We found the best way to avoid this was to mix up common assignments on the team so that no one became pigeon-holed.
The Misconception of Chaos and the Importance of Transparency
Many people—including some of our own management—were under the impression that Agile methods imply random chaos and cowboy development. What we have learned is quite the contrary—Agile methods drive accountability, ownership and responsibility down to the lowest and most appropriate levels. Team members are held accountable not by some faceless process, but by their peers - peers that they must face each and every day at the daily stand-up meeting. Some of the core practices of Agile promote these characteristics naturally, and as we navigated our transformation we developed further practices to drive accountability and transparency across the organization.
Chaos Buster #1: The Daily Stand Up
Every day, each team member has to give an oral account of what they have done since the last stand-up, what they plan to do before the next stand-up, and any impediments that might be keeping them from performing their tasks. Gone are the slick, useless, status reports and emails. Everyone has to stand there day after day and tell their peers what they've accomplished. Talk about peer pressure - these daily oral reports promote accountability and ownership within the team.
Moving to meaningful, concise documentation
Let me dispel the myth that Agile development is only ad-hoc development which leads to spaghetti code that is totally unmanageable. We create design documents at each sprint that we call mini-specs which are linked to a broader architecture specification that is maintained by the product architect and evolves as the product needs to evolve.
These mini-spec documents are written collaboratively by the engineer who has taken on the user-story to develop the feature and the technical lead for the product or functional area. They are usually maintained by the technical lead as a living document and extended or modified as the feature or functionality is expanded. Further, they are posted on our Wiki pages so they're open for review and comment. We keep them brief and relevant—we don't try to capture all aspects of the software, instead we concentrate on recording the reasons for design choices, major components, and any assumptions at the time. My experience with traditional documentation is that it almost never captures this type of information. In Agile, standard best practice development guidelines still apply. The difference is that the team is free to alter and adopt those practices that best fit their needs.
Fostering improvement through retrospectives
Retrospectives do an excellent job of helping the team learn from their mistakes and ensure continuous improvement. As the trust grows among the team members, they start to feel more comfortable asking questions and making suggestions for improvement. They helped us build an environment of openness and objectivity. Further, we've extended our retrospectives to include planning sessions. We found that these were necessary to plan the changes we were going to make to improve on our process. In some cases we may even write user stories that apply to making the process work better. For example, in the area of functional test, we may have user stories that apply to improved test automation or more detailed feature documentation. These planning sessions not only help us improve both the product and our process, they are tremendous in engaging the entire team in the process of shaping a release (critical to driving a sense of ownership). Transparency in an Agile environment really means the entire team owns the release and must have enough information in order to determine a true course of action.
Extending and supporting collaboration and visibility in an enterprise environment
As we scaled our Agile efforts, we began to understand the need for tools to support the teams' collaboration, enable information sharing, and ease the process of reporting to management. It was quickly apparent that the traditional cork teamboards, index cards and sticky notes we were using were not going to scale as we expanded the reach of Agile in our organization. In addition, since our transformation was an initiative that had visibility at the highest levels within our company, we needed to provide visibility into our processes, a defined way to gauge our progress, and the mechanism to measure it. As such, we developed a set of applications to support our Agile transformation. TeamDemand connects marketing and the field with product management and engineering, providing a way for us to collect, evaluate, prioritize and track all of the requests that come into the development organization. We use TeamFocus to manage the execution of both our Agile and traditional projects, and our teams use its "virtual corkboard" in their daily stand ups and sprint reviews. Our managers, including myself, use their TeamFocus dashboards to track "in-flight" metrics—defects, test coverage, requirements volatility—across their project portfolios, while upper-level management gets the "big picture" on how the organization as a whole is performing through TeamAnalytics' business intelligence dashboards.
Beyond Acceptance: Infusing Performance Testing into Agile
A great deal of the discussion in the Agile world as it relates to testing seems to be centered around the need to do acceptance testing of features and functionality. However, moving to Agile development does not in any way eliminate the need to do performance and scalability testing. In the more traditional development models this testing is done at the close of the release cycle—the absolute worst possible time to uncover performance issues (especially ones that are related to architectural design). Done properly, Agile provides a mechanism to overcome the shortcomings of the final "testing" phase.
During our transformation, we found that to achieve the benefits of Agile while maintaining acceptable product performance levels we had to make an upfront investment in the code development effort and in performance benchmarking. Basically, in order to enable performance testing to take place iteratively, our development teams had to slow down and learn how to develop the code in a way that would make this possible. First, we made it clear to all teams that performance was a team responsibility and that working code performing badly may not be considered "complete." The result was a slight slowdown in code development, which ultimately enabled us to speed the overall process and improve performance.
As we got more sprints under our belts, we learned many practices that would help us translate this principle into results. We now ensure that there are sufficient hooks in the code to do performance benchmarking and to avoid the need to re-work critical performance testing scripts. We also make the distinction between performance testing and scalability testing, which includes things such as memory leaks or time to-restart which require extended calendar time. Making this distinction allows us to performance test as we go - not just at the end.
With some minor investment we've made performance testing a part of the build verification process, and we use the final build of the day to performance test the most complex areas of code. We also create an ongoing performance benchmark that evolves as the complexity of the application evolves. This allows us to compare the transaction response times between nightly builds to detect and address problems early. We introduce performance testing into each sprint with a simple user story such as "as a user of [application] I want to experience the same performance level when performing [some action] when there are [x] number of users on the system." Any performance-related issues that we uncover that cannot be addressed during the current sprint for "whatever reason" are addressed in a "hardening sprint"—a final sprint for every release that has no planned features, but is dedicated to quality improvement. In some cases this sprint has no other purpose than to address outstanding minor issues to improve the overall quality of the product.
The Journey Continues
Our Agile transformation is far from complete. My guess is that we'll always be on the journey. In fact, continuing to learn, refine and improve is really what Agile is all about. However, we have already reaped tremendous benefits from our efforts. Our teams are happy and productive, and we have doubled our yearly release capacity. Finally, we have driven down costs, especially in the area of "reporting overhead"—those days (or weeks) every month where management is gathering data and reporting status are gone. I certainly feel the evidence is building that the Agile approach can yield transformational results.
Visibility
By nature, Agile promotes visibility. Daily stand up meetings and sprint reviews give excellent views into how projects and teams are progressing. However, in the enterprise, visibility must extend beyond the team room. Project managers need to be able to see how their teams are progressing, department heads need visibility across a portfolio of projects, and management needs to be able to look at all of these projects and teams from an organizational perspective for effective decision making. The key is to have the visibility be automatic—not require team members to change the way they work - reporting has become more of a monitoring effort and we have learned that we must carefully handle real-time unfiltered information.
The Right Tools
Trying to cram a tool down a team's throat is a recipe for failure. However, scaling Agile to an enterprise is impossible without them. If you provide tools that truly map to the way the teams work, supporting them and enabling them to be more efficient and collaborate, they will adopt them.
Executive Commitment
Borland's executives are committed and enthusiastic about the transformation. They attend sprints and sprint reviews (obeying all the protocols), and they make it a point to showcase and reward good sprints. They have also learned to accept that with visibility comes empowerment and responsibility and[BU9] to treat news as not good or bad—just news.
Agile for the business, not for the sake of Agile
For Agile to work in an established software organization, management must be flexible and pragmatic in the application of Agile principles. They must also be judicious in selecting projects and teams for transition. Allowing teams to be a little bit different in their Agile adoption and adjusting to account for different types of work projects can yield big results. And, it is absolutely critical to have the historical data and analysis capabilities to understand how your Agile projects are performing (are you actually getting the benefits you anticipated), evaluate which projects are good candidates for Agile, identify areas that have room for improvement, and ensure the quality of output.
Results/Benefits
- 100% increase in number of product releases per year
- Greatly improved relationships with strategic customers, who have participated in over 50 sprint reviews
- Reduced administrative and planning overhead by an average of 15 hours per sprint
- Eliminated 6 days a month of vice president and director time spent reporting—per product group
- Increased product quality, reducing issues from release to release by 50%
- Increased team productivity and employee retention through enhanced morale
About the Author
Chuck Maples is the Vice President of Product Development for Borland. He is a 25 year veteran of the software industry and has held a variety of executive positions with such companies as BMC Software, IBM Tivoli, Dell and Vignette Software. He has extensive experience in enterprise software development, and managing internationally distributed teams. Chuck is currently responsible for overseeing the delivery of both the Borland Management Suite and the Borland Silk line of quality management products. Chuck has been involved in the Agile transformation at Borland since its onset.
