In order to stay competitive, IT companies integrate offshoring and outsourcing in their corporate strategies. When these strategies are implemented it is often found that offshoring is full of challenges and much more difficult than expected. This article suggests that the challenges of offshore software development can be met by using agile values, principles, and practices. First the article will present some of the challenges in offshore development, then it will investigate how agile principles and practices have helped offshore development in becoming more competitive by eliminating the challenges usually found in traditional offshore development. However due to the added distance it was found that agile development will run into new challenges. Some challenges, such as time zones, are hard to meet and will impede direct agile communication.
The material for this article was collected when doing research at the IT-University of Copenhagen in the year 2004/2005 and consists of 22 cases, scaling from five purely waterfall plan-driven cases (in no accordance with the four values from the agile manifesto) to seven agile cases (in accordance with the four values from the Agile Manifesto). The size of the projects varied in size from $20 million to $40 million US. Fieldwork in Europe and Russia consisted of face-to-face interviews, while interviews with American, Chinese, and Indian contributors were done by phone. Interviews were taped and written down for further analysis.
Taking software development offshore amplifies traditional challenges and creates new challenges, such as cultural issues and communication problems due to time zones. Some of the biggest challenges include communication, transfer of business understanding, poorly {sidebar id=1} defined projects that demand a lot of co-operation, time zones, high attrition rates offshore, imprecise or unknown (by the customer) requirements, insufficient customer involvement, cultural differences, customer/vendor bottlenecks, lack of common expectations, organizational differences, high attrition rates, and geographical distances. These challenges have to be attended in order to be successful offshore. The following sections investigate how agile practice can help offshore development meeting these challenges.
Face-to-face Communication
In all 22 cases (plan-driven as well as agile), face-to-face communication was stressed as a means to improve communication in the project. In plan-driven projects, this type of communication often took place in the beginning of the project. In big projects it also took place during development and in a few cases it even took place at the end of the project. On the agile projects, it took place throughout the whole project. Face-to-face communication took place in many ways such as at meetings where onshore product managers travelled offshore or when offshore requirement managers travelled onshore. But the biggest impact would be when a person from the offshore team worked onshore in order to absorb knowledge which could be brought back to the offshore developers. He would work as an enabler for the offshore developers, making sure that questions were answered instantly and that decisions were made onshore in order for the offshore team to progress quicker. Having people from the onshore team with business knowledge work offshore was also an enormous help. Many experienced that their naive faith in asynchronous communication tools were replaced by face-to-face meetings.
Face-to-face communication helped projects overcoming many challenges. Sometimes it would be easier to communicate face-to-face simply due to the fact that telephone lines between shores' often would be noisy and have echoes. This, combined with heavy accents, could make telephone conversations practically impossible. For others, it would be easier to align cultural and organizational differences when meeting face-to-face, since conversations would be much richer and informative and in that way counteract any misunderstandings. Direct face-to-face communication would make discussions and transfer of knowledge much easier. The instant feedback also made it possible for the onshore team to make sure that the offshore team had understood the business requirements and knowledge was transferred properly.
Talk Often to the Customer
Many cases were greatly benefited in their communicative setup by putting a lot of emphasis on good communication. But by putting emphasis on this challenge using some of the agile practices, such as trying to be as close to the customer possible, communicate daily, and delivering early, having weekly planning games or Scrums, projects were helped in overcoming some of the communicational issues normally found in offshore development.
Having a close relationship to the customer in order to understand business requirements was considered important in nearly all cases. However, this was not always possible due to long distances. Therefore, the offshore companies either recommended having an offshore person onsite, often a project manager or a business analyst. Onshore companies using offshore resources used a three-tier model where onshore developers or consultants could work closely with the onsite customer and transfer requirements and project knowledge on to the offshore developers.
Daily communication took a wide range of forms, ranging from daily face-to-face meetings with stakeholders to daily e-mails and chats with a customer representative. Daily synchronous communication was normally performed with a representative onshore first talking to the customer and then to the offshore team. Often the stakeholder was busy with the core business and would allocate a representative that would be the customer's proxy to the offshore development team. This person could help the offshore team to understand the business issues. This would improve the close relationship to the customer and would help against communicative bottlenecks. All cases found this approach helpful, whether they were agile or plan-driven.
Not only would the closeness to the customer help accelerate the transfer and understanding of the business requirements, it also helped align the organizations working together. The information-rich feedback cycles provided by daily communication would help coordinate differences in teams and culture that normally would impede project progress. Including daily communication in the project process made informational bottlenecks (one of the biggest challenges for offshore organizations) much smaller.
As experienced by a European participant, customer satisfaction often declines after project kick-off, as the customer feels he does not know where the project is going. An agile American participant therefore considered early deliveries to be extremely helpful in removing the customers' fear of offshore development. Many, especially Indian, offshore organizations would use a waterfall approach. This means the customer has to wait a long time after project quick-off to see the result. If something goes wrong, it goes really wrong and the customer has to settle with a wrong product or delays and increased expenses. Early deliveries give the customer the chance to correct and coordinate and participate in the project's success.
Another agile tool found helpful in this matter was using Scrums or planning games. They were often considered helpful in order to accommodate the complexities of the project and to get the customer to take an active part in the process. This approach helped overcome a lot of the challenges found in offshoring including bad communication, organizational and cultural differences, bottlenecks, establishing common processes, aligning common expectations, getting the customer involved, transferring business understanding offshore, and implementing projects that were not well-defined.
Small Teams and Projects
Some companies ran big offshore projects and expected big cost savings. One very large project ($40 million US) faced the challenge of being too big and too complex. The team started out with a traditional large-scale plan-driven approach using a waterfall method. However, it simply couldn't control the size and the complexity of the project by using this approach. It had to accommodate the use of subteams and practices from the XP methodology in the middle of the project in order to avoid project failure. This helped the team tremendously and it delivered the project successfully to the customer. Another plan-driven company, having an offshore development center in India, found that its most successful projects were those of smaller size.
Most cases found that keeping the projects and teams small helped communication. Cultural differences and issues were not as visible in smaller projects, whereas they were more distinct in bigger projects. In some cases it was easier to meet the financial expectations of the customer when the projects were kept small. Small- and medium-sized projects generally did better meeting cost savings than big projects.
Team Together In One Room and Pair Programming
One of the big challenges working with offshore organizations in India and China is the high attrition rates–often one will find a high fluctuation of employees in the offshore organization. To a knowledge-intensive discipline such as software development, this is a problem. However, many cases benefited from an agile approach to this issue. Having the team gathered in one room and using pair programming, they had the advantage of quick communication and transfer of knowledge between the team members offshore. Due to the high feedback cycles and quick exchange of information in the room, one project manager mentioned that their projects were never delayed, even when developers left in the middle of the project.
Do Requirements In Iterations/Use Short Iterations
As communication is impeded in offshore development, one has to focus on ways to improve communication and thereby the ability to transfer understanding. Many cases found that iterating through the requirements and implementing short iterations helped the offshore developers understand the onshore customer's business needs and logic.
Very rarely would requirements be in place at the start of the project. For plan-driven cases, this would be a serious problem, while those using an agile approach could accommodate the problem. In one case, a European participant believed that his project would have been a catastrophe if a Waterfall Model had been used when specifying the initial requirements. The case found that iterations helped the team understand and explore the requirements. Another European case (using a Russian offshore center) had only 50 percent of the requirements in place when the project started. This team would not have been able to deliver in time if it had required stable requirements before the team could start developing the project.
Another case had learned from its onshore projects that doing requirements in iterations was the only thing that worked, as it gave them a chance to focus on important requirements first and thereby give more value to their customer. This created conflict when its CMMi Level 5 offshore center preferred to have all requirements upfront. Onshore managers found that delivering detailed upfront requirements to this standard would simply be too expensive and would impede the economical benefit of using offshore resources.
A Russian case started out with a plan-driven approach, but experienced that the upfront requirements from the customer were too vague. It had to abandon the Waterfall Model and do the requirements in iterations in order to get hold of the complexity of the project.
The use of short iterations was another way to circumvent bad communication in offshore development. The size of short iteration would vary from one to four weeks. Some cases experienced that the offshore iteration would be a little longer compared to onshore iterations due to the informational overhead. Other cases kept onshore and offshore iterations the same size. One agile case aligned the size of the iteration with the need for feedback: the heavier feedback cycles needed, the shorter the iterations. Some found that if they had longer iterations, they would be disturbed in the development process all the time. By having short iteration of one week developers could work undisturbed for a week, while project managers were collecting new requirements for the weekly planning game.
The iterative framework was also found beneficial for the agile offshore vendors. By using an iterative approach, the customer could pay after each iteration. In this way the customer did not risk as much as when paying for the whole project. This made it easier for the customer to commit to the project and gave better business to the offshore vendor. To many, the idea of short feedback cycles and improved communication went hand-in-hand; this approach was supported by using short iterations.
Challenges Mapped With Agility
Figure 1 maps the agile practices and methods talked about in the previous sections with some of the challenges in offshore development. These challenges include:
- Communication
- Transfer of business understanding
- Poorly defined projects that demand a lot of cooperation
- Making requirements precise enough
- Getting the customer involved
- Cultural differences
- Handling the customer/vendor bottleneck
- Defining common expectations
- Time zones
- Unknown requirements by the customer
- Organizational differences
- High attrition rates
- Geographical distances
As seen from the table and as we learned from the previous sections in this article, agile development is helpful in meeting many offshore challenges. However, not all challenges in offshore development are helped by using agile methodologies. Some of the methods are very hard to implement in an offshore scenario. The team cannot be onsite with the customer and knowledge was transferred offshore either over the phone or by other electronic means, which is a step down in communicational richness. Even using the phone could be difficult. One issue is the time zone issue, but very often team members having English as a second language would speak with a very heavy accent. Combined with noisy telephones it would make it very difficult or impossible to transfer knowledge using direct communication tools, such as the telephone or videoconference equipment.
Summary
Agile methodologies contain a wealth of principles and practices. A number of those were found to be very helpful to offshore development. In addition plan-driven offshore cases have adapted and used agile principles with great success. The most important reason for this would be that the agile practices helped to improve communication and knowledge transfer, things that are heavily impeded in offshore development due to different time zones, geographical distance, and misunderstandings due to different cultural backgrounds.
Those cases with a strong agile background but new to offshore development found that they got competitive advantages such cost savings, access to resources and talent, or time-to-market benefits. Despite the fact that these cases had an advantage due to their agile experience, it was not always easy to be agile offshore. The price for improved agile communication could be high. Rich agile communication demanded the use of more onshore consultants onsite with the customer, more hotels, and more travel. These things have to be calculated into the equation before one adds agile values to the offshore project or takes the agile approach offshore. Still, integrating agile ideas into offshore development helps overcome some severe challenges and improves the ability of making complex projects offshore.
Whether engaging in a purely agile setup offshore or combining an agile methodology with a more traditional approach, participants in the investigation found that they benefited from using this communicative approach. However, despite the fact that agile value, principles and practices can help overcome some offshore development challenges, offshore development is still no silver bullet. Offshore development is ridden with challenges which must be attended in order to avoid project failures and must be used with care and attention in order to archive project success.
About the Author
Henrik Munkebo Christiansen(MSc in IT and Software Engineering) has five years' experience as a developer and IT consultant. He works for NNIT A/S in Denmark and has a keen interest in software processes and methodologies, especiallyfor global software development. His latest article is on communication in offshore software development is quot;Meeting the Challenge of Communication in Offshore Software Development,quot; Proceedings of the First International Conference on Software Engineering Approaches For Offshore and Outsourced Development (SEAFOOD), Zuuml;rich, 5-6 February 2007, Springer 2007.
