How do companies face the challenge of setting up their infrastructure when they've just started agile development? One option is to move your infrastructure to the cloud. In this article, we help you decide what's the best fit for your team and project by addressing this issue head on.
Over the last several months, I networked with companies who had implemented agile (primarily smaller ones) and asked what some of their agile challenges were. Interestingly, I heard from several companies that getting their infrastructure set up was a challenge when trying to get agile going, as Agile had instant demands on the infrastructure. I immediately had a flashback to 1998; I had a similar experience when implementing scrum. The ScrumMaster had the developers coding in the first sprint (as is expected), yet there was no real infrastructure or CM system to speak of. What was there was clunky at best. Coding was quite the mess and after the first release came out, the next release had a lot of regression. This all revolved around the lack of focus on infrastructure. Interestingly enough, even after 10 years, the infrastructure challenges persist.
Given that when applying agile you start coding almost immediately, how do you get the infrastructure set up quickly? Even when you implement an iteration 0, it still does not leave you with much time. When you are in an established company, this may seem like a no brainer since it may appear that infrastructure is plentiful. However, even in established companies when resources are tight, getting infrastructure set up for a new product is challenging. When you are small company and have little to no funds, what are your options? By moving the development action to the beginning as agile does, you squeeze the time it takes to get your infrastructure set up. What do you do now?
This article focuses on this area and what I believe to be a gap in the coverage of topics for agile methods. While it is not an agile practice or method per se, this article attempts to support agile teams (particularly the new ones) by offering considerations and options for establishing infrastructure on new product lines where none previously existed.
Considering Your Options
Unlike the past, where organizations and product lines spent considerable time and large amounts of capital to establish local infrastructure, the future presents us with other options. They include (but are not limited to) renting infrastructure in the clouds (e.g., in the internet), owning the server and renting space in the clouds (hosting service provider), and owning everything locally and on-premises (within your company).
All of these options come with a set of advantages and disadvantages. While there are many factors in determining the best choice for an organization, there are two deciding factors. The first is how much budget exists for the product team. This includes the ability to purchase the infrastructure that is needed including servers, software, data center space, and staff. The second factor is how much control a product team or organization wants to have on their infrastructure. This includes how much perceived sensitivity there is to the product under development and data therein, and how much direct control is desired over access to the infrastructure. Based on your budget and ability to buy infrastructure and the control level you want, it will provide input to the more suitable approach.
With the advent of agile methods, there is now a third driving factor that may influence the choice. I have seen many agile projects, particularly those focused on brand-new product lines, struggle with getting their infrastructure up and running. Much of the reason is the time and effort that is needed to get infrastructure established far exceeds the time it takes to start development using an agile method, effectively the first iteration. With these new cloud infrastructure and co-location options, the "establish product infrastructure" game has been changed.
So, the question is, should you physically own infrastructure, rent space and own servers, or rent infrastructure in the clouds? It is important to better understand each option and assess your circumstances and needs before formally deciding on an approach.
Owning Server and Renting Space (aka, Co-Location as a Service)
Co-location as a service refers to renting space in the clouds by a service provider in order to physically host your server or servers. This is slightly different from cloud infrastructure where you rent a service that provides you access to infrastructure without necessarily knowing the details behind it.
You would utilize this option because you want to have the ownership and control of the box and content therein but do not have the budget to establish a physical datacenter on premises therefore minimizing expenses. This includes not having to hire and manage datacenter staff for to support this facility.
This approach can be particularly advantageous when you are a small company or start-up organization that has a website that you rely on for revenue but you are not in a position to establish a data center on-premises. Other advantages of a co-location service are that they typically provide scalability bandwidth based on your need, generator backup, and remote console access.
A distinct advantage for those who are agile proponents in a startup or small company, is that you do not have to spend time establishing a datacenter; instead, you can focus on the development of the project. This allows you to minimize datacenter infrastructure debt and scale to the space needs over time.
The biggest disadvantage is that there may be security and control concerns. Any servers you own are in someone else's datacenter. It is important to ensure that there is rigorous security access into this facility. However, because it is the primary business of this service provider, there will most likely be fairly extensive security access control. You also want to ensure the datacenter staff is experienced troubleshooters and quick to resolve connectivity and network performance problems. Finally, it is important to gauge the profitability of the co-location service provider you select. If this service provider goes out of business, it may be difficult to get your server out of the service provider datacenter. Understand how portable the data on the server is and ensure backups are readily available from another site, especially if this server is hosting production data.
Some companies that provide co-location services include ColoSpace and Colocation.com and amongst others. They all provide either strictly collocation services or co-location as part of their set of services. Since co-location is most advantageous when the service is within your location, it is best to perform a search on co-location and possibly your city or state.
Renting Infrastructure in the Clouds
Renting or leasing from a cloud infrastructure is a more recent concept and option that some organizations are utilizing where the infrastructure (servers, software, etc.) is effectively in the internet cloud. This service is sometimes known as cloud computing but have a variety of other names each with slightly different focuses and approaches including software as a service (SaaS) sometimes known as the application service provider (ASP) model, platform as a service (PaaS), application infrastructure provider (AIP). In this article we will call it "cloud infrastructure" where you have a hosting service provider that may provide hardware, tools, software, network, etc., that is available to subscribers. It follows a pay per use approach that may be seen as utility computing, since you are charged for the portion you are using.
The advantage of renting the cloud infrastructure is that it helps you minimize capital expenses and lower upfront costs since you do not have to buy hardware, software, and other components. Effectively the infrastructure (e.g., servers, software, etc.) becomes more of an operating cost instead of a capital expenditure. Another advantage of this approach is that you do not have to establish and manage the infrastructure. This includes not having to hire and manage IT staff for maintenance and upgrade work. However, with this being said, it is important to have people on your staff that knows how portable the data is that is being hosted and how easy (or challenging) it is to get it off the cloud should the service provider go out of business.
This infrastructure will include the same type of infrastructure and network components that a locally owned infrastructure would include and in some cases have servers that have some of the database and tools already installed. This approach can be particularly advantages when you are a start-up organization that has not yet released a product or are early in the product business. It may not be right for mission critical applications and in a situation where data is highly confidential. In general, the size and newness of company and product line is a factor where renting the cloud can be advantageous.
A distinct advantage for those who are agile proponents is that the cloud infrastructure provider approach enables users to only use what they need; this is directly in line with agile. This "Use what you need" approach minimizes infrastructure debt and allows the product team to adjust and scale to their need in a just-in-time manner.
The biggest disadvantage is that there may be security, control, and privacy concerns. Remember, when renting space in the cloud via a provider service, all of your product code, data, and documents will effectively be sitting on someone else's systems (e.g., the service or platform provider). It is important to ensure there is security related software, network, and protocol that support this service from authentication to access control. Interestingly enough because cloud service providers know this is a concern for many, security measures are often well managed in these environments. Finally, it is important to gauge the profitability of the cloud infrastructure service provider you select. If service provider goes out of business, it may be difficult to get your data converted and hosted by another provider. Understand how portable your data will be to another platform and ensure you have backups readily available in another location.
There are other disadvantages to this approach: the ability to customize the infrastructure and software therein is reduced, many of the integration issues may remain if there is a high degree of complexity in the working environments, and that there is less control over the skill level of the support staff meaning sometimes they will be experienced and quick troubleshooters and other times they will not.
When using this approach, it is important to ensure that the cloud infrastructure provider being used has prescribed service levels, established backup, recovery, and fail-over processes, is able to meet regulatory compliance standards, has the ability to allow you to scale up, provision, and instantiate quickly, and other factors for support, load balancing, and performance. Other factors that should be considered before selecting a cloud infrastructure provider is their financials such as their cash position, spend rate, profit margins, etc.
When using either the physically owned or rented cloud approach, you should have an application architecture and network diagram established which includes the platforms, operating environments, interfaces, services, locations (physical or cloud). The specifics of the design will help in defining the physical infrastructure to purchase or the cloud infrastructure to rent. With owning the physical infrastructure, there needs to be more in-house expertise in establishing the architecture, infrastructure, and network design. With renting from a service provider in the cloud, in most cases the provider can help you with this and walk you through the process of your design needs.
Some companies that provide cloud infrastructure include Force.com, Right Scale, Rackspace, GoGrid, and Caspio Bridge. Not all provide the exact same set of services, but they each provide aspects of cloud infrastructure services. Consider performing a search on "cloud infrastructure" for more service providers.
Owning On-Premises
Physically owning infrastructure is the traditional approach and is still the prevalent method of infrastructure for a product. For man,y it is perceived to be the safer, more secure approach and is better suited for organizations that want to have total control of their infrastructure. Owning the infrastructure also means that you have the capital to purchase the hardware, software, databases, network, and other components needed to host product development, testing, and production. Even when you consider physically owning infrastructure, you may have multiple sites of development. You have choices to place infrastructure at each site or share infrastructure at one site. In addition, owning your infrastructure comes with it the full breadth of administration, upgrades, maintenance, license management, and security that goes with supporting infrastructure. You also have to focus on performance, scalability, integration, and it implies that you hire staff to manage the infrastructure.
Owning on premises does not mean you have to set up the full infrastructure (servers, datacenter, etc.) each time a new product is envisioned for development. Organizations can use strategies similar to "in the clouds" options. This can include hosting a provisioning model, implementing an internal platform as a service (Paas), and sharing common support services (e.g., common configuration management or testing environments and tools).
It’s wise for organizations to share space in existing datacenters, much like the co-location as a service option. Also, some organizations may utilize renting in the local cloud optio,n meaning that the organization can establish infrastructure co-ops that can already has many of the infrastructure needs of a product team. The reality is that most product teams have fairly similar infrastructure needs (platform, development environment, CM tools, test environments, etc.). Organizations would be wise to deploy an infrastructure service model to make it quicker and easier for new product lines to get their infrastructure established. This can be particularly advantageous to new product lines utilizing agile methods.
Summary
As you consider each of these approaches, take a good look at the amount of your budget, need for control, and even the development methodology you are considering. These factors can help you better evaluate the best infrastructure model that is best for your needs. While lesser in importance, there are other inputs that can factor into your decision making to own or rent. These include the consideration of the various development support tools that may or may not be available including configuration management (CM), test, defect tracking. Their availability may lean you toward one model or the other. Ultimately, it is good to know that, while there are choices that keep you on the ground (on-premises infrastructure), you can also reach for the sky (cloud infrastructure!).
