Continuous Integration and Testing

CITCON North America 2009 was held last weekend in Minneapolis. I'm on the organizing committee, but was unable to attend this year due to a conflict. I was bummed, because I found the one last year to be one of the most valuable conferences I've ever attended.

The biggest feedback received at the conference is that the participants wished there had been more testers, or at least, more "tester-programmers." Most attendees were programmers or "programmer-testers," so there weren't as many testing-oriented sessions--especially business-facing-type testing.

We've been discussing this on the CITCON mailing list ([email protected]). I've also been polling on Twitter and agile-testing ([email protected]) to see if any testers had seen the announcement to CITCON, considered attending, and decided not to. Why wouldn't they be interested in a free, open-space conference about testing?

Economics came up a lot—many testers said they couldn't afford to go to any conferences, their employers have no budget for that, and they don't have resources to pay for themselves. Time was another factor. However, these economic and time issues apply to everyone, not only testers.

Some people suggested that continuous integration (CI) doesn't really provide value for testers, only for developers, and maybe it would help if the conference were named Testing and Continuous Integration instead. I'm not opposed to changing the name (although TCICON doesn't roll off the tongue quite as well as CITCON, which is pronounced with two hard Cs), but I'm disturbed at the suggestion that CI isn't important to testers.

Without a continuous integration and build process, how can testers possibly get stable code builds to deploy for testing? What if you deploy a build, only to find there's a "test-stopper" bug, and you can't go back to a previous stable build?

If there's no CI process running a safety net of unit tests, testers will likely spend all their time finding and reporting unit-level bugs and have no time for the most important exploratory testing, functional and non-functional testing.

There was a time not so long ago when a lot of teams worked without source code control systems. Now it's impossible to imagine working without good source control and versioning ability. I think all testers would agree that this is absolutely necessary.

Continuous integration is undergoing the same transformation. It's not a "nice to have." If your team has no automated continuous integration and build process, stop everything you're doing and get one in place. It's not that hard, it doesn't take long, and there are plenty of excellent open source and vendor tools to support it. Without CI, you're going to waste a lot of time and never get to the most important types of testing such as exploratory testing.

Don't know much about CI? The CITCON mailing list is a good place to start. There are free services such as the CI Guys. Google "continuous integration"; Martin Fowler's excellent explanation of it comes up first, and that's a great place to start.

No matter what your role is on a software development team, you need to know why CI is extremely valuable to you and your team's success.