Testing Innocence

Read the Code
I am a software tester, and I read production code.

I always have read production code. I learned to test software by running code in the debugger. Software developers I have known have argued that this makes me an ineffective tester, since I know how the sausage is made. Software testers I have known have argued that this makes me an ineffective tester, because I'm not behaving like a user would.

I argue that software testers who don't understand code are less effective testers, fail to improve the craft of testing, and probably are stifling their careers.

I started my career reading an odd language called TAL, the proprietary programming language for Tandem systems (now Hewlett Packard NonStop), but it was COBOL that really opened my eyes about how to test software by reading the code. For all its flaws, COBOL is a great language for beginners to learn. It is a procedural or imperative language with libraries and subroutines instead of classes and methods, so the action in COBOL tends to be easier to follow than in more modern languages.

COBOL is designed to handle enormous quantities of data while being readable to reasonable people. Data structures are explicit and data sizes are specified. COBOL is still in wide use in financial and telecom systems, because it remains an efficient way to process immense amounts of relatively simple data. For the tester, this is great: COBOL compilers and COBOL debuggers are incredibly effective, and it is possible to learn an immense amount about the code base using these simple tools.

I read some C back in those days as well. Without going into any detail: COBOL is easier.

Understand Networking
When I found a needed to begin writing my own code for testing purposes, interpreted languages were coming into their own. I set out to teach myself Perl, because Perl was the "Internet glue" at the time. I never became a very good Perl programmer, but I learned enough to write some really effective scripts to help me in my testing.

I'm most grateful to Perl for showing me network programming: how to talk to IP networks using various data formats over various connection protocols, all based on low-level IP sockets. It is hard to overestimate the value of this deep, low-level understanding when evaluating errors in networked systems. Also, don't underestimate the ability to surprise programmers by mocking the systems their code is supposed to talk to using an interpreted language like Perl.

Testing the Web
Although I learned my trade as a tester on mainframe systems, it was clear that the Web was the future, and I was deeply interested in Web-testing technologies. When Paul Rogers rewrote Brian Marick and Bret Pettichord's Web-testing tool based on Chris Morris's Ruby IEC library, he blew the roof off the joint. They called it "Watir", for Web Application Testing In Ruby.

Suddenly, we had a usable, maintainable, viable way to drive at least one browser for some really sophisticated testing. I consider Watir the project that really opened the Web-testing field. I switched from Perl to Ruby in order to use Watir, and I've been using Ruby ever since.

While Watir was starting to take off, Jason Huggins was writing an internal time-and-expense application for his job as the support guy at ThoughtWorks. He wanted to test it as he went along, and he discovered that he could open the app inside a frame and inject JavaScript into the frame in order to exercise all the elements on the page. This little ad-hoc testing framework became the tool called "Selenium" after ThoughtWorks released it as an open source project.

Watir and Selenium are the premier open source tools for browser-based testing. The code for Watir in particular is designed to be readable by beginners. If you haven't tried out Watir and Selenium yet, you should.

Reading Object-oriented Language is Hard-Use Logs
About this time, I got my first serious exposure to Java and C#. My experience with Ruby had given me some clues as to how good object-oriented code is constructed. But as an outsider-someone not involved in creating the particular code base-I find good OO code hard to read. If you, as a reader, surf through well-written OO code, it appears that nothing ever actually gets done; the code just shifts around actions and data, always getting more and more granular, until little tiny methods tucked away in odd corners actually have an effect on the system. If you have ever seen a stack trace from an error deep in an OO system, you know exactly what I mean.

So, the tester reading the source code needs to approach OO languages like Java or C# (or even Ruby) in a different way. Debuggers aren't as effective in systems that delegate operations the way well-written OO systems do.

Instead, well-written OO systems create detailed logs-incredibly detailed logs, if you tell them to. A tester who wants to look deep into the guts of the system he is testing should learn how to parse these logs, at least. A very serious tester also will understand how each log record is generated. A tester working in an OO system with poor logging will anticipate a large number of errors to be found.

So Who Needs to Write Code?
Someone who has spent his entire career becoming a great tester (while reading code) almost certainly will not be an excellent application developer. Application developers and technical testers on a software development team have the same relation that guitarists and saxophone players have in a jazz band: They recognize each other's expertise, and while one might be capable of doing the other one's job, it would be poorly done and would make no one happy (including the audience!).

A constructive attitude toward testers and code looks like this:

Developers: If you work with testers who want to read your code, welcome them. Show them around, give them the tour. Your job is not at risk. That tester is thinking about your code in a totally different way than you think about your code.

Testers: Don't be afraid to dig into the production code. You probably will not understand everything you see, so ask a question or two. The worst possible outcome is that you learn something. The best possible outcome is that you spot a problem no one has seen.

Finally, learn and be prepared to write code to help you do your job. You work with computers. The way to make computers "go zoom zoom" (as a developer friend of mine liked to say) is with code, so be able to read it and write it in the service of your work.