Selecting a testing tool is hard work. If you look on vendor websites, you'll get marketing material promising the world and then a call from their sales staff. If you look on forums, you'll mostly get people trying to solve problems relating to their product: "Why won't SuperUITester++ click Save on my Flash application when there are 4 other modals open?"
Let’s take a look at why you might choose Selenium as your UI testing tool with information based on real experience with real software projects—rather than a marketing page.
Why the UI
There are lots of different places you can begin testing software. I like to think of them as different layers in a product, just like sedimentary layers archaeologists dig through when unearthing ancient cities. You want to use the most appropriate tool for each layer of software you are looking at: smaller units, seeing how methods work together, fully formed APIs, simple scenarios in the UI, and complex product usage. All the layers are important; the question is how much of each you want.
The API and everything below that will give you a feel for code quality and some basic functionality. Testing the UI will help you know things from a different perspective: the user’s.
Why Selenium
The WebDriver object triggers real events in the browser: mouse clicks, button clicks, entering text, and events from the keyboard. Think of each step as a building block. Stacked together, they can enable a technical team to do some powerful things. Here are a few of the more common reasons for using WebDriver.
Building Automated Checks
Probably the most common reason people elect to use the Selenium suite of tools is to drive a specific set of commands and check status—to see if the user is logged in, if the book goes into the shopping cart, or if a transaction processes.
This includes checks to make sure buttons or labels are present on a page, data that you created saved correctly, procedural aspects of the software under test work accurately, or any number of other attributes of your product.
Put simply, you'll use WebDriver to ask questions you would normally be able to answer with a yes or a no, and WebDriver will kick out a report with the number of assertions, the ones passing and failing, and where the errors occur.
Once you have finished building your checks, you can run them once or many times and use them as a tool to discover problems, or even unexpected changes, between builds.
The questions you ask in the scripts are binary, but usually the process of writing and running them a few times will uncover important bugs that the scripts aren't even looking for.
Taking Over Long-Running Tasks
Repetitive and very long tasks are expensive (in terms of time, which means money) and can be difficult for people to get through. Humans lose focus and concentration over time or lose interest in the goal, which leads to mistakes. Scripts don't make mistakes; they do the exact same thing each time. Normally, when tasks like this are needed, the most junior person on the team is given instruction, access to the hardware they need, and the time to complete the task. This unloads the work from more senior people, but some people still occasionally end up doing work that a program may be better suited for.
If you need to create hundreds of slightly different users or gigabytes of data in a realistic way, Selenium can help.
Management Said So
Sometimes, you don't have a choice in the tooling you use. Your manager (or your manager’s manager) may have heard of this new tool that can work magic. Because it’s magic, of course you should be using that new tool to solve the testing bottleneck for your project.
Maybe your manager’s hunch is right and Selenium can help solve some testing problems. Here are a few questions you can ask to make sure:
- What problem are we trying to solve?
- Are your expectations appropriate for the work this tool can facilitate?
- Is this tool appropriate for your product and how your teams work?
- Are we ready to build and support another software project?
These questions will help you figure out the motivations behind the request. Knowing that will help set reasonable expectations for what you can do and help frame your achievements as a success.
What about Another Tool?
There are lots of tools available for testing the UI, including other free options besides Selenium. I'm sure you can list five without even bothering with a Google search.
So, why should you choose this tool when there are so many others available?
Growing Browser Support
The W3C WebDriver specification will be finalized soon, and with that will come WebDriver implementations from the browser makers. What that means to you is that you don't have to worry about browser upgrades breaking the tests you spent so much time making. A specification means that vendors will ship updates of their WebDriver implementation along with new browser versions.
A Large Support System
Selenium is already one of the most popular testing frameworks in the world. It also happens to be open source. This means all changes that happen in the project happen because of community involvement. This community involvement also applies to getting help for problems. Rather than purchasing maintenance contracts and submitting a support ticket, you can go to any of a number of forums, or even straight to the developers if you choose.
It ’s Fully Featured
Selenium development follows the needs and desires of the community that uses it. You can easily see the newest features available and what is coming soon by looking at the code repo. If making a feature request isn't fast enough, you can always write the code and submit a pull request for yourself.
Something to Watch Out For
There is at least one downside to the large network of support provided by the open source community: There are lots of users all over the globe submitting feature requests and bug reports to only a few developers. The chances of getting one of your requests in the project quickly are, well, pretty small. Commercial software may not always perform better or be more fully featured, but they do have support teams at bay.
Tools and frameworks can be difficult and time-consuming to switch from once you have made an investment, so it's important to consider what is important to your project first.
I hope this outline of the benefits to Selenium helps.
User Comments
Hi Justin,
I enjoyed reading your article.
Selenium is in my opinion a great open source tool for Web UI testing. It does currently have a major drawback: newer browser versions not working with older versions of Selenium and vice versa, this causes nightmares for us when for example firefox automatically updates on testers machines and suddenly the tests stop working. I'm delighted to hear that the W3C is working on a spec for the WebDriver! Do you know if the the browser makers have into bought writing and taking control of the web driver implementation for their respective browser?
Hey there!
For realism of test data, a browser driven way for creating users seems like a good idea. However, I would argue that most of the time a http request based approach can be just as realistic, less flaky and much, much faster.
I would thus offer something like the Python requests library or even JMeter as a valid alternative to selenium for this task.
You can say it again!
UI automation scope should be minimized since it is very expensive (to create, to run and to maintain) and less productive comparing with other approaches (such as http api call)
I totally agree. The way web applications are developed now, they are very service (api) heavy. Makes sense since web services aren't impacted by the browser. The way I've seen test automation change in the last 20 years, it has shifted from automating functionality to automating service testing. It just makes sense.
Services, once fleshed out, very rarely change other than adding methods so maintenance is at a minimum. Once they are all scripted in a tool like SoapUI Pro, they can be put together to test any flow you want and run as part of the build process or scheduled to run any time of the day or night. I can run a large suite with many, many assertions in about 2 hours whereas the same tests run through the UI take 12 hours or more. The debugging and updating can be an all day job.
I'm not saying that UI tests no longer need to be automated, but I think the value of UI automation has changed and should be assesed to determine exactly what needs to be automated and what can be executed manually.
I think a tremendoius amount of time and money can be saved by taking a fresh look at the automation plan.
I die a little inside every time I hear a test manager say "I expect 100% automation!" LOL
Well, we all know about slenium testing is the best open source functional web testing tool. Here, Justin Rohrman you are written very clarifiying details which is is based on our real practise work.
Main reason of using selenium as automation tool is
layers archaeologists dig through when unearthing ancient cities. You want to use the most appropriate tool for each layer of software you are looking at: smaller units, seeing how methods work together, fully formed APIs, simple scenarios in the UI, and complex product usage. All the layers are important; the question is how much of each you want.