Sponsored Article: Enhancing Your Selenium Testing

Test automation is a must nowadays for software companies wanting to stay competitive. Augmenting manual testing efforts with automated solutions is essential.

When it comes to deciding on a tool for their test automation, most teams will consider options based on a variety of factors: cost, features, ease of use based on the team’s technical abilities, the type of testing the team wants to automate, how the tool aligns with existing organizational standards, and more.

Selenium offers free, open source software; is highly extensible and can run across multiple browsers, platforms, and operating systems; allows for running tests in parallel; and lets testers create tests without having to learn a scripting language, so it has been the choice for many teams.

Despite the Selenium suite’s wide-scale adoption in organizations of varying sizes and across many industries, it comes with some challenges. As with any open source tool, there’s no official user support, and the community element can complicate use. Its features are also not as rich as some other offerings, and it has problems with maintainability and scalability.

We talked to some experts in the test automation industry about Selenium’s reign as the tool of choice for UI testing, whether that crown is warranted, and what they think is important for teams to focus on when it comes to their test automation efforts.

Why is the industry standardizing on Selenium for web UI testing?

“Selenium is the most popular tool because it has such a large community,” said Adam Auerbach, vice president of quality engineering at EPAM Systems. “There are frameworks and information out there for free, and it’s very stable.”

He said there are other tools on the market that are starting to eat into Selenium’s presence, but Selenium is still one of the best examples of an open source tool and how it can be successful.

“Selenium is the gold standard,” Auerbach said.

Max Saperstone, director of test and automation at Coveros, agreed about its popularity.

“I wouldn’t necessarily say Selenium is a standard for web UI testing, but it absolutely has the greatest market share,” he said. “One of the reasons is Selenium has been around for 15 years. It’s been tested, and it’s trusted. It’s also free and open source, which makes it really valuable.”

Saperstone pointed out that one of the other big things that has really propelled Selenium’s success is the fact that there are many enterprise-level organizations using it for their UI testing and throwing their support behind it, including Google, IBM, and LinkedIn.

Despite adoption from large companies, Paul Grizzaffi, principal automation architect at Magenic, thinks the price is what drives the most interest.

“People like the reality or illusion of free,” he said. “The allure of free is a big powerful draw. Also, everyone else is using it, so we should, too. That’s not to say Selenium isn’t worthwhile; I like Selenium. I use it a lot.”

Still, Grizzaffi said, people often are willing to spend money on designer tools because they feel like the open source ones aren’t meeting their needs.

What else should organizations be doing beyond UI testing?

Saperstone brought to mind the test pyramid popularized by Mike Cohn and Martin Fowler.

“If you’re thinking about your testing pyramid, you have unit tests at the bottom, so you should have the most of those,” he said. “You have your API testing above that, and you should have not quite as many of those as your unit testing but more than your UI. If you have those other quicker tests, you can use those to complement and speed up your UI tests.

“If I have a UI test that breaks, I don’t know if it’s a problem with the UI, the API sitting below it, or the code underneath. But if I know that my unit tests work and I know that my API tests work, I know it has to be at the UI level, which gives me a lot more confidence in my UI tests.”

Auerbach agreed about emphasizing the other levels of the test pyramid.

“You need to have good unit testing. You need to have quality gates in the process to enforce good unit testing and coverage,” he said. “You need to be doing API-level tests, because they’re fast, and you can do a large amount of permutations through the API.”

How should teams quantify the value of testing?

“The value of testing is based off of the impacts to IT and the business,” Auerbach said. “The value of testing is around how effective and efficient the testing progress is so that we’re able to go to market quickly and we don’t have to spend lots of extra time on rework. The value of testing is that we’re able to give developers fast feedback, and it’s not hindering or slowing us down, and we’re not getting dinged with production issues or complaints from the customers, and thus have to spend time on rework.”

Grizzaffi agreed about assigning business value to testing.

“We should align the value of testing and automation with a business goal. We have a business goal, and we have to measure it with a certain number of quality metrics by a certain date,” he said. “How is our testing strategy and implementation and results we learn from that going to help us meet that goal? If we can’t answer that, any kind of metrics outside of that are vanity metrics. They might be interesting, but what value do they add to the business goals?”

Saperstone said that the value testing brings is actionable.

“The whole point of testing your system is to try to understand the risk that exists for what happens if the application goes out into the world and doesn’t work. Is there a high risk to that, or a low risk to that?” he asked. “If you’re not going to do anything with the test results, you should not test. If, on the other hand, you’re going to look at the results and make an informed judgment based on the high- or medium-critical things, that’s the value.”

So you have chosen to go with Selenium—now what?

“One of the biggest issues in doing test automation in general is the mindset shift,” Saperstone said. “I’m no longer doing the same thing I was doing with manual testing.”

However, he said he often sees organizations take their manual tests and try to turn them into automated tests; that’s not what you want to do.

Auerbach also cautioned against jumping right into test automation without proper training and forethought.

“You have to treat test automation like any other engineering project. You can’t just start blindly using Selenium without any thought,” he said. “You have to have a well thought out, architected framework. You need to make sure that you follow good coding practices. You don’t want to create a maintenance nightmare.”

What are the top challenges when doing web UI testing with Selenium?

“The first main challenge I see with using Selenium for web UI testing is people don’t know what Selenium is or what it does,” Grizzaffi said. “Selenium is a tool, but it’s not necessarily a testing tool the same way you would use like of a UFT or TestComplete. It drives the browser and can go and interrogate the DOM, but assertions are not built in, reporting is not built in, the test framework is not built in.

“That makes Selenium hugely powerful, because you can take it and do something non-test case–based to help you with other types of testing. But that means you either have to develop the other stuff, like assertions or reporting, or you have to use something that’s out there already. Most people don’t know that necessarily when they usually start using Selenium.”

Auerbach talked again about going in with a good start.

“When we don’t have a stable UI automation suite of tools and frameworks, then ultimately, that becomes a bottleneck, so you will slow down the process. You might not even be able to keep up with development.

“The ever-changing UI is challenging,” he added. “When you’re doing automation through the UI, your tests have to constantly be updated. If you have a thousand-plus tests, and many of them fail because of specific UI changes, now that represents rework that you’re going to have spend a considerable amount of time doing.”

Solving UI testing challenges and making Selenium more maintainable

To address some of the issues testers have with using Selenium alone, Parasoft released a new product in October, Parasoft Selenic. It works with existing Selenium tests, providing self-healing at runtime and AI-powered recommendations post-execution.

“We decided to focus on how could we bring AI to bear to help solve the challenges people were facing in test automation,” said Mark Lambert, Parasoft’s VP of products. “When we spoke to folks it became quite clear that Selenium was the de facto standard in the industry. It’s open source, and it’s community-supported in a fantastic way. If you have a problem with Selenium, you do a quick Google search and you’ll find the piece of code that will get you unstuck very quickly.”

However, Lambert said, there were a few key challenges that many people were facing with creating their UI automation, and Selenium itself wasn’t addressing those.

“The first challenge was the maintainability of the test scripts,” he said. “Because the website is your primary interface tool for the end user, you’re constantly refreshing it, changing the style, tweaking the workflows, and trying to give a really exceptional customer experience. The trouble is a small change in the UI can have a massive impact in your test automation. Building tests which are resilient, not only to the internal stability of the application, but also the UI, is critical.

“The second challenge is related to scaling of the test suites. Once I’ve got my tests going and I’ve started to create hundreds or thousands of test cases, the longer my tests take to run. When you’re talking about a hundred tests, it might not be a big deal, but when you get up into the thousands, now you might be waiting overnight or sometimes over the weekend before you get true feedback from your CI/CD pipeline. The whole point of DevOps is you want to increase the cadence of feedback you get about the quality of the application.

“The third challenge, which was interesting, was the challenge of trying to get going with Selenium. A lot of test automation engineers or manual testers who were trying to get into automation using Selenium saw it as having a steep learning curve, because it’s Java code under the hood. The thing is, the Java code isn’t that complicated. You’re not building a multi-layered Java microservice application. You’re writing a test script in Java code. It’s not rocket science. We wanted to develop a way to help organizations get over that learning curve.”

Parasoft Selenic was designed to address those three challenges.

“We have an AI engine embedded into the execution of an organization’s existing Selenium scripts. That AI engine monitors how the tests are being run and what the tests are doing so that when it detects a change in the application, it automatically recommends the change you should make it your test, but then will also apply that at run time,” Lambert explained. “It won’t actually change the test for you. It tells you that it applied a self-healing repair at run time, and if you agree with me, you should update your test, and this is how you should update your test.

“As the tests are executing, we’re monitoring what parts of the application are being exercised, then we cross-correlate that test execution with changes we detect in the Java binary. By cross-correlating these two things, we’re able to optimize and reduce the number of tests that need to be executed as part of the CI/CD pipeline to get immediate feedback,” he said. “I’m not saying don’t ever execute all your tests, but the whole point is to get quicker feedback.”

Selenium tests can suffer from long execution times, maintainability, and scalability, due to their complex and brittle nature and inability to automatically adapt to changes in the application. The AI powering Parasoft Selenic changes that.

“We implemented functionality that records the UI interactions and translates those to code. There are a whole bunch of free or open source tools that are focused on creation. But we’ve taken a different approach,” Lambert said. “We’re going to record the actions, translate them to the Page Object Model so that you can make your tests more maintainable, then we’re going to give you the code with no proprietary Parasoft code inside of it to get you started.”

Rather than being another proprietary "Selenium-powered" platform, Parasoft Selenic works with your regular, existing Selenium tests.

“We want to deliver a product that somebody wants to use, and not a product that somebody is forced to use,” Lambert said. “By that I mean, we plug right into what you’re doing today; if you don’t like it, we plug right back out. The tests we capture with our recorder are pure Selenium, meaning I have not injected any proprietary code in there, which means you can record and walk away.”

If UI testing with Selenium has become a bottleneck in your DevOps process, adding Parasoft Selenic could help speed up your delivery process—and you can keep your current Selenium practices and test environment.

For more information, contact Parasoft about a free trial of Parasoft Selenic.