Automation Testing Part 1: Where Do I Start?
This is post 1 of 3 in the series “Automation Testing Series”
- Automation Testing Part 1: Where Do I Start?
- Automation Testing Part 2: API Testing Using Rest-Assured
- Automation Testing Part 3: Mobile Automation Testing with Appium
Automation testing should not be a burden for the development process, it should be part of it. In this first part of the automation testing blog series, we will explore a widespread process to choose where to start your automation efforts, tools that can be used and advantages they present. In the second post, I explored REST API testing, its importance, how to create tests using Rest-Assured and how to integrate them with your ongoing development efforts. In the last post, I cover mobile automation testing, specifically using Appium.
Ever wonder if there is a universal process for automation testing? Well, it turns out there is! No matter what industry you are in or the technologies used to build applications, there is a simple, three-tiered process you can follow and specific tools that can be used in each case. In this post, I will explain three tiers of testing and the results generated from applying these three steps, as well as compare some of the most common tools used for each stage.
As seen in the table above, there are many areas in which automation testing can be applied. It is important to follow the three stages (1 through 3) to maximize efficiency and coverage.
Tier 1 – Primary Functionality
What is the first step in your automation process? First, you need core functionality, your REST API architecture, and any static content that you need available for your applications to run as expected. Keep in mind this level of testing should be focused on the most important functionality of your applications and the correct way they should behave. Testing the expected behavior helps deliver a suite of functional tests that ensures the primary functions of the applications are working and give an overall sense of the state of your applications. It does not necessarily mean that your API, nor your applications, have been tested completely. However, it does ensure a good first step in a full automation process. Focus on delivering this first level of testing as soon as possible to benefit the whole development process. You can detect high-level issues quickly and make any abrupt code changes that might have affected other related, or non-related, areas. This will serve as a flag to development when developing or modifying their REST API or any other core, static functions.
Postman is a very commonly used tool not only within the QA community, but also in development. Postman is essentially an easy, user-friendly tool that tests out your API requests and makes assertions based on the expected responses with JavaScript. Postman has a feature called Collection Runner, which basically runs a series of these requests, in the order the user chooses, and applies the assertions in an automated fashion. Combine this with a test runner, like Newman, and you will now have a Jenkins job running a smoke test of your API architecture. Newman presents numerous other advantages to Postman, as it is a tool dedicated specifically to running these types of test and it runs using Java, a very common programming language.
RestAssured is a Java based DSL (Domain Specific Language) used with the sole purpose of testing out your REST API. In a similar way to Postman, any POST, GET, PUT, DELETE or PATCH request can be sent. Parameters, body and header options can also be inspected in both your requests and your responses. Then, assertions can be made. One great advantage of using RestAssured is the easy integration with Maven or Gradle, which in turn can help run with Jenkins.
Tier 2 – Engagement
Up next come some of the most exciting forms of automation testing that exist: UI/UX testing, dynamic content within your applications, Interrupts (testing your application’s behavior after common forms of interruption in a mobile device – like receiving a phone call or text), and behavior across different devices and/or specific OS features. In this level of testing, many efforts have been put in place over the past few years because companies greatly benefit from testing their UI. Whether it’s mobile applications or web based applications, there is always great value in testing what users interact with on a day-to-day basis. However, this still constitutes only the second level of testing. Because all of these UI’s depend on your core functionality, you will most likely execute a REST API architecture in the first tier of testing. As always, and particularly for UI/UX, there are many great tools for device and OS testing.
Selenium is one of the most popular software frameworks used for automated testing of web based applications. It is popular for two major reasons. First, it has its own DSL (Selenese) which allows for automated tests to be written in a great number of popular programming languages, such as Python, Java, C#, PHP and others. Second, Selenium has the ability to run tests against most modern web browsers on multiple operating systems. This makes it a very attractive option for companies with multiple applications on different platforms, as just one implementation of Selenium can be reused throughout different applications. Most popular test runners will execute Selenium, so feel free to choose among JUnit, NUnit, Gradle, Maven, and Ant PHPUnit.
Appium is an open source test automation framework specifically used to test mobile applications as well as mobile web pages on both Android and IOS platforms. It is a cross-browser platform, which allows testing between Android, IOS, and Web to be done under the same API. Using each vendor provided test framework, Appium runs on each platform instead so users do not have to run on a “specific” Appium runner. These are then wrapped in a client-specific server protocol called the JSON Wire Protocol. This protocol allows each client server to run their own individual code, making them automation libraries that allow you to modify any way you want.
Tier 3 – Device Impact
The third and last tier is all about your hardware. At this level, the automation process should focus on testing an application’s performance and the impact it generates on the device it is running on. It should also be checking for battery consumption and memory usage so users are able to monitor these two very important aspects and determine if there is a change needed back in the development stage for that specific area. This type of testing can be considered as fairly new and companies are not always aware this can be done. It is more traditional to test your UI rather than external or hardware dependencies. Let’s take a look at some tools.
BatWEB is a power monitoring tool that can precisely measure the amount of power and data usage consumed by an application. This tool can greatly benefit an application because it can monitor this consumption as new versions are released. If a newer version of the app is increasing battery or data usage, the development team can revisit and find where this unnecessary use is. For this hardware level test, one device is attached to a single Power Monitor. This monitor will then communicate through the same network and link directly to the device’s battery and data usage store to monitor its activity through the BatWEB (Application Monitor).
There are many tools that are used for memory testing purposes. However, it usually comes down to what programming language is being used. As opposed to the rest of the tools mentioned, memory testing is dependent on the language in which the software was created. For this we can present several options, one example is .NET Memory Profiler. The tests in the .NET Memory Profiler are normally written in C# and are run by the .NET default test runner NUnit, which makes the automation process very straightforward. For C or C++ apps we have Memcheck. Memcheck is able to detect memory-related issues and help avoid crashes or unexpected behavior.
Conclusions
Automation testing should not be a burden for the development process, it should be part of it. Managing the automation process using the three-tiered system presented above should greatly benefit the applications under test. In the next post, I will discuss how testing applications in a specific order will increase the degree of certainty the Quality Assurance team has and help deliver superior software by taking advantage of the tools presented. Stay tuned!
—
