End-to-End Telecom Software Testing in Action

This Industry Viewpoint was authored by Erik Fogg, Chief Operating Officer at ProdPerfect.

Telecom software is relied on by millions of companies and individuals around the globe and is expected to handle large loads of bandwidth at a rapid pace while maintaining data integrity. Telecommunication networks form an integral part of the wider global market, so ensuring that the software provided by telecoms companies works correctly before being deployed is of paramount importance. Telecoms have a much higher-than-normal burden to test rigorously: questions about reliability in the market would crush any telecom’s value. What are the ways that this testing is conducted, and how can companies be sure their products work as expected? Read on to learn about the processes involved in E2E testing in telecom software.

The Purpose of E2E Testing

End-to-end (E2E) testing is used to test that telecom software (as well as any software-enabled system) works at all points in the process of making a call, from the sender to the receiver, without error or data loss. This is particularly important in the telecoms industry, where providers are expected to ensure that vast amounts of data is transmitted in real or near-real-time without undue delay or errors, while also communicating with many different networks across different protocols and environments.

While individual components and modules in software can be tested in isolation – and in many software contexts such lower-level testing is “good enough”- in practice telecom software must communicate with many different hardware, software and networks components. This transfer of commands and data must work seamlessly, but it is hard to discover errors in this process in isolation. E2E testing fills this gap, to ensure that the entire process works flawlessly, from start to finish.

What Does End-to-End Testing Mean for Telecoms?

E2E software testing for telecom applications focuses on the end-user experience and tests real-user scenarios that are likely to occur. Tests are used to determine how well it handles a range of cases representing real-world user actions that may occur in the duration of a call. This means measuring the call success rate under a range of test actions, which may include:

• Call transfers
• Putting calls of hold
• Emergency calling
• Roaming
• Redirecting to voicemail

Protocol Testing

Protocol testing is used to test that different software components send and receive data correctly according to a defined protocol. For telecom software, there are many different protocols for different parts of a network. Protocol testing involves dealing with how data is handled across these different protocols.

These are generally divided into routed protocols and routing protocols. Routed protocols, such as IP and IPX, deal with sending data from one network to another. Routing protocols, such as RIP and IGRP, define routes for routers and help direct traffic flow.

Protocol testing generally involves testing data for correctness, latency and bandwidth. Correctness and latency tests ensure that data arrives on-time and without data loss. Bandwidth tests calculate how many packets can be sent and handled per second.

Integration Testing

Integration testing is a necessity for telecom software, which regularly involves transferring data between different software components and across different protocols. Integration testing tests the relationship between these different components to ensure that software that works in isolation does not produce errors when integrated as part of a larger process.

Integration testing can be achieved through two different approaches, either by ‘big bang’ testing or incremental testing. Big bang testing advocates for connecting all components and testing them as a whole. This approach makes debugging issues more difficult, as the relationships between components are more opaque, but it is faster for smaller systems and maximizes the likelihood that errors will in fact be caught.

Incremental testing involves testing a small number of logically connected components before slowly integrating and testing more components over time. This takes longer but makes it easier to spot precisely where and when errors occur. This can be approached from either a top-down or bottom-up approach, testing either the higher-level or lower-level components first, respectively.

System Testing

System testing generally follows on from integration testing. System testing looks at the system as a whole and focuses on the E2E process. It is based around testing the external user experience with the software, rather than looking at the internal software components. For telecom software, system testing typically involves the use of physical test hardware that simulates interactions with the software. These can be controlled by a CLI or GUI interface to produce responses that mimic the responses of real hardware. For telecom applications (such as web applications), this System Testing is identical to the conventional browser-level E2E testing that other software industries perform, often using Selenium or a similar testing framework.

Challenges of E2E Testing

The software testing process is often subject to the competing pressures of covering as many test cases as possible with the resources available. These pressures are also present with E2E telecom software testing, so test coverage is rarely completely comprehensive.

Telecom software also has some unique challenges for testing. Data accuracy and data integrity are of paramount importance for telecom networks, as packet loss can result in a noticeable drop in quality of service for the end-user at best, or disrupt the user experience completely at worst. Having software that handles data loss gracefully is therefore very important for telecom software.

Scalability is also important for E2E testing, as it can be difficult to simulate real-world bandwidth loads in a test environment that also coordinates testing processes to run in the same sequence of execution as they would in a live environment.

Best Practices in E2E Testing Automation

Using a rigorous test automation framework is a must for E2E testing. This framework should be capable of adapting to a wide range of environments and configuration setups. Testing software should be capable of communicating with hardware and network components via CLI or web-based GUI commands.

A test automation framework is only as good as the tests it contains. The focus of E2E tests should always be on the end-user stories. The focus should be on working features over correct implementations. In order to determine priorities in testing, risk analysis should be performed to highlight the features that are higher risk, with the end-user experience in mind.

E2E testing should take place after other software components have been tested in isolation. This is because E2E testing involves communicating with many different devices and networks, which makes isolating the location of an error in E2E testing very difficult. Making sure everything else works first before initiating E2E testing saves a lot of debugging time when errors do occur.

If you haven't already, please take our Reader Survey! Just 3 questions to help us better understand who is reading Telecom Ramblings so we can serve you better!

Categories: Industry Viewpoint