A Strategy for Testing SOA

by Randy Rice

September 2008

Abstract

Service-Oriented Architectures are getting a lot of attention as businesses look for ways to be more responsive and flexible in meeting customer needs through new and existing technology.
The adoption of SOA as both a technology and business strategy is on the increase. For the past few years, people have focused on the development aspects of SOA. However, the need for testing SOA has become increasingly apparent as companies are deploying SOA and are learning that they are different from other architectures in key ways.
In this article, we will explore why you need a strategy for testing SOA and what you need to consider in defining your SOA testing strategy.

Why a New Strategy is Needed for SOA

Traditionally, testing strategies lag behind development strategies by a matter of months. In the case of SOA, the tools have been around for some time, but the awareness of the need for SOA testing has been lagging.
Whenever a new technology emerges, one of the first things to do is to define what makes the testing of the technology unique. That’s exactly what a test strategy defines.

What to Consider in Your SOA Testing Strategy

The uniqueness of SOA is seen in how services are built and how they support business processes. Therefore, an SOA testing strategy needs to consider both structural and functional perspectives.

The Structural Perspective

This is the perspective of testing that focuses on the internals of the SOA. This can include the code used to create services as well as the architecture itself.
For many people, this has been the focus of their SOA testing. One reason for this is because as services are created, this is the first opportunity for the SOA to be tested. Although the structural testing perspective is very important, it is only one aspect of the SOA. We will explore the other types of testing later in this article.
One example of structural testing for SOA is examining the Web Services Description Language (WSDL) to learn how data elements are supplied to services. WSDL uses XML to describe network services as collections of communication endpoints capable of exchanging messages. By learning how to read and understand WSDL, testers can learn many important things to test. However, some testers may not be suited to spend time at the WSDL level, so having developers involved at this level of testing can be helpful.

The Functional Perspective

To ensure that the services support the business processes, functional testing is needed. Services can and should be tested individually, but the most critical type of testing is the integration testing of services and business processes. Unless services are tested in the context of business processes, you lack the confidence they will perform correctly when used to perform business functions.
An example of this type of testing is to model tests based on transactions and processes. This is normally called scenario-driven or process-driven testing. In this approach, business processes are described so that distinct scenarios can be identified. Services that support each scenario are also identified so they can be tested in relation to each other.

Other Types of Testing

When defining a test strategy, critical success factors define the risks associated with the technology or project and the types of testing to be performed that can help reduce the risks.
For SOA, some major critical success factors include:
Correctness – Do the services and architecture deliver correct results?
Performance – Does the SOA deliver results quickly and efficiently?
Security – Is the SOA adequately protected against external attacks? Is data protected to keep it from unauthorized users?
Interoperability – Do the services work together in the context of business processes to deliver correct results?
There are other success factors that you may choose to address, depending on your project. These include usability, maintainability, reliability and portability.

Tools

Tools add leverage to the testing of any technology, but SOA has unique characteristics that almost require the use of tools. The big issue in test tools for SOA is whether traditional test tools can be extended, or if totally new tools are needed.
Much depends on the type of tools you may currently have in place and their ability to handle SOA testing needs.
Before investing a lot of money and time in acquiring an “SOA specific” tool, perform some tests to make sure your existing tools can’t handle the job. People can have large investments in tools and the automated testware, so make sure you consider that investment before switching to a new tool set.

Tools are needed for SOA because:

1. They can provide a harness for accessing services that may otherwise not be easily tested. It may be impractical to test services because they often lack a user interface. This is called “headless testing” because there is no single access point to the services.
2. They can test faster than humans once the automation has been created.
3. They can provide precision in tests such as regression and performance testing.

Fortunately, SOA test tools appeared early in the emergence of SOA, so the tools that are currently on the market have had time to mature.

People

Testing is a very human-based activity. When you list the major problems people face in testing any software, the great majority are human in nature. While SOA has a major technical element, you will still need to consider who will plan, perform and evaluate the test.
Developers have a great perspective on structural testing, but may not be the most willing participants. However, with some encouragement, tools and management support for testing, developers can test their own work.
Stakeholders, especially end-users, can add the business perspective to the test. The issue with end-user testing is to get enough of their time to adequately focus on the test.
Testers can bring a wealth of testing knowledge to the project. They can adapt previously defined tests for SOA and also can help acquire or adapt the right tools for the job. You may need to get specialists involved in tests, such as performance and security testing.

Controlling the SOA Test Environment

Your test is only as good as your test environment. If you can’t identify what is in the test environment, such as the services, you will not be able trust the test results. With SOAs, governance is needed to control when services are created or modified, and when they should be placed into the test SOA environment.
Unfortunately, people often fail to appreciate the importance of test environment control until they experience a failure due to an incorrect test.

Conclusion

SOA brings new test considerations, but the good news is that many of the techniques can be adapted from past technologies. Effective testing is a matter of getting the right balance of people, processes and tools, all working together in an integrated test environment. SOA is no different. By taking time at the start of your first SOA project to define the uniqueness of the technology, you can approach the project knowing that the major points have been considered. This will ensure you have not only planned a good test, but have planned the right test.