The Scalability Index

PushToTest's methodology for checking an information system's scalability is presented in a Scalability Index chart that looks like the following.

This shows the system's scalability as measure by throughput. In other words, how many transactions can the system handle as it receives more and more requests. The test measured how many transactions-per-second are being handled at 4 levels of concurrent requests.

A perfect information system is be able to keep handling requests at the same speed regardless of how many requests come in at the same time. Charting a perfect system's scalability shows transactions-per-second rates increasing in equal proportion to the number of requests sent its way. For example, in the perfect system if you made 100 concurrent requests in a 10 second interval and the system responded to each of the requests in 2 seconds, then the perfect system should let you make 200 concurrent requests in a 10 second interval with each request still taking 2 seconds. The resulting chart of the perfect system would look like the following.

It just keeps going up-and-to-the-right in equal proportion to the number of requests made. At each measured level of concurrent requests, the system is handling requests at a measured number of transactions-per-second. When you increase the number of concurrent requests the system handles more requests and so the overall average number of transactions completed increases in equal proportion. For example, at 100 concurrent requests the system handles 1000 requests in a 10 second period (100 transactions-per-second) and at 200 concurrent requests the system handles 2000 requests in the same 10 second period (200 transactions-per-second.) That's perfect scalability. The more requests you throw at the system does not slow down the proportional speed the system responds to the requests.

A typical system that hits a bottleneck would look like the following.

This chart shows that as the system receives larger numbers of concurrent requests it slows down on responding to all requests. If you kept going past 400 concurrent requests in this system eventually the system would reach 0 transactions-per-second. Many systems PushToTest checks for scalability have this problem. The Scalability Index shows systems managers how to plan the capacity of their systems against the desired throughput to make user's happy. The Scalability Index also helps developers to understand the impact their design and coding decisions have on performance.

In my experience, the more common situation is shown in the following chart.

In the first three columns of the above chart, the system is able to handle increasing levels of concurrent requests up until a certain point. In the fourth column the system is showing that it has hit an upper limit to being able to handle transactions. Many times this is caused by a database indexing problem, a data cache that is now full, or a network connection that is saturated.

Additional information on the Scalability Index is available in Java Testing and Design: From Unit Tests To Automated Web Testing by Frank Cohen (Prentice Hall 2004).

If you find yourself with questions about the Scalability Index, please contact PushToTest at info@pushtotest.com.

(c) PushToTest 2004. All rights reserved. While this article is the copyright of PushToTest we encourage you to share this article in its entirety with your friends and associates.