Little’s Law in Performance Testing
Problem Statement
Performance testing requires the development of a workload model that takes into account user load, the number of requests issued per hour, and page response time in order to emulate the real-world scenario. Little’s law makes adjustments to these three indicators in order to achieve the necessary server load. Many times, when a performance tester applies Little’s rule (as it specifies) to create an accurate workload model, he discovers some unexpected outcomes and must then conduct manual calculations in reverse or that need complicated reasoning to produce the appropriate server load. This article addresses the following as a remedy to this issue:
- What mistake did you do while calculating the workload model metrics using Little’s law?
- How to correct the mistake?
- What is the new (extended) form of Little’s law which will calculate the correct workload without any manual (on-paper) calculation?
Problem Description
Performance testing is used to assess how well an application or system performs in terms of its ability to sustainably handle user demand and the responsiveness of the server. Every customer is concerned about the application’s user experience in today’s environment. According to the study, if a website page doesn’t reply in less than 10 seconds, the visitor either closes the page or leaves, which has a direct influence on the company’s income. Companies are placing a strong emphasis on responsiveness and the website’s capacity to handle user traffic in order to satisfy user expectations.
User load: The number of virtual users that are placed on the server during testing.
Response Time: The amount of time the system needs to react to a user request or input. In addition, network delay is included.
The user’s journey around the website, also known as iteration Home Page -> Login -> Product Search -> Product Order -> User activity on a page is referred to as the logout transaction.
Think Time: The amount of time a genuine user spends on a website before making a request.
Pacing: The extra time a performance tester adds between two user journeys to reach the target load.
John Little wrote a paper in 1954 that used a novel law to explain the queueing idea. Little’s Law is the name given to it afterward. He created three fundamental terms: the typical customer count, the typical customer arrival rate, and the typical customer dwell time.
Little’s Law Statement: The long-term average number L of customers in a stationary system is equal to the long-term average effective arrival rate λ multiplied by the average time W that a customer spends in the system. Expressed algebraically the law is:
L = λ * W
To imitate real-world load in the testing environment, the same formula was approved for use in performance testing. The three fundamental performance testing parameters of number of users, transaction rate, and reaction time are related by the formula above. Hence,
Little’s Law in Performance Testing Term: The number of users (U) active on an application is equal to the rate of transactions (T) sent by the users multiplied by the average response time (R).
So, the formula becomes:
U = T * R
User Load = Transactions per second * Response Time
The above-mentioned formula based on Little’s law looks simple so what are the issues faced by a performance tester during test scenario creation and what is the need to make this formula more simplified and accurate?
Let’s consider an example to understand the missing points. A client has given the requirement of 100 users which hits the servers at the rate of 5 transactions per second and the expected response time for each transaction is 2 seconds. Now, put these figures into the above formula. We have 100 as a user load, 5 as transactions per second, and 2 seconds as a response time. Hence,
=> 100 = 5 X 2
=> 100 = 10 ………. (1)
Equation (1) does not satisfy the relationship between the metrics. Then what is missing?
To make the scenario realistic, a performance tester adds think time. As described above Think Time is the time taken by a real user to provide the input on a page. It includes the time required by a user to read the content of the page or fill out the form or select a value from the given list etc. Let’s say 5 seconds is the think time that a user takes to read the content of each page and provides input in the form of clicking the link or submitting the details. Since there are 5 pages (Home Page, Login Page, Search Product Page, Product Order Page, and Logout Page), the total Think Time will be 20 (= (5-1) X 5). The last page is a logout page which does not require Think Time. Hence 4 (= 5 -1) pages are considered. A point to be noted here is Think Time will also be considered as response time, so as per Little’s law the equation becomes
=> 100 = 5 X (2 + ((5-1) X 5))
=> 100 = 5 X (2 + (4 X 5))
=> 100 = 5 X (2 + 20)
=> 100 = 5 X 22
=> 100 = 110 ………. (2)
Still, something is missing which is restricted to satisfy the equation. Another key value which is called Pacing (delay between two journeys of a user) is used by a performance tester to achieve the desired TPS. Let’s consider 70 seconds as the pacing time which will also be included in the response time, so the equation becomes
=> 100 = 5 X (2 + 20 + 70)
=> 100 = 5 X 92
=> 100 = 460 ………. (3)
Again, LHS is not equal to RHS and to equalize the equation 460 user load will be required which is not as per the client’s requirement. But after considering think time and pacing, the modified formula based on Little’s law becomes
U = T * (R + TT + P)
User Load=Transactions per second * (Response Time + Total Think Time + Pacing)
This is a simple formula that a performance tester would use to estimate workloads, however, when the results came in, they showed that the client’s expectations were not met or that the computation was incorrect, causing the server to be overloaded. They must manually calculate the data in such circumstances.
The reason behind the wrong calculation
The average arrival rate and the waiting time are connected to a person who is waiting in the queue, according to Little’s rule, which I discovered after performing my research on more than 50 samples. When performance testing metrics are computed, however, this is not the case. When waiting time is mentioned, it refers to the entire amount of time the user spent on the website, from login to logout, rather than the time it took to execute a single transaction.
Where was the problem?
- As per Little’s Law, only 3 metrics satisfy the formula of queuing theory, but in performance testing, 6 metrics are used to simulate the real-world situation and satisfy the equation.
- Think time and pacing were not considered and hence equation (1) was false.
- Total Think time had been added to the response time, but the pacing was missing in equation (2).
- In equation (3), the total thinks time and pacing are added with the response time of a single transaction (not overall response time) which makes equation (3) wrong.
- In equation (4), iteration metrics (total think time, pacing, and total response time) are multiplied by transaction metric (transaction rate) and hence RHS value is more than the LHS value and is unable to satisfy the equation.
- Iteration metrics (total think time, pacing, and total response time) are multiplied by another Iteration metric (transaction rate/no. of transactions), and hence equation (5) is true and derived a new formula.
The benefit of Little’s Law Extension
The number of anticipated users, transaction rate, and reaction time are submitted by the client (Project) during requirement collecting in accordance with performance testing guidelines. The term “non-functional requirements” refers to these measurements. Once the script is finished, the number of transactions is shown. The value of the think time is arbitrary and is determined by the page’s characteristics. In the real world, 5 seconds is seen as a typical amount of time to ponder. Therefore, the Pacing is calculated using the extended formula. A valuable measure that can be changed to approximate a precise workload model is pacing. The predicted user load and transaction rate are directly impacted by the incorrect pacing estimate. As a result, the website’s actual performance cannot be determined. The recently developed recipe offers the following benefits over the restriction of Little’s law:
- Calculating the correct pacing
- Achieving the desired workload
- No manual on-paper calculation is required
Conclusion
Little’s law may be applied to a queuing system, but in performance testing, other metrics have an influence on the equation as well. As a result, the workload model’s base formula does not produce accurate results. Because they are unfamiliar with the mathematical reasoning underlying the legislation, many performance testers still undertake manual calculations on paper. They also worry that applying the original Little’s Law to calculate the figures will result in their falling short of the SLA.
