Non-Functional Requirement Gathering

/ performance testing / By

What are the Non-Functional Requirements?

Testing objectives that are designed specifically for performance testing, security testing, usability testing, etc. are known as non-functional requirements. It applies to all kinds of non-functional testing as a whole. Simple examples of non-functional requirements include the following:

  • The number of users handled by the application
  • Page Response Time
  • The number of requests processed by the application per unit of time
  • CPU Utilization
  • Memory Utilization
  • Error rate etc.

In non-functional requirements, some of the objectives are load-bound, such as real-world user load, throughput, etc., while some of the goals are time-bound, such as the response time of a page, requests per second, resource utilization, etc.

Difference between functional and non-functional requirements:

The primary distinction between functional and non-functional requirements is that the former place more emphasis on the proper output and are end-user outcome focused. As an illustration, if a user accesses a banking application and clicks the ‘Account Balance’ link, the program must provide the accurate balance that is present in the user’s account. The non-functional criteria, on the other hand, focus on the system’s performance in terms of responsiveness and load-bearing capability. For instance, if 1000 users simultaneously click the “Account Balance” link, the program must react to each user within a specified amount of time, let’s say 3 seconds.

Purpose of NFR Gathering:

Performance testing expectations are established by the project team or customer as non-functional requirements. The NFR collection phase of the PTLC (Performance Test Life Cycle) includes the procedures of obtaining the requirements, analyzing them from the standpoint of performance testing, and finalizing the quantitative NFRs. The Non-Functional Requirement Document lists organizes, and concludes all the requirements. The output of this phase’s last stage offers quantitative NFRs that aid in creating an accurate workload model prior to performance testing.

Accountability:

The Performance Test Manager or Performance Test Lead has a responsibility to collect, discuss and finalize the Non-functional requirement.

Approach:

The Performance Test Manager or Lead must arrange a meeting with the project team to discuss the client’s expectations once the performance testing scope has been established during the risk assessment phase. He may get the request in plain language, calling for careful research and in-depth analysis to identify the testable NFRs. Additionally, there is a good chance he won’t collect all the necessary information from the project stakeholders in one meeting, so he’ll need to schedule many. The non-functional requirements should be appropriately documented in the non-functional requirement document and received approval from the project stakeholders as soon as all of the non-functional requirements are finished and he has the tested NFRs.

How to proceed practically?

“How can I get the performance requirements from a non-technical customer?” Everyone has a good question like this. The non-functional needs are typically either poorly stated or more conceptual than measurable. By posing the appropriate questions and translating the conceptual requirements into quantitative goals, a performance test manager or lead has to spend his time learning about the client’s expectations. He must act as a liaison between the terms used in performance testing and the language used by inexperienced users.

The performance test manager or lead is in charge of gathering the key information from the client. The consumer must be instantly understood, and communication should begin in their language. Avoid employing performance testing terminology that gives the impression that the client is uneducated when this is not the case. Expecting the client to provide the performance targets all at once is unrealistic. Start the conversation with a description of performance testing’s history and goals. With inquiries like these, the conversation can get underway.

  • What is the expected user load they are looking for?
  • How much is the expected response time for a page?
  • What types of performance tests can be included? Explain each type of performance test and its purpose.

Some additional Tips:

Attempt to establish a positive rapport with the client as well. Inform the customer during the meeting or call by defining key performance testing words and providing clear examples so that he may articulate his expectations. The main thing a performance manager or lead should always try to avoid is scaring the customer by asking too many questions all at once or by being too technical. To impress the consumer and earn his trust, the request should be made in a very polite manner.

Draught the NFRs in the Non-Functional Requirement Document when they have been gathered, along with the meeting date and topics. All project participants must certify the Non-Functional Requirement Document before it can be considered final.

Challenges:

Gathering non-functional requirements is the most challenging part of performance testing. It might be difficult for a performance test manager or lead to gather the appropriate performance testing requirements and formulate them into adequate customer expectations. The majority of the time, applications are brand-new, therefore a performance test manager or lead has no idea what the NFRs (non-functional requirements) are. When the NFRs do not match at the end of the test, the client is often driven to change the NFRs in order to pass the test at any cost. This happens frequently when the customer selects some random numbers over the phone and requests to do NFT (non-functional testing) against those numbers.

The purpose of these performance tests is to demonstrate that performance testing was carried out and that the application functioned admirably in the test environment. However, as user load grows in production, 82% of these apps fail because of performance problems within 6 months or even sooner. Therefore, it is vital to properly study the NFR, refrain from considering (agreeing) to any random number as the NFR, and finalize the quantitative NFRs without any qualification.

Deliverable:

Once the scope is decided, a non-functional requirement paper must be drafted. Each meeting’s conclusions might result in a number of revisions to the document. All elements, as well as significant and small modifications to the need, should be included in non-functional requirement papers. The acceptable error % is one of the factors that will be used to determine the RAG status at the conclusion of the testing, so be sure to include it in the document.

Question: How many types of users are using this application (GUI)?

Answer. 4 types of users

  1. Admin
  2. Seller
  3. End-user
  4. Call center employee

Question: What are the business scenarios of every user?

Answer: Business scenarios for each type of user are like this:

Admin:

  1. To approve/reject a new seller
  2. To verify a newly added product and approve/reject it

Seller:

  1. To add a new product
  2. To delete the existing product

Buyer:

  1. To buy a product
  2. To cancel the order

Call Center Employee:

  1. To register the complaint

Question: What are the AUT current and predicted peak user load for all its users’ actions over time?

Answer: Admin

  • Current: 4
  • Predicted: 10

Seller

  • Current: 25
  • Predicted: 100

Buyer

  • Current: 438
  • Predicted: 2500

Call Center

  • Current: 8
  • Predicted: 24

Question: What is the average active user count (including all types of users) at a time?

Answer:

  • Total: 304
  • Admin: 3
  • Seller: 15
  • Buyer: 278
  • Call Center: 8

Question: What could be the active user count during peak hours?

Answer:

  • Total: 500
  • Admin: 4
  • Seller: 50
  • Buyer: 438
  • Call Center: 8

Question: Anytime during a day or month when the average user count suddenly increased?

Answer: 31st of every month there is a 1-minute sale from 09:00 to 09:01 and 10:00 to 10:01. During this period the active buyer count becomes three times i.e. 834 and the active seller count becomes 23. None of the other (Admin/Customer Care) users’ counts changed.

  • Total: 868
  • Admin: 3
  • Seller: 23
  • Buyer: 834
  • Call Center: 8

Question: What is the request count received at the Admin end?

Answer: 200 requests per hour

Question: What would be the response time NFRs for all the scenarios?

Answer: None of the pages should breach 3 seconds average response time NFR. This NFR is applicable to all types of users, scenarios, and pages.

<Rest of the figures are displayed in the below NFR table>

Question: What would be the server-side NFRs?

Answer:

  • CPU utilization must not be more than 60%; except stress test.
  • Pre, post, and steady-state memory utilization differences must not be more than 15%; except for the stress test.
  • Pre, post, and steady-state disk utilization must not be more than 25%; except stress test.

Question: What is the size of the performance testing environment with respect to production?

Answer: 100% (Live like the environment)

A typical NFR format

NFR IDCategoryDescriptionImpact to
NFR01ApplicationThe solution must be able to support 500 active users.
1. Admin: 4 (2 for seller and 2 for product approval)
2. Seller: 50
3. Buyer: 438
4. Call Center: 8		1. Admin
2. Seller
3. Buyer
4. Call Center
NFR02ApplicationThe solution must be able to support the future volume of active users i.e. 2634
1. Admin: 10
2. Seller: 100
3. Buyer: 2500
4. Call Center: 24		1. Admin
2. Seller
3. Buyer
4. Call Center
NFR03ApplicationThe solution must be performed well during a long period of time with average volume. i.e. 304
1. Admin: 3
2. Seller: 15
3. Buyer: 278
4. Call Center: 8		1. Admin
2. Seller
3. Buyer
4. Call Center
NFR04ApplicationThe solution must be able to support the spike load of the buyer and seller during the sale period.
1. Admin: 3
2. Seller: 23
3. Buyer: 834
4. Call Center: 8		1. Admin
2. Seller
3. Buyer
4. Call Center
NFR05ApplicationAdmin gets an average of 200 requests per hour every time.1. Admin
NFR06ApplicationThe number of orders:
1. Peak Hour Volume: 1340
2. Sale Hour Volume: 2830
3. Future Volume: 7500
4. Average Volume: 600
Note: 4% of the users cancel the order in every scenario.		1. Buyer
NFR07ApplicationSellers add an average of 180 products per hour and delete 18 existing products every hour1. Seller
NFR08ApplicationThe call center employees get 40 complaints per hour1. Call Center
NFR09ApplicationThe response time of any page must not exceed 3 seconds except stress test1. Admin
2. Seller
3. Buyer
4. Call Center
NFR10ApplicationThe error rate of transactions must not exceed 1%1. Admin
2. Seller
3. Buyer
4. Call Center
NFR11ServerThe CPU Utilization must not exceed 60%1. Web
2. App
3. DB
NFR12ServerThe Memory Utilization must not exceed 15% (Compare pre, post and steady-state memory status)1. Web
2. App
3. DB
NFR13ServerThe Memory Utilization must not exceed 15% (Compare pre, post, and steady-state memory status)1. Web
2. App
3. DB
NFR14ServerThe disk Utilization must not exceed 15% (Compare pre, post, and steady-state memory status)1. Web
2. App
3. DB
NFR15ApplicationBuyers order at the average rate of  1. Peak Hour Rate: 3.06 products per hour2. Sale Hour Rate: 3.39 products per hour3. Future Volume: 3 products per hour4. Average Volume Rate: 2.15 products per hour1. Buyer
Scroll to Top