Regular Expression Extractor in JMeter | Efficient Data Extraction

JMeter – Regular Expression Extractor

The term “map” describes how dynamic values received from the server are processed. This dynamic data, such as session ID and authorization token, are the distinguishing values that are generated by the server for security purposes. Dynamic values can also relate to site content such as dropdown options, dates, item and product IDs, order numbers, etc. You can capture these dynamic values through the map and pass them on to subsequent lookups. This is how JMeter’s ‘Correlation’ or ‘Dynamic Value Handling’ fundamentally works.

Why are the dynamic values required to be correlated?

You must first understand what exactly happens during script recording, script playback, and after mapping the dynamic variables to get the answer to this question.

Let’s see how the client and server behave when exchanging some dynamic values.

What are the common values which require correlation?

Session ID
Access Token
Customer Name / ID
Order Number
Bill Number
Number of records displayed on a page
Current Date and Time

Depending on the type of application and the phrase used to describe them, there may be more values. Remember that your ultimate goal should be to identify and correlate all of the dynamic values that lead to the script failing. The next question is “HOW?”

Regular expression is important in JMeter. The regular expression is used to recognize dynamic values in response. The mapping in Microfocus Performance Center/LoadRunner captures dynamic values. A sequence of strings required to get a dynamic value is specified using a regular expression pattern. It is crucial to understand how to generate regular expressions using literals and metacharacters to prepare them. I won’t go into too much detail in this section regarding the regular expression (token) syntax, as it would divert our attention from the main topic. You can find a brief explanation of regex tokens along with examples in my post if you want to learn how to write them.

Now go back to JMeter’s regular expression extractor component. If the response contains dynamic values, you should capture those dynamic values and pass them in the next request (if necessary). The Regular Expression Extractor is a post-processor that runs after the response has arrived.

How to add a ‘Regular Expression Extractor’?

Follow the below steps:

Select the ‘Sampler’ element whose response contains the dynamic value and you want to capture it.
Right-click on the element
Hover the mouse on ‘Add’
Hover the mouse on ‘Post Processors’
Click ‘Regular Expression Extractor

What are the ‘Regular Expression Extractor’ input fields?

Input fields for “Regular Expression Extractor” include:

1. Name: To mention the post-processor name

2. Comments: to offer random remarks, if any

3. Use for: Defining the search parameters for dynamic values.

4. Subsamples and the main sample: Use this search scope if the request is being redirected so that dynamic material in both the main response and the response to the redirected request may be searched.

5. primary example only: Use this search scope when the request isn’t being redirected or when the dynamic value appears solely in the main request’s response.

6. Only sub-samples: You can utilize this search scope when a request is redirected and a dynamic value is provided in the response to the redirected request.

7. Useful JMeter variable name: Select this option and enter the name of the JMeter variable in the text field if the dynamic value needs to be taken from the value of any JMeter variable.

Field to check:

Depending on the choice you made in the “Apply to” section, the search scope is narrowed and JMeter is told to search for dynamic values in a specific region of a JMeter sample, subsample, or variable.

Body – Selecting this option tells JMeter to look in the response body. The response header excludes from the search scope.

Body (non-escaped): All escaped HTML characters, such as &amp, &quot, and &lt, are included in the search. It is recommended to choose this option only when absolutely necessary, as search scope affects JMeter performance.

Body as a document: This option allows JMeter to query the dynamic value on the document’s return from the server.

Response Headers – Only the header part of the response is searched and this captures the dynamic value. Non-HTTP request is not covered by this option.

Request Headers – Selecting this option tells JMeter to look at the request header section. When the request is redirected and the dynamic value is passed in the subrequest header, it is beneficial. Non-HTTP requests are not covered by this option.

URL – When you choose this option, the search is limited to URLs only. When the request is redirected and a dynamic value is present in the URL part, this option is used. The top copy

Response Code: The response code is captured with this option. Consider a scenario where you have two transaction streams and you need to choose one based on the success (Response Code = 200) or failure (Response Code! = 200) status of the previous request. In this case, you can select “Response Code” as an option, which will return the response code.

Reply Message – This option is used to retrieve the reply message. The response can contain words like OK, Gateway Timeout, etc.

Newly Generated Variable Name: The name of the variable that will contain dynamic data. Another name for this is a RegEx variable.

The regular expression declaration that captures the dynamic value is known as a regular expression. If you want to learn how to create a regular expression, check out this post.

Template – Using a template makes it easy to extract multiple values from a single regular expression. Each template designates a particular group. $1 represents Category 1. $2 represents Group 2, etc. Entering $0$ refers to the entire captured string.

Match No. (0 for random): If more than one string matches in the response data and you need to capture a dynamic value that appears in a specific location (say, the fifth place), you should specify the number 5. JMeter recognizes all matching values in the page, but stores only the fifth value in the regular expression variable. It is identical to the LoadRunner ordinal. While ‘0’ selects a random value from the list of dynamic match values, ‘-1’ is used to capture all values.

Default Value – The regex variable is set to the default value (such as Not_Found) if the regular expression does not match. This is especially beneficial for debugging.

Use an empty default value: If this checkbox is checked, JMeter will default the value of the regular expression variable to an empty string. Since you can’t tell if a regular expression will work or not, it’s not recommended.

Use this example to learn:

Consider that from a page’s response, I was able to extract two dynamic factors (code and execution). I can see that there are two of these on the same page:

<div>

</div>

I’m going to use the following regular expression to extract these data.

The output will be:
secureID_1=dcsncj-93c51-455d-f-4dadxsdc5-c5c54-nbnd-155xsxcssx127b90fdf-24d5-4986-a7f0-5be4a3e9f5b8343984ya-689d-4b98-8ff5-98561dfre851
secureID_1_g=2
secureID_1_g0=code=dcsncj-93c51-455d-f-4dadxsdc5-c5c54-nbnd-155xsxcssx127b90fdf-24d5-4986-a7f0-5be4a3e9f5b8&execution=343984ya-689d-4b98-8ff5-98561dfre851″
secureID_1_g1=dcsncj-93c51-455d-f-4dadxsdc5-c5c54-nbnd-155xsxcssx127b90fdf-24d5-4986-a7f0-5be4a3e9f5b8
secureID_1_g2=343984ya-689d-4b98-8ff5-98561dfre851
secureID_2=47YezIPEqm_yGCDjCPdAnJhWGtQqZsEHef53NKz5L2Q.0b180542-f3f2-4ba2-aff9-cbcea3ecfd2c371997f4-6c0d-4b98-88ec-d74b27af9e86
secureID_2_g=2
secureID_2_g0=code=47YezIPEqm_yGCDjCPdAnJhWGtQqZsEHef53NKz5L2Q.0b180542-f3f2-4ba2-aff9-cbcea3ecfd2c&execution=371997f4-6c0d-4b98-88ec-d74b27af9e86″
secureID_2_g1=47YezIPEqm_yGCDjCPdAnJhWGtQqZsEHef53NKz5L2Q.0b180542-f3f2-4ba2-aff9-cbcea3ecfd2c
secureID_2_g2=371997f4-6c0d-4b98-88ec-d74b27af9e86

All of the dynamic values that originally appeared are displayed in the first parameter, “secureID_1”. There is no separation between the extracted values from this parameter, therefore you cannot distinguish between them.

How many groups are formed, as indicated by the second row “secureID_1_g”? In our case, two groups have been formed.

The desired value is extracted using the complete string value of the third option, “secureID_1_g0.”

The extracted dynamic value of the code field that initially appeared in the response is represented by the fourth argument, “secureID_1_g1”.

The fifth parameter “secureID_1_g2” denotes the dynamic value of the execution field that was extracted and displayed in the response’s first field.

Now, all I have to do is just change the original values with the following ones if I want to use the second occurrence value of the code and execution parameter:

The process of capturing the dynamic value and passing it to the script’s subsequent queries is rather simple and easy. By the way, the performance testing interviews typically question about Regular Expression Extractor, which is the heart and soul of Apache JMeter.