Get Knowledge from Video instead of Content:

Monday 30 January 2017

JMeter - PostProcessor - "Regular Expression Extractor"

As a performance tester, we are very well aware of dynamic value (data which changes) coming from the server that needs to be handled or captured and pass into the next request as a parameter. This process is called "Correlation" or "Handling of Dynamic Values". When we talk about the handling of dynamic values then the first question comes in mind is:

 Why we need to handle the dynamic values? 
To answer this question I will show you what exactly happens at the time of script recording, script replay and after handling the dynamic values. First see, how client and server act when some dynamic values are passed between them? 

 This is the scenario while you record a script:

When you replay the script then:

When you capture the dynamic values and pass into the next request while replaying the script then:

Hope now you understood, why dynamic values need to be captured? 

Now, the second thing that you should know is:
What are the common dynamic values?
1. Session ID
2. Access Token
3. Customer Name / ID
4. Order Number
5. Bill Number
6. Number of records displayed on a page
7. Current Date and Time

There are many more values which depend on application to application. Here, I would say your ultimate goal must be to find out each and every dynamic value which causes failure of the script. Now, the question is HOW?

 In JMeter, Regular Expression plays an important role. Regular Expression is used to identify dynamic values come in a response. In Microfocus Performance Center/LoadRunner, dynamic values are captured by Correlation. Regular Expression is a pattern which is used to specify a set of strings required for getting dynamic value. To prepare a regular expression, it is important to learn how we can create it using metacharacter and literal character. Here, I will not discuss too much about regular expression syntax (token), otherwise, it will be distracted us from our original topic. If you want to learn how to write regular expression then refer to my post where you can learn regular expression token in a simple way along with examples. 
Learn - How to write the regular expression?

Now, come back to the regular expression extractor element of JMeter. Regular Expression Extractor is a post processor (execute after the response arrives) and it is always added under a sampler whose response contains dynamic value(s) and you need to capture and pass that dynamic value(s) in the next request (where required).   

 How to add "Regular Expression Extractor" element?
You can follow the below steps:
1. Select "Sampler" node whose response contain dynamic value and want to capture
2. Right-click on the node
3. Mouse hovers "Add"
4. Mouse hovers "Post Processors"
5. Click "Regular Expression Extractor"

How to remove "Regular Expression Extractor" element?
You can follow the below steps:
1. Select "Regular Expression Extractor" node
2. Right click on "Regular Expression Extractor" node
3. Click "Remove" (Mouse) or "Delete" button from the keyboard
4. Click "Yes"    

What are the attributes of "Regular Expression Extractor" element?
"Regular Expression Extractor" has following attributes:
  • Name: To provide the name of the post-processor
  • Comments: To provide arbitrary comments (if any)
  • Apply to: To define the search scope of dynamic value. 
  •  Main sample and sub-samples: In case, the request is redirected then use this search scope, so that dynamic content can be searched in the response of both main and re-directed request.
  • Main sample only: When request is not re-directed or dynamic value is present only in the response of main request then use this search scope.
  • Sub-samples only: When request is re-directed and dynamic value is available in the response of re-directed request then you can use this search scope.
  • JMeter Variable Name to use: If dynamic value needs to be extracted from the value of any JMeter variable then you need to select this option and provide the JMeter Variable name in the text field. 
  • Field to check: It refines the search scope and instructs JMeter to search the dynamic value in the specific part of a sample, sub-sample or JMeter variable depends on the option you selected in "Apply to" section. 
  • Body: This option instructs JMeter to search in the body of the response. The header of the response does not include in the search scope.
  • Body (unescaped): The search includes all the HTML escaped code like &amp, &quot, &lt etc. This search scope impacts JMeter performance, hence it is recommended to select this option when really required.
  • Body as a Document: This option allows JMeter to search the dynamic value in the document returned by the server.
  • Response Headers: The search will be conducted only on the header part of the response and captured the dynamic value. This option does not applicable for non-HTTP request.
  • Request Headers: This option instructs JMeter to search in the header part of the request. It is helpful when resquest is reditected and dynamic value is passed in the header of sub-request. This option does not applicable for non-HTTP request.
  • URL: The search scope will be URL only when you select this option. This option is used when request is re-directed and dynamic value is available in the URL part. The best example of this is OAuth access token capturing.
  • Response Code: This option is used to capture the response code. Let's say you have two transaction flows and selection of any flow depends on the pass (Response code = 200) and Fail (Respose code != 200) staus of previous request. In such scenario you can choose "Response Code" as an option which returns the response code.
  • Response Message: To capture the response message this option is used. The response message could be like OK, Gateway Timeout etc.
  • Name of created variable: The name of the variable in which dynamic value will be stored. This is also called as RegEx variable.
  • Regular Expression: The regular expression statement to capture the dynamic value. If you want to learn how to write regular expression then refer to this post.
  • Template: Template helps to capture more than 1 value from a single regular expression. Each template denotes to a group. $1$ represent group 1. $2$ represent group 2 etc. If you use $0$ then it refers to the entire captured string. 
  • Match No. (0 for random): If more than 1 strings are matched in a response data and you need to capture a dynamic value which comes at a particular place (say 5th place) then you have to give 5 as an input and JMeter will recognize all the matched values on the page but store only 5th value in the regular expression variable. It is as same as the ordinal in the LoadRunner. '-1' is used to capture all the values while '0' is used to pick a random value from the list of match dynamic values.
  • Default Value: If the regular expression does not match, then the regex variable will be set to the default value (e.g. Not_Found). This is particularly useful for debugging tests.
  • Use empty default value: If this check box is selected then JMeter set the empty string for regular expression variable. This is not recommended because you can not identify whether regular expression working properly or not?

Learn with an example:
Let's consider, I have identified two dynamic parameters (code and execution) from the response of a page. I can see that their number of occurrences on the same page is 2:

 <div>
<form id="kc-form-login" class="dialog-form" action="https://perfmatrix-public-gateway.com/auth/realms/be55d902-3d75-49b1-b703-322196853ef0/login-actions/authenticate?code=dcsncj-93c51-455d-f-4dadxsdc5-c5c54-nbnd-155xsxcssx127b90fdf-24d5-4986-a7f0-5be4a3e9f5b8&execution=343984ya-689d-4b98-8ff5-98561dfre851" method="post"> 
<form id="kc-form-auth" class="dialog-form" action="https://perfmatrix-public-gateway.com/auth/realms/be55d902-3d75-49b1-b703-322196853ef0/login-actions/authenticate?code=meet9xwRFX4noYrnYFuVRw1xDHOA0_xQOkG5GUyeZzo.d7b90fdf-24d5-4986-a7f0-5be4a3e9f5b8&execution=371997f4-6c0d-4b98-88ec-d74b27af9e86" method="post">
</div>

 To extract these values I will apply following regular expression:

 The output will be:
secureID_1=dcsncj-93c51-455d-f-4dadxsdc5-c5c54-nbnd-155xsxcssx127b90fdf-24d5-4986-a7f0-5be4a3e9f5b8343984ya-689d-4b98-8ff5-98561dfre851
secureID_1_g=2
secureID_1_g0=code=dcsncj-93c51-455d-f-4dadxsdc5-c5c54-nbnd-155xsxcssx127b90fdf-24d5-4986-a7f0-5be4a3e9f5b8&execution=343984ya-689d-4b98-8ff5-98561dfre851"
secureID_1_g1=dcsncj-93c51-455d-f-4dadxsdc5-c5c54-nbnd-155xsxcssx127b90fdf-24d5-4986-a7f0-5be4a3e9f5b8
secureID_1_g2=343984ya-689d-4b98-8ff5-98561dfre851
secureID_2=47YezIPEqm_yGCDjCPdAnJhWGtQqZsEHef53NKz5L2Q.0b180542-f3f2-4ba2-aff9-cbcea3ecfd2c371997f4-6c0d-4b98-88ec-d74b27af9e86
secureID_2_g=2
secureID_2_g0=code=47YezIPEqm_yGCDjCPdAnJhWGtQqZsEHef53NKz5L2Q.0b180542-f3f2-4ba2-aff9-cbcea3ecfd2c&execution=371997f4-6c0d-4b98-88ec-d74b27af9e86"
secureID_2_g1=47YezIPEqm_yGCDjCPdAnJhWGtQqZsEHef53NKz5L2Q.0b180542-f3f2-4ba2-aff9-cbcea3ecfd2c
secureID_2_g2=371997f4-6c0d-4b98-88ec-d74b27af9e86
The first parameter "secureID_1" shows all the dynamic values appeared at first occurance. You may not differentiate between extracted values from this parameter because there is no separation between them. 
The second row "secureID_1_g" denotes how many groups are formed? In our example, there are 2 groups created. 
The third parameter "secureID_1_g0" has the value of full string which is used to extract the desired value. 
The forth parameter "secureID_1_g1" represents the extracted dynamic value of code field which appeared at first place in the response. 
The fifth parameter "secureID_1_g2" represents the extracted dynamic value of execution field which appeared at first place in the response. 
Now, if I want to use 2nd occurrence value of code and execution parameter then I simply need to replace the original values like this:

 It is pretty straight forward and easy way to capture the dynamic value and pass in the next requests in the script. By the way Regular Expression Extractor is a soul of Apache JMeter and frequently asked in the performance testing interviews.

 The regular expression can be tested using RegEx Tester.

 Next Topic:
Apache JMeter - "Boundary Extractor" Element

Previous Topic:
Apache JMeter - "Debug PostProcessor" Element

Related Topics:

Labels: , , , , ,

7 comments :

  1. Unknown9 August 2018 at 03:52

    Thanks Gagandeep sir to shared the valuable Knowledge for Jmeter beginners and also result analysis basic part is excellent.

    ReplyDelete
  2. Pallawi7 January 2019 at 03:20

    hi
    i have taken one sample site (Login-Logout) where i want to extract cookie value but here i can see that the cookie value in response of one URL is not matching with request of the another URL. how is that possible? Also i m unable to find any other dynamic value , so does that mean it is needless to do that in every test case

    ReplyDelete
    Replies
    1. PerfMatrix7 January 2019 at 04:52

      The first rule, not need to capture all the values come under cookie data. You can use cookie manager, it automatically takes care.

      Another thing if there is any specific value like CSRF token in the cookie data then that you need to RegEx.

      Last and important thing, some times requests are re-directed and dynamic values are found in second request/response, so you need to carefully identify where and which data is being generated.

      Delete
  3. Pallawi7 January 2019 at 21:18

    ok.
    1. if request is failing due to cookie then only we can use cookie manager or it is a good practice to do that?
    2. how to identify such dynamic values, do i need to go through all the request & responses in Jmeter one by one? or is there any other way.

    ReplyDelete
    Replies
    1. PerfMatrix7 January 2019 at 21:56

      1. It is a normal practice to use cookie manager to simulate browser behavior.
      2. You can follow the normal method, no need to go scan all the requests.

      Delete
  4. Pallawi7 January 2019 at 23:27

    thank you for the quick response but normal method means what as i m new to this i have gone through most of the sites but still confused

    ReplyDelete
    Replies
    1. PerfMatrix8 January 2019 at 21:43

      Check out this link: https://moduscreate.com/blog/how-to-handle-correlation-in-jmeter/

      If still you are confused, then I need to know your exact confusion.

      Delete

Subscribe to: Post Comments ( Atom )