Pages

Wednesday, 30 November 2016

What is Think Time in Performance Testing?

Think time defines the time between two actions of a user. These actions may be navigating the pages, click on the submit button, type keyword and then click "Search" button etc. Still not clear... Ooppppssss?

Let's take one example, you want to buy a Titan watch from Amazon online shopping portal. You liked one watch from the catalogue and you clicked on its image. Before adding into the cart you want to read its "Product Specification" which is on the same page. It took 20 seconds for you to read full specifications and decided to buy this watch then you clicked "Add to Cart" button. So, from the moment when this page was loaded until you read the specification and clicked "Add to Cart" button would be "Think Time" means 20 seconds. See below GIF


How can we calculate think time?
In the real world, think time is totally dependent on user's actions. It may be less than a second for a techie guy while more than a minute for a new user. So, instead of worrying about think time, better you go for an average think time value and use this value between two transactions. Generally, a user spends an average 8 seconds on a website page with average content over there, so you take any figure between 5 to 10 second as a think time. 

You can also calculate think time using below formula (provided that you should have all the required parameters)


Example: If you have - 
  • No. of users = 6
  • End to End Response Time = 25 seconds
  • TPS (Transaction per second) = 0.5
  • Pacing = 15 seconds
  • No. of Transactions = 5
  • Total Think Time = 20 seconds
Average think time would be 5 seconds

You can also verify like this:
Transaction 1 (Home Page)- 3 seconds
Think Time  - 5 seconds
Transaction 2 (Log-in) - 4 seconds
Think Time  - 5 seconds
Transaction 3 (Search) - 10 seconds
Think Time  - 5 seconds
Transaction 4 (Add To Cart) - 5 seconds
Think Time  - 5 seconds
Transaction 5 (Log-out) - 3 seconds
Pacing  - 15 Seconds
Refer Think Time Calculator to calculate think time. 

How can we apply Think Time in our script?
1. Microfocus LoadRunner:
a. Use lr_think_time() between two transactions like:
lr_start_transaction("LogIn");{
........
}
lr_end_transaction("LogIn", LR_Auto);
lr_think_time(5);
lr_start_transaction("Search");{
...........
}
lr_end_transaction("Search", LR_Auto);
b. Use parameter value  for lr_think_time():
int tTime = 5;
lr_start_transaction("LogIn");{
........
}
lr_end_transaction("LogIn", LR_Auto);
lr_think_time(tTime);
lr_start_transaction("Search");{
...........
}
lr_end_transaction("Search", LR_Auto);
How to get the total think time passed in a transaction in Microfocus LoadRunner?
lr_start_transaction(“t1”);{
...
lr_output_message(“%f”,lr_get_transaction_think_time(“t1”));
}
lr_end_transaction(“t1”);
The function lr_get_transaction_think_time() should be put before the end of the respective transaction. In the above case, the lr_output_message displays the transaction think time in the log.

Below options are available for think time in Microfocus LoadRunner under Run Time settings:

Ignore think time:
Think times are ignored. The script is going to run without any delay between each transaction. Use this setting when you want to generate maximum load to heavily stress your Web server. Do not use it when you are trying to create more realistic user interactions with a Web server.

Replay think time:
Think times are used exactly as they were recorded in the Web performance test. Simulates multiple users running the Web performance tests exactly as recorded. It looks all good in Vugen at first but when you are in the Controller, it looks weird as all VUsers will be replaying at the same amount of think time. If you have transactions created for the script, in the Controller graphs, all transactions will perform at the same speed. Because a load test simulates multiple users, using the same think time could create an unnatural load pattern of synchronized virtual users.

Multiply Recorded think time by:
This setting multiplies the recorded value with the numeral provided, extending the time delay in the script. Can increase or decrease the think time as recorded. Means we can increase the think time by multiplying the recorded think time with 1,2 or to decrease the think time multiply the recorded think time with 0.5 or so. The range of value we can enter is 0.01 to 10000.

Use random percentage of recorded think time:
The script will replay the percentage range of the recorded think time. With the recorded think time, we can allow 50% – 150% deviation emulating the expert users (lower range:50%) and novice users (upper range:150%) assuming the speed we are working (recording) with the application is at an average user speed. Provides a more realistic simulation of virtual users by slightly varying the think time between requests.

Limit think time to:
This last option allows the script to replay the maximum think time limit. Suppose while recording or using the above two options we have different think times in different transactions like 60, 90, 120 etc. If we select the option Limit think time to 40 seconds. The think time allowed will be only 40 seconds. If the think time is more, the remaining time is ignored and if it is less it will as it is.

2. Apache JMeter: In JMeter, think time is termed as "Timer". You can use different timers to solve your purpose. Below is JMeter’s Timer list:
Below is typical hierarchy to get more understanding
        Test Plan
o   Config Element
o   Thread Group
🔃  Logic Controller
  Sampler1
⇥  Pre-processor
⇤  Post processor
⦽  Assertion
⥁  Timer
  Sampler2
⇥  Pre-processor
⇤  Post processor
⦽  Assertion
⥁ Timer
  Sampler3
⇥  Pre-processor
⇤  Post processor
⦽  Assertion

Tips:
1. It is recommended to use Average Think Time to make your performance test more realistic.
2. Ignore think time while performing graph analysis
3. Think time also used to control transaction rate
4. Pacing and Think time are two different things

Related Topics:

16 comments:

  1. Hi,
    Thanks for info,
    If we don't have all the required parameters for the above given think time formula and if i have only expected no of transaction with user's load, So now i can calculate script time using tool from sample run (excluding Think time and pacing time). Let's say for 1 hr Expected Trxn =2000, Expected User =50, so iteration/user=40, the Observed script time= 8 sec (without thinktime and pacing) so and totally 8*40 =320 sec finally i can get 3600-320= 3280 (remaining time). Here my question is can i use this remaining time to distribute the 80% as thinktime and 20% as pacing time. Can you please share your suggestion.

    ReplyDelete
    Replies
    1. Yes, you can. But keep one thing in mind:

      "Pacing achieves Load while Think Time achieves Concurrency"

      The more your think time (80%) the less is user concurrent.

      You can get more information on this in my blog:
      http://perfmatrix.blogspot.com/2016/11/importance-of-pacing-and-think-time-in-performance-testing.html

      Also, you can use

      Think time calculator:
      http://perfmatrix.blogspot.in/2016/11/think-time-calculator.html

      Pacing Calculator:
      http://perfmatrix.blogspot.in/2016/11/pacing-calculator.html

      Delete
    2. Hi Gangandeep,
      1. As iam in US for job searching this materials are very handy to brush up my skills. i will use these calculators based on the parameters available.
      coming to this 80% thinktime, i thought number of occurrence for think time is always high comparing with pacing. so thought to maintain as 80%. but your comment("The more your think time (80%) the less is user concurrent") makes more sense. so let me calculate script wise as per your suggestion (like in avg 5 to 10sec for website pages and respective time for filling forms senario) from script remaining time, so rest of remaining time per total iteration, i can distribute it as pacing.correct me if i am wrong.
      2. Also i found some lines from your links meant "Ignoring pacing/less pacing on each biz cycles can cause sessions to be spaced after one another almost instantly" so my question here is, is there any specific session time out maintains in applications usually, do we need to check with developer for that. should we consider that in pacing time allocation. please suggest me. Thanks

      Delete
    3. Thanks Rathina for following my blog.

      1. "Expected Trxn =2000, Expected User =50, so iteration/user=40". Does your mean 1 Trxn = 1 Iteration? If this is the case then think time is irrelevant.

      If it is a typo and I consider 1 iteration would have multiple transactions (let's say 5) then I would suggest to use 10 sec think time and remaining will be your pacing.
      If you have 1 transaction = 1 Iteration then yes pacing time will be more (the reason is [a] total response time is less (8 sec) [b] Total transaction in an hour is also less (40 in an hour by 1 user)).

      2. Not every application has session time-out. The best example of sessioned application is Banking application. So if your application has session time-out then you need to maintain the session. In that case Pacing+responsetime will be less then session time out.

      Let ,e know if you have further questions.


      Delete
  2. 1. Yes you are correct, I meant 1-iteration = multiple(5) Trxn. so i took as script time = 8sec in average for over all trxn(5), not as particular one trnx response

    time from the script. So my understanding is as follows,

    For the given example As you suggested i can use 10 secs for each think time for the script, means(5 trxn-1)= 4 think times for one iteration -->4*10=40 secs, for 40

    iteration---> 40 * 40=1600 secs, Hence i can get remaining time--->3600-320-1600=1680 secs, so finally pacing time for each iteration= 1680/(40-1). correct me if i am

    wrong

    2.Coming to session time out, Yes your points are understandable and correct. Sorry to bother you much, Still have one doubt on this, session normally gets closed when

    we log out and one of the Main reasons for session time out in between transactions is inactiveness/Delay, So think time also a time delay may cause session timeout.

    Let's say if my scripts have login and logout with some transaction inbetween.So the session time out can happen only if there is more delay in between transaction,

    not after the log out because after logout session id itself get closed then pacing delay happens and new session would start when the same script again goes for next

    iteration with same user. So in that case, did you mean Thinktime + Response time for a transaction should be less than the session timeout?

    ReplyDelete
    Replies
    1. 1. Your calculation is correct. Just one correction Pacing = 1680/40 => 42 seconds, because LR counts Pacing even in the last iteration and then exit the uses.

      2. Yes, to clarify more

      Session Time Out > (End to End Response Time + Total Think Time)

      Delete
  3. Thank you much Gangandeep. it's more informative.

    ReplyDelete
  4. Expected Trxn =2000, Expected User =50, so iteration/user=40,

    Should the above be rephrased as Transactions/ user = 40 ?????

    Pls advice...

    ReplyDelete
    Replies
    1. This is only possible when you have 1 Transaction = 1 Iteration.

      Delete
  5. Shall I consider that total transactions achieved for 50 users is 50*2000. because each user achieved 2000. please correct me if I am wrong.

    ReplyDelete
    Replies
    1. Hi Chandan,

      If you reference the above comment chat where 1 transaction = 1 Iteration then yes; total transactions achieved by 50 users will be 50X2000.

      In case, 1 iteration having N no. of transactions then total transactions will be same 50X2000, but total iterations will be (50X2000)/N.

      Delete
  6. Hi Gagan, I think the above calcalation is wrong in Rathina's Example.

    For the given example As you suggested i can use 10 secs for each think time for the script, means(5 trxn-1)= 4 think times for one iteration -->4*10=40 secs, for 40

    iteration---> 40 * 40=1600 secs, Hence i can get remaining time--->3600-320-1600=1680 secs, so finally pacing time for each iteration= 1680/(40-1).

    If one iteration contains 5 transactions then how can no of iterations be considered as 40. It should be 8 ideally.

    Also i am confused with my statement about total transactions achieved=50*2000 which i think wrong.

    Kindly solve this puzzle in detail considering 1 iteration = 5 transactions.

    ReplyDelete
    Replies
    1. Chandan's Query: If one iteration contains 5 transactions then how can no of iterations be considered as 40. It should be 8 ideally?

      Gagan: Correct Chandan. Her statement is contradict. Expected transactions 2000 and Iteration per user 40. Ideally it should 8. The above calculation will be correct only when expected no. of Iterations will be 2000.

      Chandan's Query: Also i am confused with my statement about total transactions achieved=50*2000 which i think wrong.

      Gagan: If we consider 50 users will complete 2000 iterations in one hour, then total transactions will be completed by 50 users in one hour is 10000 transactions (=200*50). Hence you calculation is correct.
      Explanation:
      No. of Users = 50
      Expected Transactions in 1 hour = 2000
      No. of transactions in 1 iteration = 5
      No. of Iterations = 100 (=2000/5)

      Delete
  7. Hi Gagandeep,

    I just want to know, what is the difference between Think time and Page think time??

    ReplyDelete
    Replies
    1. Meaning of both the terms are same.

      "Think Time" is used by LoadRunner wheseas "Page Think Time" is used by LoadComplete tool.

      Delete