After apply a real workload in my infrastructure, I want to get following things from report.
1. Resources Utilization: actual uses of CPU / Memory / Network 2. Reponses time : find out for only crucial transactional operation/page 3. Server throughput :
Q1: I am new in jmeter. So, I have some confusion that Is Jmeter providing appropriate result in report? Can I make some decision (bottleneck in code, slow/good performance) from that jmeter report? Can you please share some professional jmeter report which meet clients SLA or not? Which jmeter report/which report metrics need to show management?
Q2 : Which performance metrics should be added to a load test report?
Q3 : which type of jmeter report is appropriate for above points (points 1, 2) ?
Q4 : what is best practice that how to more accurate Jmeter report ?
Q5 : work load issue : How to make a real work load model for my test plan ? Can you please share some professional jmeter real work load model?
My using work load model is : I have 50 workstation where different user perform regularly some business operation: cash in , cash out , loan acc opening, deposit acc opening , Ledger report, customer opening report etc (approximate 20 operations)
Work load model is: Thread group (users) : 50 Request: all (10) http request which are regularly used 50 user from different workstation. Ramp up time: 10 loop count : 1
06: what is good practice of Jmeter script writing way?
Exp: i make two test plan with same workload and same scenario (http request)
Here, I got different result for this two test plan. but two script has no big actual difference Without (.js , cs, web related file) . Those file missing manual writing script. Which is appropriate way/best practice for jmeter script writing for finding accurate jmeter result?
Q7 : please give me you have any other guideline for successfully complete load testing task .
Q1a: Is Jmeter providing appropriate result in report? [PerfMatrix]: You can extract result using .log, .csv or .jtl file. For JMeter report generation you can use method mentioned in above blog
Q1b: Can I make some decision (bottleneck in code, slow/good performance) from that jmeter report? [PerfMatrix]: Yes, decision making ability purely depends on your skill and professiency. JMeter or other tool (like LoadRunner) can provide you detailed graphs/statistics and you have to verify whether SLA meets or not. Example: You can analyse the response time graph and compare with given SLA. If some of the transactions breached response time SLA then you can carry out further investigation by refering other graphs. To get exact root cause you need some profiling tools.
Q1c. Can you please share some professional jmeter report which meet clients SLA or not? [PerfMatrix]: You can get performance test result template (.doc) on Google. You can use it and add JMeter graphs.
Q1d. Which jmeter report/which report metrics need to show management? [PerfMatrix]: Generally, we can show i) Running Thread (User) ii) Response Time over the time iii) Throughput iv) Transactions per second v) Top 5 errors vi) CPU Utilization (not in JMeter) vii) Memeory Utiliation (not in JMeter)
Q2 : Which performance metrics should be added to a load test report? [PerfMatrix]: Answered in Q1d.
Q3 : which type of jmeter report is appropriate for above points (points 1, 2) ? [PerfMatrix]: There is no type of JMeter reports. You can generate HTML report or take some .doc template and add the graph and summary of test result in that report.
Q4 : what is best practice that how to more accurate Jmeter report ? [PerfMatrix]: Report is not so much important. It is just a way of representation. Your concentration should be more on analysis and bottleneck finding.
Q5 : work load issue : How to make a real work load model for my test plan ? Can you please share some professional jmeter real work load model?
My using work load model is : I have 50 workstation where different user perform regularly some business operation: cash in , cash out , loan acc opening, deposit acc opening , Ledger report, customer opening report etc (approximate 20 operations)
Work load model is: Thread group (users) : 50 Request: all (10) http request which are regularly used 50 user from different workstation. Ramp up time: 10 loop count : 1 [PerfMatrix]: Please provide some more details i) Have you prepared different scenario for all the test case ii) Are they doing back-to-back transactions (means no think time) iii) What is the distribution% of users for all the above mentioned scenarios iv) Duration of peak and average load inproduction
06: what is good practice of Jmeter script writing way?
Exp: i make two test plan with same workload and same scenario (http request)
Here, I got different result for this two test plan. but two script has no big actual difference Without (.js , cs, web related file) . Those file missing manual writing script. Which is appropriate way/best practice for jmeter script writing for finding accurate jmeter result? [PerfMatrix]: JMeter provides recording provision for scripting and you should use it. Manually you may miss some of the backend request and increase the changes of failing the script. Generally we ingore .js, .gif, .cs file. I would prefer to go via recording instead of manually creation of the script
Q7 : please give me you have any other guideline for successfully complete load testing task. [PerfMatix]: I always give more attention on analysis portion. You can get a series of Analysis tips and trick in my blog. Please refer them.
If you have any specific question then I am there to assist you.
So how can i compare performance test stats with CPU,Memory,Disk i/o etc. stats. For example how can i measure that request response get high due CPU consumption or any network problem .....
JMeter is a freeware tool and it has certain limitations. Since past few years many new features has been added into JMeter through plug-ins to make performance testing more convenient and easy. JMeter HTML report is used for purely reporting purpose, but you can carry-out analysis using this report up-to some extent. This is because many basic feature of analysis is still missing like graph comparison, graph correlation, filter on Dashboard etc.
If you want to analyse JMeter result with resource utilisation graphs then you have to either add some plug-ins and do the analysis separately via JMeter listerner or You can use Blazemeter.
Q_1: I have multiple http requests (like login and open different dashboards)in jmeter. When I run this test for 200 Users it works fine without generating any error. But when i run the same test for more than 200 users, jmeter starts failing some of test (some time its missing csrf_token) thats why login get failed so on the base of this some test cases failed. Please tell me how can i get rid from maximum number failures.My machine specs are (7th generation ci7, 16 GB RAM and 512 SSD). Jmeter HEAP size set up to 4GB. I am using stepping thread to increase the load on web application. I am adding 10 users after every 10 seconds and so on, when it reach maximum then decrease the 10 users after 10 seconds. Most of test case failures observe when decreasing the number of user step wise. Q_2: Please tell me, is it possible that during load testing i have login the web application once in whole test? Because i want to test response time of dashboards instead of login. My test is login the application every time and go on specific dashboards after this. Please provide the answer of both question.
Ans_1: What is the response of the server when you apply load beyond 200 Users? Is csrf_token captured by regular expression in your script?
Ans_2: Yes it is possible. Use "Once Only Controller" and place Login transaction inside this controller. Also, enable "KeepAlive" setting for every request in JMeter.
Server remain responsive when I made load above 200 users, i think only jmeter missing the scrf_token. No crash,error or exception observe on our application server. Yes, I am using regular expression to extract the csrf_token. Please guide on this issue.
Issue is still not clear, because it may be possible beyond 200 users server unable to send token. Apply one logic here,
1. Capture the scrf_token into regex variable and give '0' as default value in regular expression. 2. Use "If" Controller 3. Place all the successive requests inside If controller
Likewise you can conclude whether scrf_token is present in the server response or not.
Can I login into web application with different users concurrently? For example I have 100 users in Jmeter and login with 100 different login and users at the same time in single loop.
Hi Gagandeep, I am Charitha, working as Quality engineer from the past 5 years, now i got assigned into project on Performance testing using Jmeter.i am quite new to this tool,but i was exploring on procedure to design GUI based script and generating of graphs, till now i could able to design script in GUI mode for API Testing and also have generated the graphs using Graph generator plug-in.
But how can i analysis the graphs and reports returned by jmeter? How can i submit my analysis in a better/easy way to the client?
Can you provide me some real example sheets which can submit to client? My Email id : lakshmin@menlo-technologies.com
Hi Gagandeep, You are doing very good work by helping all of us. Could you please shate performance Test result document to my id as well? Email id: rajkumar6779@gmail.com Thanks, Raj
i have generated my html dashboard report from non gui mode ,can i share this local path (c: \\apchae jmeter 3.3\bin to other team members or stackholders
Hi Gagandeep, I am Geetha, working as Test Analyst ,working on Performance testing using Jmeter.i am quite new to this tool,but i was exploring on procedure to design GUI based script and also have generated the graphs using Graph generator plug-in.
But how can i analysis the graphs and reports returned by jmeter? How can i submit my analysis in a better/easy way to the client? We have 600 users with 12 different scripts for different programs as our's is ecommerce application
Can you provide me some real example sheets which can submit to client? My Email id : geethapdit@gmail.com Thanks.
JMeter provides all the basic graphs which you can use for analysis. There are some limitation like merging of graphs is not possible, but you can overcome from this by comparing the graphs manually and check what exactly happen at a particular time in those graphs.
To prepare the analysis report you can copy the graphs and paste them in the report format which I sent to you on your email ID.
I'm using jmeter 5.0 version and i'm running jmx file using and generate html report. But the report needs to be customized as per the below requirement 1) Statistics table should display the result according to Execution timings(Exact structure like View Result Table ) 2) How customize the Statistics table so that samplers result.
- Can you guide me in Selenium python WebDriver integration with Jmeter? - And What sampler should I use to execute the selenium python driver test case?
I need to automate my test cases through selenium and want to performance testing by using jmeter integration with selenium test cases.
Jmeter Report related problem:
ReplyDeleteAfter apply a real workload in my infrastructure, I want to get following things from report.
1. Resources Utilization: actual uses of CPU / Memory / Network
2. Reponses time : find out for only crucial transactional operation/page
3. Server throughput :
Q1: I am new in jmeter. So, I have some confusion that
Is Jmeter providing appropriate result in report?
Can I make some decision (bottleneck in code, slow/good performance) from that jmeter report?
Can you please share some professional jmeter report which meet clients SLA or not?
Which jmeter report/which report metrics need to show management?
Q2 : Which performance metrics should be added to a load test report?
Q3 : which type of jmeter report is appropriate for above points (points 1, 2) ?
Q4 : what is best practice that how to more accurate Jmeter report ?
Q5 : work load issue : How to make a real work load model for my test plan ?
Can you please share some professional jmeter real work load model?
My using work load model is : I have 50 workstation where different user perform
regularly some business operation: cash in , cash out , loan acc opening, deposit acc opening , Ledger report, customer opening report etc (approximate 20 operations)
Work load model is:
Thread group (users) : 50
Request: all (10) http request which are regularly used 50 user from different workstation.
Ramp up time: 10
loop count : 1
06: what is good practice of Jmeter script writing way?
Exp: i make two test plan with same workload and same scenario (http request)
1. Jmeter script recording (Http Script Recorder) : (Test plan 1)
2. Jmeter manual script writing (not script record) : (Test plan 2)
Here, I got different result for this two test plan. but two script has no big actual difference
Without (.js , cs, web related file) . Those file missing manual writing script.
Which is appropriate way/best practice for jmeter script writing for finding accurate jmeter result?
Q7 : please give me you have any other guideline for successfully complete load testing task .
Q1a: Is Jmeter providing appropriate result in report?
Delete[PerfMatrix]: You can extract result using .log, .csv or .jtl file. For JMeter report generation you can use method mentioned in above blog
Q1b: Can I make some decision (bottleneck in code, slow/good performance) from that jmeter report?
[PerfMatrix]: Yes, decision making ability purely depends on your skill and professiency. JMeter or other tool (like LoadRunner) can provide you detailed graphs/statistics and you have to verify whether SLA meets or not.
Example: You can analyse the response time graph and compare with given SLA. If some of the transactions breached response time SLA then you can carry out further investigation by refering other graphs. To get exact root cause you need some profiling tools.
Q1c. Can you please share some professional jmeter report which meet clients SLA or not?
[PerfMatrix]: You can get performance test result template (.doc) on Google. You can use it and add JMeter graphs.
Q1d. Which jmeter report/which report metrics need to show management?
[PerfMatrix]: Generally, we can show
i) Running Thread (User)
ii) Response Time over the time
iii) Throughput
iv) Transactions per second
v) Top 5 errors
vi) CPU Utilization (not in JMeter)
vii) Memeory Utiliation (not in JMeter)
Q2 : Which performance metrics should be added to a load test report?
[PerfMatrix]: Answered in Q1d.
Q3 : which type of jmeter report is appropriate for above points (points 1, 2) ?
[PerfMatrix]: There is no type of JMeter reports. You can generate HTML report or take some .doc template and add the graph and summary of test result in that report.
Q4 : what is best practice that how to more accurate Jmeter report ?
[PerfMatrix]: Report is not so much important. It is just a way of representation. Your concentration should be more on analysis and bottleneck finding.
Q5 : work load issue : How to make a real work load model for my test plan ?
Can you please share some professional jmeter real work load model?
My using work load model is : I have 50 workstation where different user perform
regularly some business operation: cash in , cash out , loan acc opening, deposit acc opening , Ledger report, customer opening report etc (approximate 20 operations)
Work load model is:
Thread group (users) : 50
Request: all (10) http request which are regularly used 50 user from different workstation.
Ramp up time: 10
loop count : 1
[PerfMatrix]: Please provide some more details
i) Have you prepared different scenario for all the test case
ii) Are they doing back-to-back transactions (means no think time)
iii) What is the distribution% of users for all the above mentioned scenarios
iv) Duration of peak and average load inproduction
06: what is good practice of Jmeter script writing way?
Exp: i make two test plan with same workload and same scenario (http request)
1. Jmeter script recording (Http Script Recorder) : (Test plan 1)
2. Jmeter manual script writing (not script record) : (Test plan 2)
Here, I got different result for this two test plan. but two script has no big actual difference
Without (.js , cs, web related file) . Those file missing manual writing script.
Which is appropriate way/best practice for jmeter script writing for finding accurate jmeter result?
[PerfMatrix]: JMeter provides recording provision for scripting and you should use it. Manually you may miss some of the backend request and increase the changes of failing the script. Generally we ingore .js, .gif, .cs file. I would prefer to go via recording instead of manually creation of the script
Q7 : please give me you have any other guideline for successfully complete load testing task.
[PerfMatix]: I always give more attention on analysis portion. You can get a series of Analysis tips and trick in my blog. Please refer them.
If you have any specific question then I am there to assist you.
Can we generate HTML report from GUI?
ReplyDeleteHi Abhijeet,
DeleteYes, you can generate HTML report in GUI mode as well.
For that purpose you need to save the result of "View Results Tree" in a .log/.csv file and then execute the below command from command prompt:
C:\apache-jmeter-3.1\bin>jmeter -g "" -o ""
Note: Output result folder must be empty.
Is there any way to generate HTML report from GUI?
ReplyDeleteAnswered your query in above comment.
DeleteCan I add cpu,memory,disk i/o, network etc. stats in html report?
ReplyDeleteNo, you can not add resource utilisation graph/stats in the basic JMeter HTML report
DeleteSo how can i compare performance test stats with CPU,Memory,Disk i/o etc. stats. For example how can i measure that request response get high due CPU consumption or any network problem .....
ReplyDeleteJMeter is a freeware tool and it has certain limitations. Since past few years many new features has been added into JMeter through plug-ins to make performance testing more convenient and easy. JMeter HTML report is used for purely reporting purpose, but you can carry-out analysis using this report up-to some extent. This is because many basic feature of analysis is still missing like graph comparison, graph correlation, filter on Dashboard etc.
DeleteIf you want to analyse JMeter result with resource utilisation graphs then you have to either add some plug-ins and do the analysis separately via JMeter listerner or You can use Blazemeter.
JMeter Plug-ins:
https://jmeter-plugins.org/wiki/PerfMon/
Is Blazemeter also a open source and free tool?
ReplyDeleteNo, it is not. Blazemeter charges some bucks to provide the support and services.
DeleteQ_1: I have multiple http requests (like login and open different dashboards)in jmeter. When I run this test for 200 Users it works fine without generating any error. But when i run the same test for more than 200 users, jmeter starts failing some of test (some time its missing csrf_token) thats why login get failed so on the base of this some test cases failed. Please tell me how can i get rid from maximum number failures.My machine specs are (7th generation ci7, 16 GB RAM and 512 SSD). Jmeter HEAP size set up to 4GB.
ReplyDeleteI am using stepping thread to increase the load on web application. I am adding 10 users after every 10 seconds and so on, when it reach maximum then decrease the 10 users after 10 seconds. Most of test case failures observe when decreasing the number of user step wise.
Q_2: Please tell me, is it possible that during load testing i have login the web application once in whole test? Because i want to test response time of dashboards instead of login. My test is login the application every time and go on specific dashboards after this.
Please provide the answer of both question.
Ans_1: What is the response of the server when you apply load beyond 200 Users? Is csrf_token captured by regular expression in your script?
DeleteAns_2: Yes it is possible. Use "Once Only Controller" and place Login transaction inside this controller. Also, enable "KeepAlive" setting for every request in JMeter.
Server remain responsive when I made load above 200 users, i think only jmeter missing the scrf_token. No crash,error or exception observe on our application server.
ReplyDeleteYes, I am using regular expression to extract the csrf_token.
Please guide on this issue.
Issue is still not clear, because it may be possible beyond 200 users server unable to send token. Apply one logic here,
Delete1. Capture the scrf_token into regex variable and give '0' as default value in regular expression.
2. Use "If" Controller
3. Place all the successive requests inside If controller
Likewise you can conclude whether scrf_token is present in the server response or not.
Can I login into web application with different users concurrently?
ReplyDeleteFor example I have 100 users in Jmeter and login with 100 different login and users at the same time in single loop.
Yes, you can.
DeleteI would suggest to use .csv file for User's loginID and password. You can get more details on my blog (Please refer below link):
http://perfmatrix.blogspot.in/2017/02/apache-jmeter-csv-data-set-config.html
Hi Gagandeep,
ReplyDeleteI am Charitha, working as Quality engineer from the past 5 years, now i got assigned into project on Performance testing using Jmeter.i am quite new to this tool,but i was exploring on procedure to design GUI based script and generating of graphs, till now i could able to design script in GUI mode for API Testing and also have generated the graphs using Graph generator plug-in.
But how can i analysis the graphs and reports returned by jmeter?
How can i submit my analysis in a better/easy way to the client?
Can you provide me some real example sheets which can submit to client?
My Email id : lakshmin@menlo-technologies.com
Thank you in advance.
Charitha.
Hi Charitha,
DeletePerformance Test result document is sent to you on given email ID.
Hi Gagandeep,
DeleteYou are doing very good work by helping all of us.
Could you please shate performance Test result document to my id as well?
Email id: rajkumar6779@gmail.com
Thanks,
Raj
Hi Rajkumar,
DeletePerformance Test Result Template has been shared to given emailID.
Hello Gagandeep,
ReplyDeleteCan you please share sample Performance Test result document to my id ?
Email-Kjeetender@gmail.com
Test Result Document Format has been sent to given email ID.
DeleteHi Gagan,
ReplyDeleteCould you please share me the Performance Test result documents, yours blogs are amazing.
Thanks and Regards,
Hitesh Bansal
Thanks Hitesh,
DeletePlease share your email ID, I will share the format of Performance Test Result Document.
hiteshbansal4@gmail.com
ReplyDeleteDocument sent...
DeleteHi Gagandeep,
ReplyDeleteCan you please share Performance Test result document to my id is shaiksadhiq@gmail.com
Hi Sadhiq,
DeleteThe Test Result Document format has been sent.
Hi Gagan,
ReplyDeleteCould you please share any reference Performance Test documents to tejaravi05@gmail.com
Hi Ravi,
DeleteDocuments sent!
Hi Gangadeep can you send the document on shilpa.dalvi99@gmail.com
Deletei have generated my html dashboard report from non gui mode ,can i share this local path (c:
ReplyDelete\\apchae jmeter 3.3\bin to other team members or stackholders
Hi Vinay,
DeleteIf given folder is in shared mode then you can give access to particular user (team member) within organization.
Rather than this I would suggest to zip the result folder and either upload to share drive or directly send in the email.
Hi Gagandeep,
ReplyDeleteI am Geetha, working as Test Analyst ,working on Performance testing using Jmeter.i am quite new to this tool,but i was exploring on procedure to design GUI based script and also have generated the graphs using Graph generator plug-in.
But how can i analysis the graphs and reports returned by jmeter?
How can i submit my analysis in a better/easy way to the client?
We have 600 users with 12 different scripts for different programs as our's is ecommerce application
Can you provide me some real example sheets which can submit to client?
My Email id : geethapdit@gmail.com
Thanks.
Hi Geetha,
DeleteJMeter provides all the basic graphs which you can use for analysis. There are some limitation like merging of graphs is not possible, but you can overcome from this by comparing the graphs manually and check what exactly happen at a particular time in those graphs.
To prepare the analysis report you can copy the graphs and paste them in the report format which I sent to you on your email ID.
I'm using jmeter 5.0 version and i'm running jmx file using and generate html report. But the report needs to be customized as per the below requirement
ReplyDelete1) Statistics table should display the result according to Execution timings(Exact structure like View Result Table )
2) How customize the Statistics table so that samplers result.
Hi Abrar,
Delete1) This is not possible, because HTML report shows a glance on the result in a tabular and graphical format.
2) i did not get your question.
HiGagandeep,
Delete- Can you guide me in Selenium python WebDriver integration with Jmeter?
- And What sampler should I use to execute the selenium python driver test case?
I need to automate my test cases through selenium and want to performance testing by using jmeter integration with selenium test cases.
Hi Abrar,
DeleteUnfortunately, I never worked on this. Please google it.
I found the solution for that thanks
DeleteHi Gagandeep,
ReplyDeleteCan you please share Performance Test result document with varuntakkar.20@gmail.com
Please refer the below link:
Deletehttps://perfmatrix.blogspot.com/2017/06/performance-test-report-template.html
can you email or share the dashboard report html report?
ReplyDeleteYou can generate the HTML at your end by running a sample test.
Delete