Get Knowledge from Video instead of Content:

Saturday 3 September 2016

Do you really know all type of Performance Tests (Non-Functional Tests)?

Today, I am gonna discuss the very basic thing….Type of Performance Testing (Non-functional Testing). We know that software system or an application should be tested against the Speed, Stability and Scalability under a variety of load conditions; so that it’s true performance can be find-out before Go-Live. Non-functional testing encompasses a range of different tests which enable analysis of various aspects of the system. Non-functional testing tells about what needs to be fixed before going live (mainly the issues faced under the variety of load conditions). Many people have the different opinion on Load, Endurance, Stress and Volume Test. Some are considered them as one type of tests and some are having different views. I will explain what are the types of non-functional test and how they are different from each other:

Performance Test: Performance test is performed, to ascertain how the components of a system are performing in a particular situation. Resource usage, scalability and reliability of the product are also validated under this testing. Load SLA of performance test are either average load or expected load (not to break the system). Performance Test usually conducts in regression manner where small changes are being applied on the daily or weekly basis on the application. Generally, the time duration of the performance test is 1 hour (steady state); it may vary as per your SLA/requirement. The typical user load graph of the performance test is given below:
Performance Test Graph
Note: Performance Testing is a generic term which is used to represent Non-Functional Testing. Performance Test is one of the tests under Non-functional testing.

Load Test: Load Testing is a type of non-functional testing to check system with constantly increasing the load on the system until the time load reaches its threshold value. This type of test is conducted with an increase in load (number of concurrent users, transactions) and then checks the behaviour of an application under test. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation. E.g. If the number of users is increased then how much CPU, the memory will be consumed, what is the network and bandwidth response time. Load testing is performed to make sure that what amount of load can be withstanding the application under test without impacting the response time. The typical user load graph of Load test is given below:

Load Test Graph

Stress Test: The main idea behind stress testing is to determine the failure of the system. Stress testing is a negative testing where we load the software with a large number of concurrent users/processes which cannot be handled by the systems hardware resources. This testing is also known as Fatigue testing. It puts greater emphasis on robustness, availability, and error handling under a heavy load, rather than on what would be considered correct behaviour under normal circumstances. The typical user load graph of the Stress test is given below:
Stress Test Graph

Spike Test: Spike testing is a subset of Stress Testing. A spike test is carried out to validate the performance characteristics when the system under test subjected to workload models and load volumes that repeatedly increase beyond anticipated production operations for short periods of time. Generally, the time duration of the spike test is 1 hour; it may vary as per your SLA/requirement
Spike Test Graph 
Volume Test: Volume test as the name suggested; refers to a test with the bulk of data to be processed to check the efficiency of the application. The main goal of this testing is to monitor the performance of an application under varying database volumes. Generally, the time duration of volume test is 1 hour or time required to process n number of records; it may vary as per your SLA/requirement. The best example of volume testing is to test a batch with millions of records.
Volume Test Graph

Endurance testing: Endurance testing involves testing a system with a significant load extended over a significant period of time, to discover how the system behaves under sustained use. That is, to ensure that the throughput and/or response times after some long period of sustained activity are as good as or better than at the beginning of the test. It is basically used to check the memory leaks. Generally, the time duration of Endurance test is 6-8 hours; it may vary as per your SLA/requirement. The typical user load graph of the Endurance test is given below:
Endurance Test Graph

Soak Test: Another name of Endurance Test. Soak Test is carried out on average load.

Scalability Test Graph
Scalability Test: Scalability Testing is conducted to determine the application’s capability to scale up in terms of user load, the number of transactions, the data volume etc. The purpose of a scalability test is different from stress or load test. Suppose; a company expects the six-fold load increase on servers within the next two months. They may need to increase the server performance and shorten the request processing time to better serve visitors. If the application is scalable then you can shorten this time by upgrading the server hardware, for example, you can increase the CPU frequency and add more RAM. Also, you can improve the request performance by changing the server software, for example, by replacing text-file data storages with SQL Server databases. To find a better solution, first, you can test hardware changes, then software changes and after that compare the results of the tests.

Recovery Testing
Recovery Test: Recovery testing is done in order to check how fast and better the application can recover after it has gone through any type of crash or hardware failure etc. Recovery testing is the forced failure of the software in a variety of ways to verify that recovery is properly performed. In recovery testing, the main focus is given on recovery time, % of the recovery, % of loss and performance of the application after full recovery.


There are some more types of Non-functional test which are considered as NPNFT (Non-Performance Non-Functional Test)

Compatibility Testing
Compatibility Test: Compatibility testing is performed to ensure compatibility of the system/application/website built with various other objects such as other web browsers, hardware platforms, users (in case if it’s very specific type of requirement, such as a user who speaks and can read only a particular language), operating systems etc. This type of testing helps to find out how well a system performs in a particular environment that includes hardware, network, operating system and other software etc.

Security Testing
Security Test: Security testing is used to check whether the application or the product is secured or not. It checks to see if anyone hacks the system or log in to the application without any authorization then the application should not be vulnerable to attacks. The security testing is performed to check whether there is any information leakage in the sense by encrypting the application or using a wide range of software and hardware and firewall etc. The six basic security concepts that need to be covered by security testing are confidentiality, integrity, authentication, availability, authorization and non-repudiation.

Usability Testing
Usability Test: In usability testing basically the testers tests the ease with which the user interfaces can be used. It tests that whether the application or the product built is user-friendly or not. Usability testing also reveals whether users feel comfortable with your application or Web site according to different parameters – the flow, navigation and layout, speed and content – especially in comparison to prior or similar applications. Usability Testing tests the following features of the software.
  • How easy it is to use the software?
  • How easy it is to learn the software?
  • How convenient is the software to end user?


Accessibility Test:  This is a subset of usability testing, and it is performed to ensure that the application being tested is usable by people with disabilities like hearing, colour blindness, old age and other disadvantaged groups.
Accessibility Testing


Localization Test:  Localization Testing is a software testing technique, where the product is checked to assure that it behaves according to the local culture or settings. In other words, it is a process of customizing software application as per the targeted language and country.
The major area affected by localization testing includes content and UI.
It is a process of testing a globalized application whose UI, default language, currency, date, time format and documentation are designed as per the targeted country or region. It ensures that the application is capable enough for using in that particular country


Internationalization Test: Internationalization is a process of designing a software application so that it can be adapted to various languages and regions without any changes.

28 comments :

  1. Hi,

    In Spike test, How to simulate the over load repeatedly during the course of test?

    ReplyDelete
    Replies
    1. Hi Rathina,

      In HP LoadRunner, when you open the scenario in controller(Design tab), you will get "Add Action After" button in "Global Schedule" section. Using this button you can add "Start Vuser", "Duration" and "Stop Vuser" rows in the scenario. If you want I can send you screenshot of a spike test scenario.

      Delete
    2. can u send screenshot for spike testing on
      choudharishivani95@gmail.com

      Delete
    3. Hi Shivani,

      Please refer to the below link: https://www.perfmatrix.com/what-is-spike-test/

      Delete
  2. Please send screenshot for this spike testing.

    ReplyDelete
  3. Replies
    1. Sir, please send the screenshot for spike testing to email:- kumar090529@gmail.com. Also, please forward some performance reports that we show to client.

      Delete
    2. Hi Kumar,

      Do you want screenshot of the spike test scenario?

      Delete
  4. While performing spike testing after reaching the breaking point of the server (where the server fails responding properly), will the server recovers
    after reducing the number of Concurrent users ?

    ReplyDelete
    Replies
    1. Hi Pradeep,

      It is totally depends on the code written for crash management of the server. If server auto-recovery case lies then you should also perform a recovery test (mentioned in this blog), so that you will get to know how much time the system takes to fully recover.

      Delete
  5. Hi Gagandeep,

    How to correlate client side graph with sever side graph, for example average response time graph with cpu utilization graph.

    ReplyDelete
    Replies
    1. There is no different method to correlate client and server side graph. The only think you have to integrate monitoring/profiling tool with LoadRunner/Performance Center before starting the test.

      Once your test is finished you could see server side graph which can be easily correlate with client end graphs using normal method.

      Delete
  6. Load testing description seems different for me anybody check it out

    ReplyDelete
  7. Hi Gagandeep,

    Is the recovery test and Fail-over test is same?

    ReplyDelete
    Replies
    1. No, in failover testing the load is shifted to secondary (back-up) system to check its performance when primary system is down.

      Refer the recovery testing definition from above post.

      Delete
  8. Hi Gagan
    How to configure stress test scenario in loadrunner can please share the screen

    ReplyDelete
    Replies
    1. Hi Sadhiq,

      There is no difference in the scenario creation of stress test. The scenario creation is an art which totally depends on your requirement.

      You can prepare stress test scenario in 2 ways:
      1. Gradual Ramp-up of User load: In this method, user load is gradually increases on the system and forms a linear patter (same shown above in this post) and at a particular point the system will break that will be breaking point.

      2. In another method you can increase the load in the Step-up pattern. Like 5 User ramp-up and execute for 15 minutes then another 5 users (total 10) ramp-up and run for next 15 minutes and so on.

      If you need detailed information on Workload modelling then refer my this post:
      http://perfmatrix.blogspot.com/2017/06/ptlc-performance-test-execution.html

      Delete
  9. can yu plz confirm below points-
    -Endurance test is carried out with total number of users & for Soak , it will be average load.
    -Scalability test ,we scale up the resource configuration and then we test new SLA.
    Did i understand it correctly ?

    ReplyDelete
    Replies
    1. Endurance Test: Yes
      Scalability Test: It is a predictive test and conduct to predict what could be increase to scale the application performance.

      Delete
  10. Sir, is Recovery Test and Failover test same ?

    ReplyDelete
    Replies
    1. No, in failover testing the load is shifted to secondary (back-up) system to check its performance when primary system is down.

      Delete
  11. Also, sir what is Capacity testing, is it same as Volume Testing. And please give a detailed answer for failover testing.

    ReplyDelete
  12. How to configure stress test scenario in in jmeter can you please provide the steps

    ReplyDelete
    Replies
    1. Stress test has just extra load, else scenario will be as same as load test.

      Refer Stress Test: https://www.perfmatrix.com/what-is-stress-test/

      If you want to create a step-up scenario in JMeter then refer to the post:
      https://perfmatrix.blogspot.com/2017/01/apache-jmeter-concurrency-thread-group.html

      Delete