JMeter Complete Element Reference – Part VII

In this article, I will talk you through about the next set of JMeter elements i.e Assertions. These elements plays a crucial role in validating samplers in our test scripts. Lets discuss in detail about the advantages and disadvantages in using assertions. In case if you have missed the older articles click here.

Assertions

From the previous posts, we are all aware that how to send requests and get response from the target server. Now you may have many questions running in your mind like are we receiving expected or desired response from the server, how can we validate the received response  through JMeter. In this case, Assertions come to your rescue !!

Assertions are used to perform additional checks on samplers, and are processed after every sampler in the same scope. To ensure that an Assertion is applied only to a particular sampler, add it as a child of the sampler. By default assertions can be applied only to main samples but JMeter also provides an options to include  assertions only to sub-samples or both. Scope of the assertions can be as follows:

Assertions

If a sub-sampler fails and the main sample is successful, then the main sample will be set to failed status and an Assertion Result will be added. If the JMeter variable option is used, it is assumed to relate to the main sample, and any failure will be applied to the main sample only. More than one assertion can be added to the sampler, controller, thread group or test plan. Failed assertions will cause all affected samples to fail.

Types of Assertions

Apache JMeter is built with wide variety of assertions that can be used validate the response. It is very important keep in mind that, all assertions consumes some amount of your system CPU and Memory but not all assertions are the same but some consume way more CPU and Memory than the others.  Lets dive in to understand the usages of the assertions.

Response Assertion

Response assertion is the most commonly used assertion which is used to check whether the response contain, match or equal a specified pattern. In detail, response may include text, headers, body, code or messages. The patterns can either be:

a “string” for “Equals” or “Sub-string” clauses

a “Perl5-style” Regular Expression for “Contains” or “Matches” clauses

You can also choose whether the strings will be expected to match the entire response, or if the response is only expected to contain the pattern. You can attach multiple assertions to any controller for additional flexibility.

assertion

Above picture depicts about the response assertion. Lets discuss in detail how to use and when to use this assertion.

Apply to:

This is for use with samplers that can generate sub-samples, e.g. HTTP Sampler with embedded resources, Mail Reader or samples generated by the Transaction Controller.

Main sample - assertion only applies to the main sample
Sub-samples - assertion only applies to the sub-samples
Main sample and sub-samples - assertion applies to both.
JMeter Variable - assertion is to be applied to the contents of the named variable

Response Field to Test:

This option is used to instructs JMeter which field of the Response to test.

Text response – This is for the response – which can be displayed in a browser

Document (text) – This is for anything supported by Apache Tika (it assumes apache-tika.jar in /lib folder of a JMeter installation). For example: PDF, Office, Audio, Video formats. Be careful – this can be memory consuming for high loads!

URL Sampled – This assertion is used against a requested URL to ensure that it matches expectations. It’s handy for testing. For example: you may want to check that the redirect URL doesn’t contain an “error” somewhere in the path.

Response Code – This checks that the response code is expected. For 4xx and 5xx response codes, make sure that you have checked the “Ignore Status” box (see below for a full explanation)

Response Message – This is used to verify that the response message appears as expected.

Response Headers – This is used against Response Headers to see if a specific HTTP Header is present or absent.

Ignore Status – JMeter out-of-the-box considers all 4xx and 5xx responses as failures. If your test case is “negative” and, for example, a “404” error is expected, you’ll need to check this box to suppress JMeter’s built-in status code check and substitute it with your own status code assertion.

Pattern Matching Rules:

Indicates how the text being tested is checked against the pattern.

Contains - true if the text contains the regular expression pattern

Matches - true if the whole text matches the regular expression pattern

Equals - true if the whole text equals the pattern string (case-sensitive)

Substring - true if the text contains the pattern string (case-sensitive)

Equals and Substring patterns are plain strings, not regular expressions.
NOT may also be selected to invert the result of the check.

Patterns to Test:

A list of patterns to be tested. Each pattern is tested separately. If a pattern fails, then further patterns are not checked. There is no difference between setting up one Assertion with multiple patterns and setting up multiple Assertions with one pattern each (assuming the other options are the same). However, when the Ignore Status checkbox is selected, this has the effect of cancelling any previous assertion failures – so make sure that the Ignore Status checkbox is only used on the first Assertion. 

Duration Assertion

This assertion tests that each response was received within a given amount of time. Any response that takes longer than the given number of milliseconds (specified by the user) is marked as a failed response.

duration_assertion

Size Assertion

The Size Assertion tests that each response contains the right number of bytes in it. You can specify that the size be equal to, greater than, less than, or not equal to a given number of bytes.The easiest way to check the response size is by using the ‘View Results Tree Listener‘.

size_assertion

XML Assertion

The XML Assertion tests that the response data consists of a formally correct XML document. It does not validate the XML based on a DTD or schema or do any further validation.

xml_assertion

For example:the XML test below has an unclosed root tag (<message>) on the last line:

<?xml version="1.0"?>
<message>
 <to>emilia clarke</to>
 <from>Peter Dinklage</from>
 <heading>Date Night</heading>
 <body>8 PM Dinner @ Alain Ducasse !!</body>
/message>

So the XML Assertion fails all affected samplers and reports the reason in an assertion failure message.

Beanshell Assertion

The Beanshell Assertion allows you to perform additional checks on a sampler using Beanshell scripting. We’ve already covered the JMeter API shorthands available in Beanshell scripts in our article. In addition, the Beanshell Assertion offers the following variables:

Failure – boolean (true|false) : This indicates whether the sampler is considered to be successful.

FailureMessage – string : This is a custom message, displayed as an “Assertion failure message”.

ResponseData – byte : A byte array representing response data

ResponseCode – string : This represents response code

ResponseMessage – string : This holds the response message

ResponseHeaders – string : This contains the response headers

SampleResult – org.apache.jmeter.samplers.SampleResult : This is a JMeter SampleResult class instance which contains results for preceding Sampler(s). When there are multiple parent samplers (i.e. ‘Transaction Controller’ or ‘Retrieve all Embedded resources’ getSubResults), this method returns an array of all nested requests.

For example: the following Beanshell Assertion code snippet will return an error if the word “iamgopikrishna” does not appear in the URL path:

String path = SampleResult.getURL().getPath();
if (!path.contains("iamgopikrishna")) {
   Failure = true;
   FailureMessage = "URL Path: didn't contain \"iamgopikrishna\"" + System.getProperty("line.separator") + "URL Path detected: " + path;
}

MD5Hex Assertion

The MD5Hex Assertion allows the user to check the MD5 hash of the response data.

MD5HexAssertion

Xpath Assertion

The XPath Assertion tests a document for well formedness, has the option of validating against a DTD, or putting the document through JTidy and testing for an XPath. If that XPath exists, the Assertion is true. Using “/” will match any well-formed document, and is the default XPath Expression. The assertion also supports boolean expressions, such as “count(//*error)=2”. See http://www.w3.org/TR/xpath for more information on XPath.

xpath_assertion

Compare Assertion

The Compare Assertion can be used to compare sample results within its scope. Either the contents or the elapsed time can be compared, and the contents can be filtered before comparison. The assertion comparisons can be seen in the Comparison Assertion Visualizer.

compare

Note: The Response Assertion and the Duration Assertion are more or less safe to use, whereas the Compare Assertion and other XML-based ones like XPath Assertion take up the most CPU and memory.

Assertion Results Visualization:

So we have a sample and an assertion to test the response – but how do we see what’s wrong with the response? In the GUI mode, there are two ways that failed assertions can be inspected

Assertion Results Listener:

This reveals the label under which all the assertions were taken.

Asser

View Results Tree Listener

This reveals all the assertions in the test plan.

view

Be careful! They both consume a lot of memory so don’t ever use them during load tests. They  should only be used for debugging or opening a .jtl file generated by non-GUI run.

This is pretty much all we have on JMeter Assertions right now. If you enjoy reading this article then you can subscribe our updates for FREE, just add your email id . I will keep on updating the article for latest testing information. Subscribe and stay tuned for updates, there’s lot more to come.

🙂 Happy Performance Testing !!  🙂

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s