You can define performance
requirements for elements in a test. These requirements specify acceptable
thresholds of performance and validate service level agreements.
About this task
You can set performance requirements on protocol-specific
test elements, on schedule elements, on data created by custom code,
and on collected resource usage data. You define a performance requirement
as standard or supplemental. A standard
performance requirement is a requirement that you determine is significant
enough to cause the entire run to be declared a failure if it fails.
A supplemental performance requirement, although important, is not
significant enough to cause the run to fail. For example, a supplemental
performance requirement might be a request from development to validate
a very specific data item provided by WebSphere® PMI monitoring.
Procedure
To define a performance requirement for the elements
in a test:
- In the Test Navigator, browse to the test and double-click
it. The test opens.
- In the Test Contents area, select
the page or the request that will have the requirement. You can select multiple pages or multiple requests.
- In the Test Element Details area,
click the Advanced tab, and select Enable Performance Requirements. A table
of performance requirements that apply to the page or to the request
is displayed.
- Click the performance requirement to define, and add a
definition, as follows:
Option |
Description |
Name |
You can change the name of a performance requirement to improve
readability. However, changing a requirement name causes a mismatch
between the Performance Requirements report, which uses the changed
name, and the other reports, which use the default name. Therefore,
when you change a requirement name, be sure to keep track of the original
name. |
Operator |
Select an operator. |
Value |
Type a value. |
Standard |
Select to make the requirement standard. A standard
requirement can cause a test to have a verdict of fail. Clear to make the requirement supplemental. In general,
supplemental requirements are used for requirements that are tracked
internally. A supplemental requirement cannot cause a run to fail,
and supplemental results are restricted to two pages of the Performance
Requirements report. |
- Optionally, apply the defined requirement to other test
elements:
- In the Test Contents area, select
the test elements that will have the requirement. The elements
must be of the same type, for example, all page elements.
- In the Requirements table, right-click the requirement
row, and select Copy Requirements.
- Optionally, select Hide Undefined Requirements to hide the shaded rows, which indicate that a requirement is not
defined, and improve readability.
- Select a requirement and click Clear to remove its definition. The requirement is still available and
can be redefined.
- After you have defined a number of requirements on test
elements, you might want to see all of the requirements defined for
the test. To do so:
- In the Test Contents area, click
the name (root) of the test.
- In the Test Element Details area,
click the Performance Requirements tab. The Performance Requirements page
displays a summary of the performance requirements defined in the
test.
- To navigate to the original requirement definition,
double-click the requirement row.
Example
You can define performance requirements in a test or in a
schedule. When you define a requirement in a test, the requirement
is defined individually for each test element—even if you select multiple
test elements and apply the requirement to all of them at the same
time. When you define a requirement in a schedule, the requirement
is applied to the aggregate of test elements.
For example, assume
that you select every page in a test and define this requirement: Average response time for page [ms] [For Run] must
be less than 5 seconds. This means that if one page in the test has
a response time of 6 seconds, the requirement on that page fails.
The other pages, which have a response time of less than 5 seconds,
pass.
Assume that you open a schedule and define this requirement: Average response time for all pages [ms] [For Run] must be less than 5 seconds. This measures the average response
time for all of the pages. One page can have a response time of 30
seconds, but if enough pages have a response time low enough to counter
the negative effect of that one page, the requirement passes.
For information on defining requirements in schedules, see Defining performance requirements in schedules.