Test Case Relevancy
Sometimes a test case is only relevant for specific environment. Test Case Relevancy allows to filter irrelevant test cases out.
Test case relevancy is function which takes environment parameters and returns
False or a list of environment variables.
Test case relevancy is defined by one or more
condition: decision rules.
Allowed operators are:
= == != < <= > >= &&. Anything beyond a
# sign is considered to be a comment and will be ignored.
Everything is case insensitive. First matching rule wins (terminates immediately the relevancy evaluation, the rest of the rules will be ignored).
If there is no rule specified, test case relevancy defaults to
True, that is test case is relevant and should be executed.
Acceptable parameters defining the environment are:
product name (rhel rhel-5 rhel-5.6 rhdts rhscl)
distribution (rhel-6 rhel-6.3 rhscl-1.0.0 f-28)
distro variant (Client Desktop Server Workstation)
architecture (i386 ppc64 s390x x86_64)
software collection (python27 python33 perl516…)
component to be tested (php, apache, …)
distro is always used to define the operating system the software is supposed to run on,
product is used to describe the target product subscribed and consumed by the customer (can be a layered product on top of RHEL such as RHSCL, or RHEL itself if the component is included directly in the operating system).
Distro comparison operates in two modes:
- Major mode
When comparing against a major version such as
distro < rhel-6other major versions are considered (matches any of
rhel-3 rhel-4 rhel-5).
- Minor mode
If minor version is provided as well, for example
distro < rhel-6.3, comparison is performed only within the given major (matches
rhel-6.0 rhel-6.1 rhel-6.2).
Rules which contain environment parameters which are not known at the time of evaluation will be skipped.
decision part of the rule can contain following values:
test case is relevant for given environment
test case is not relevant for this environment
- A=X B=Y C=Z
test case is relevant for the modified environment
The last option above allows to adjust the environment in which the test case is to be executed by providing the list of environment variables which will be passed to the test.
The decision value may be omitted. In such case
True is used by default. So these two lines define identical relevancy:
distro > rhel-7: True distro > rhel-7
mod_wsgi relevant for RHEL6 and newer:
distro < rhel-6: False
busybox not available for s390x on RHEL6:
arch = s390x && distro = rhel-6: False
perl-Config-General not present in the
variant = Client: False
Run python unit tests under valgrind on suitable archs only:
arch = ia64: PHASES=novalgrind arch = s390x && distro < rhel-6: PHASES=novalgrind
New component python-ctypes added in rhel-5.8:
distro < rhel-5: False distro < rhel-5.8: False
Exim present solely in RHEL5:
distro != rhel-5: False