How to add simple dist-git test for a package
There is a package somepackage, which contains a binary: /usr/bin/somebinary
The most simple and obvious way to test this binary is to run it: somebinary --help and check the exit status of the result.
How to add this test for the package?
Standard Test Roles framework provides solution for the task. It is fully supported in Fedora Rawhide.
tests/tests.yml in the dist-git of the package:
. ├── 0001-something.patch ├── somepackage.spec ├── sources └── tests └── tests.yml
tests.yml is an ansible playbook, where you describe test environment and steps to run your tests.
There are many options and examples available, but let’s focus on the task at hand: we want to run just one command.
Then, the content of the file should look as follows:
- hosts: localhost roles: - role: standard-test-basic (1) tags: - classic tests: - simple: dir: . run: "somebinary --help" (2)
|1||this is a standard test role, it takes care of the test environment, logging, archiving results, etc|
|2||this is your test command, its exit code defines the outcome of the test|
There is a package
somepackage, and there is an integration test suite
for it packaged in a separate
sometests package, which provides the
run_some_tests binary in the system path.
There goal is to trigger this binary as a test for a main package.
Same as above, you need to create a
tests.yml configuration file with
the following content:
--- - hosts: localhost roles: - role: standard-test-basic tags: - classic required_packages: - sometests (1) tests: - integration_tests: (2) dir: . run: run_some_tests (3)
|1||additional package which needs to be installed in the test environment|
|2||any string, will be used as identifier for artifacts and test results|
|3||test execution command|
Put a bash script in tests/scripts/ folder and run it from the playbook.
. ├── 0001-something.patch ├── somepackage.spec ├── sources └── tests ├── scripts │ └── run_tests.sh (1) └── tests.yml
|1||your custom test scenario|
Configure dist-git test to run this script:
- hosts: localhost roles: - role: standard-test-basic (1) tags: - classic tests: - simple: dir: scripts (2) run: ./run_tests.sh (3)
|1||same standard role|
|2||switch to subfolder (path is relative to
|3||this is the test script, its exit code is the outcome of the test|
To test the build we:
checkout dist-git repo
take latest qcow image of Fedora Rawhide
install all packages from the koji build on it
run ansible playbook defined in tests.yml
It is possible to run and debug standard test roles locally. But we highly recommend to use the pull-request workflow for it: simply create a pull-request and wait for CI to react on it.
We trigger almost the same CI machinery for PR testing as it is used for gating of new builds.
And as soon as result of the test is ready, it will appear on the pull request page in Fedora Pagure
To restart the test add a comment to PR in Pagure, with the following