SOP Add Zabbix template to Ansible

Overview

This SOP documents the process of adding a Zabbix template to Ansible, and ensuring it is applied to hosts.

Contact Information

Owner

Fedora Infrastructure Team

Contact

#fedora-admin, sysadmin-main, sysadmin-noc

Purpose

Ensure monitoring config is stored in configuration management for later (re)use.

Philosophy

The Zabbix Ansible structure differs from the old Nagios approach. Previously we used a single monolithic role which was run on noc01 to create the nagios config. However, this often lead to monitoring being forgotten, because changes there are outside the scope of the roles our teams usually work in.

With Zabbix, we use the zabbix community collection [4] to delegate actions to the Zabbix API while running playbooks. This allows the role itself to declare the monitoring it requires, keeping monitoring close to the application deployment.

On promoting templates from Stg to Prod

As you’ll see below, we use the "Export Templates" approach to output work done in the UI to a YAML file. This means you can develop on Stg, and then apply the template to Prod afterwards via Ansible.

Example code

An example set of Ansible tasks can be seen at [3] - this adds some monitoring items & triggers for IPA. This assumes you have:

  • A template to deploy

  • A custom item (and thus a custom agent drop-in to deploy on the host)

  • SELinux is not involved (there are other examples in the codebase for this, but to keep things simple here we’ll assume that works)

See SOP Developing new Zabbix checks on Staging if you haven’t played with templates/items/triggers in Zabbix yet.

Implementation

Lets break down the example tasks one at a time.

Custom drop-in task

See https://forge.fedoraproject.org/infra/ansible/src/branch/main/roles/ipa/server/tasks/main.yml#L798-L807

- name: Install Zabbix agent config drop-in
  ansible.builtin.copy:
    src: zabbix/agent-ipa-backup.conf
    dest: /etc/zabbix/zabbix_agentd.d/ipa-backup.conf
    mode: '0644'
  notify:
    - Restart zabbix agent
  tags:
    - ipa/server
    - zabbix_agent

This copies roles/ipa/server/files/zabbix/agent-ipa-backup.conf to the host, and restarts the agent. Clearly, this is only needed if you have a custom item to deploy - for default-supported items no agent changes are required.

Zabbix API Block preamble

See https://forge.fedoraproject.org/infra/ansible/src/branch/main/roles/ipa/server/tasks/main.yml#L809-L822

- name: Zabbix API Block
  vars:
    ansible_zabbix_auth_key: "{{ zabbix_auth_key }}"
    ansible_network_os: "{{ zabbix_network_os }}"
    ansible_connection: "{{ zabbix_connection }}"
    ansible_httpapi_port: "{{ zabbix_httpapi_port }}"
    ansible_httpapi_use_ssl: "{{ zabbix_httpapi_use_ssl }}"
    ansible_httpapi_validate_certs: "{{ zabbix_httpapi_validate_certs }}"
    ansible_host: "{{ zabbix_server }}"
    ansible_zabbix_url_path: "{{ zabbix_url_path }}"
  tags:
    - ipa/server
    - zabbix_api
  block:

This can be copied to your role (almost) entirely unchanged - it simply sets the connection vars to delegate actions to the Zabbix API, instead of running them on the host.

Note the tags, though. Feel free to set tags relevant to your app, but keep zabbix_api as we use this to run actions delegated to Zabbix

Define the template

See https://forge.fedoraproject.org/infra/ansible/src/branch/main/roles/ipa/server/tasks/main.yml#L823-L826

    - name: Import IPA template file
      community.zabbix.zabbix_template:
        template_yaml: "{{ lookup('file', 'zabbix/template-ipa.yml') }}"
        state: present

Easy enough, but where does template-ipa.yml come from?

With Zabbix, we always start our changes in the UI - if we’re defining new items, altering triggers, creating whole new templates, it all starts with manual changes in the UI.

Once you have your template how you like (see the Developing Checks SOP) you can go to Data Collection > Templates, check your template, and then click Export (YAML) at the bottom. This is the file we want to add/update in Ansible, so move it from your downloads folder to the git tree, eg

mv ~/Downloads/zbx_export_templates.yaml ./files/zabbix/template-ipa.yml

Don’t try to edit this by hand - the Ansible collection uses the same export mechanism over the API to check if the template needs updating, so if the info is even in a different order you will get changes on every Ansible run.

Add the template to the host

See https://forge.fedoraproject.org/infra/ansible/src/branch/main/roles/ipa/server/tasks/main.yml#L827-L831

    - name: Add self to IPA template in Zabbix
      community.zabbix.zabbix_host:
        host_name: "{{ inventory_hostname }}"
        link_templates: IPA Monitoring
        force: false

Again, easy enough. Two gotchas:

  • Make sure the link_templates name matches the template you exported/added to Ansible

  • force: false is important, or else you will remove other templates the host is part of

Execution

Once the tasks are added to the repo, you can run the appropriate role (in this example, ipa/server), and perhaps use -t zabbix_agent,zabbix_api to limit the tasks. You should see the host acquire the template in the UI, and start monitoring the items.

Resources

Want to help? Learn how to contribute to Fedora Docs ›