How to check/fix robosignatory

sysadmin-main can access robosignatory in staging, but production is only accessible to a very limited set of people.

  1. Check the status of robosignatory:

    1. Log into autosign01{.stg}.rdu3.fedoraproject.org

    2. Check the logs:

      journalctl -f -l -u robosignatory

    3. If the service is not running properly, restart it:

      systemctl restart robosignatory

    4. You will need to know the autosigning passphrase and enter it with:

      systemd-tty-ask-password-agent

  2. Check the status of the signing-vault

    1. Log into sign-vault01.{stg}.rdu3.fedoraproject.org

    2. Check the status of sigul server:

      tail -f /var/log/sigul_server.log

    3. If needed, restart the sigul server:

      sigul_server -dvv

  3. Check the status of the signing-bridge

    1. Log into sign-bridge01{.stg}.rdu3.fedoraproject.org

    2. Check the status of the sigul bridge:

      tail -f /var/log/sigul_bridge.log

    3. If needed, restart the sigul bridge:

      sigul_bridge -dvv

    4. If the service is running, but appears to be reprocessing the same message over and over again, you may need to drop that message in order to restore processing.

Cases where you may need to do this:

  • some rpm is corrupt (you can see if this is the case in the vault logs)

  • some rpm is larger than sigul can process (you can see this in vault logs)

    1. First, stop robosignatory.service on autosign01. It’s important that nothing is processing the messages when you attempt to drop one.

    2. Setup a ssh tunnel to the rabbitmq web interface:

      ssh rabbitmq01.rdu3.fedoraproject.org -L 15672:localhost:15672

    3. Next go to localhost:15672 in your browser, there should be a rabbitmq login page there.

    4. Login as admin. Password is in ansible-private vars

    5. go to 'queues'

    6. search for 'robosignatory'

    7. First get the top message on the queue and see if it’s the one you want to drop.

    8. Then, select reject message and NAK (drop from queue)

    9. Select (and requeue) the next message to confirm that the one you nacked is gone.

    10. Restart robosignatory

Want to help? Learn how to contribute to Fedora Docs ›