System Administration – Post Installation Tasks
This guide is a checklist of tasks required or recommended after a basic installation to ensure safe and reliable operation. In any case, the system administrator has to decide whether each individual task makes sense in the specific use case or not.
In a default installation, the root account is either locked or a ssh login is only possible via keyfile. A remote login via ssh is only possible via the user account additionally created during the installation (here by example
hostmin). Log in and try to acquire administrative privileges.
[…]# ssh email@example.com firstname.lastname@example.org's password: […]# sudo su - [sudo] Passwort for hostmin:
At least, it is convenient to additionally set up a key file for this account and configure easy access on the local (Fedora) desktop. Back on your desktop, create a key file (leave the password field empty for easy access on a secure desktop and choose a descriptive name for the generated key file). For easy access create a
[…]# mkdir ~/.ssh […]# chmod 700 ~/.ssh […]# cd ~/.ssh […]# ssh-keygen -t rsa -b 4096 -C "email@example.com" -f ~/.ssh/<outputkeyfile> […]# ssh-copy-id -i $outputkeyfile.pub firstname.lastname@example.org […]# vi ~/.ssh/config Host myhost Hostname host.example.com User hostmin ProxyCommand none ForwardAgent no ForwardX11 no Port 22 KeepAlive yes IdentityFile ~/.ssh/$outputkeyfile […]# ssh myhost
System users are a systematic weak point in defending a server against compromise attacks. This is certainly unintentional on the part of the users. It is due to lack of understanding, insecure passwords, falling for phishing attacks, and alike. On the other hand, it is precisely the users that a server is operated for.
Therefore, it is best practice to minimize the number of system users and instead use virtual accounts or offload applications that require system users to virtual machines or containers. In case of an intrusion, this is a kind of second line of defense, trying to keep the server itself from being infected even then.
In addition, it is a small effort to generally prevent password-based login and to enforce key files. An exception can be defined for individual, selected users. If a server is located in a data center and can only be remotely accessed without much additional effort, it may make sense to set up a "fallback user" if key-based authentication is no longer available for some reason.
As a side effect it saves a lot of warning messages in the log file and makes it easier to check them.
Login to your server and create a sshd configuration file and edit as follows:
[…]# vi /etc/ssh/sshd_config.d/60-local.conf # Local custimization: disable password login except for # one (optionally add some more) user as a fallback option. PasswordAuthentication no Match User hostmin PasswordAuthentication yes #Match User hostmin2 # PasswordAuthentication yes
Reload the sshd daemon
[…]# systemctl reload sshd
Test that everything works as expected
Is an authorised user able to log in?
Is anyone else is rejected with the message "Permission denied (publickey,gssapi-keyex,gssapi-with-mic)"?
If this does not work: Check whether the latest update has been installed. The file /etc/ssh/sshd_config.d/50-redhat.conf there should not include a line „PasswordAuthentication yes“ (as this is already the default and should not be repeated in order not to hinder other configurations).
Open Cockpit in your desktops browser
https://host.example.com:9090. Accept the security warning of your browser. Cockpit uses a self signed certificate.
If you disabled root access during installation (recommended), login as root is not possible.
Login with your administrative user account (hostmin here). At the top you will see a warning that the Web Console is running with limited permissions. Enable administrator access so that all administrative privileges are automatically available after login.
Cockpit relies on a password based login to its web interface, even for root (unless the root account is completely locked, locking the account as part of ssh does not help here). And thus it is a potential target for brute-force attacks. In most cases, it is rarely used in daily operations and it may be a good practice to remove the general accessibility in the firewall. Instead, access Cockpit in of three secure alternatives described below.
Reconfigure firewall permanently
[…]# firewall-cmd --permanent --remove-service=cockpit […]# firewall-cmd --reload
Use one of the following alternatives.
Access Cockpit via ssh tunnel
Login to the server via ssh and setup a tunnel in one go
ssh host.example.com -L 9090:host.example.com:9090
While you remain logged in, open in your local browser localhost:9090
Add Cockpit service temporarily on demand
Login to the server and reconfigure firewall
When finished remove cockpit again
Use a secure local proxy
Install Cockpit on your local Fedora workstation or on a lab server shielded by a firewall. Configure this instance to access any of your remote Cockpit instances using Cockpits remote administration capability. It uses a protected ssh connection in the public network.
In the upper left corner of Cockpit you will see the name of the logged in user and the desktop rsp. the (local) lab server name, along with an expand icon. This opens a box where you can switch to another server or add a new one.
Add new hostlink opens a simple form to fill in hostname and user. Use your administrative user name on the (remote) server. And you can assign a color. Select automatic login via ssh keyfile. Cockpit will create one for you if none exists or otherwise uses an existing one. For a newly created key, Cockpit installs the public key on the remote server, too.
Even if you activated root access during installation, you have hopefully kept the default option to restrict root to key file authentification. So you have to set up a key file prior to any ssh login. But be aware! You can still log in to Cockpit’s web interface. Therefore, in this case, be sure to implement the additional security measures described in the previous chapter!
This step is to be performed only if a pair of keys does not already exist. It is best to create the key in the .ssh directory of the desktop user. It should not be secured by password to enable automatic processing. The naming with leading 'id_' und trailing types abbreviation, e.g. '_rsa' is just a common convention, yet helpful.
Execute on the local desktop
[…]# mkdir ~/.ssh […]# cd ~/.ssh […]# ssh-keygen -t rsa -b 4096 -C "email@example.com" -f <outputkeyfile>
Although the type rsa is widely used, you may adjust your key type accordingly.
You normally use ssh-copy-id to install the public key on the server. However, this requires a password login, which was disabled for root during installation. Therefore, a detour is now required.
Log in to your server via sftp using the unprivileged administration account and transfer the public key file
[…]# sftp firstname.lastname@example.org sftp> put ~/.ssh/<outputkeyfile>.pub sftp> quit
Log in to your server via ssh using the unprivileged administration account again
[…]$ ssh email@example.com
On the server acquire root permissions, move the key file and adjust permissions
[…]$ sudo su - […]# mkdir /root/.ssh […]# cd /root/.ssh […]# mv /home/hostmin/<outputkeyfile>.pub /root/.ssh/authorized_keys […]# chown -R root.root /root/.ssh […]# chmod 700 /root/.ssh […]# chmod 600 ~/.ssh/* […]# /sbin/restorecon -R -vF /root/.ssh
On your local workstation test key file based access:
[…]# ssh -i ~/.ssh/<outputkeyfile> firstname.lastname@example.org
adjust file, file type, and domain name as appropriate.
To simplify access create a configuration file on your desktop and define a short name for the connection:
[…]# vi ~/.ssh/config # ########################################################### # my remote server, root account # ########################################################### Host myhost Hostname myhost.example.com User root ProxyCommand none ForwardAgent no ForwardX11 no Port 22 KeepAlive yes IdentityFile ~/.ssh/<outputkeyfile>
again, replace names accordingly.
Check if everything works:
[…]# ssh myhost
Both are important for trouble-free server operation. Just in case you missed its configuration during installation or it is incorrect, now is the opportunity to fix it.
Check for correct hostname
Set hostname if required:
[…]# hostnamectl set-hostname <YourFQDN>
Control of time zone, time synchronisation, time
Correct time zone if necessary:
[…]# timedatectl set-timezone <ZONE>
If necessary, activate time synchronisation:
timedatectl set-ntp true
Correct time if necessary:
[…]# timedatectl set-time <TIME>
Depending on installation efforts, network configuration consolidation may be required.
It sounds trivial, but before any change of the network configuration make sure that the entries in the DNS are correct. If you change from DHCP to fixed addresses, you may need to adjust the DNS and vice versa!
Check IP addresses, interface and which protocol stack is used
[…]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc n ... 2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER ... […]# nmcli con NAME UUID TYPE DEVICE enp3s0 dabaa33b-25b0-3bfd-8a74-b6b40847a7a4 ethernet enp3s0 […]# who am i root pts/5 2021-04-09 21:07 (2003:ca:7f05:xx00:yyyy:zzzz:479a:b36e) […]# nmcli -p -f ipv4.method,ipv6.method con show 'enp3s0' ===================================================================== Connection details (enp3s0) ===================================================================== ipv4.method: manual --------------------------------------------------------------------- ipv6.method: manual ---------------------------------------------------------------------
Adjust interface names to your custom config
Just in case IPv6 is configured as local only (fe80::….) or not static, you may set up a fixed IPv6
[…]# nmcli con mod 'enp3s0' ipv6.method manual \ ipv6.addresses <YOUR_IPv6_PREFIX>::2/64 \ ipv6.gateway fe80::1 \ ipv6.dns "2a01:4f8:xx:yy::zzz:8888 2a01:4f8:xx:yy::zzz:9999" […]# nmcli con up 'enp3s0' […]# nmcli con reload
Again, don’t forget to adjust names, prefix, and DNS IP addresses. Pay special attention to the gateway. Using a local address of 1 (fe80::1) is a widely used convention.Another is the IPV6 prefix with the address 1. But each provider may have an even different approach.
Check connectivity from your local workstation. If that fails, the gateway configuration is the first suspected culprit.
[…]# ping6 <YOUR_IPv6_PREFIX>::2 […]# # e.g. ping6 2a01:xxx:yyy:zzz::2
Optionally reconfigure IPv4 as static. But make sure the IPv6 address works and don’t change both protocol stacks at the same time (and in the worst case drop connectivity at all):
[…]# nmcli con mod 'enp3s0' ipv4.method manual \ ipv4.addresses <YOUR_IPv4>/27 \ ipv4.gateway <GATEWAY> \ ipv6.dns "<DNS1_IPv4> <DNS2_IPv4>" […]# nmcli con up'enp3s0' […]# nmcli con reload
Again, don’t forget to adjust names, prefix, and DNS IP addresses and check connectivity from your local workstation:
[…]# ping <YOUR_IPv4>
Optionally you may have a look at the NetworkManager configuration file
[…]# less /etc/NetworkManager/system-connections/enp3s0.nmconnection
Finally reboot now to check everything from ground up
Depending on how you decided on data storage during installation, different supplementary configuration may be required.
If you have chosen the Default partitioning and are content with the basic principle of creating logical volumes for user data, there is nothing to do at the moment. The creation of these logical volumes happens in the context of the installation of the corresponding application software.
You may ensure that the volume group – default name
fedora_fedora– fills the complete disk. Using Cockpit, in the section
Deviceschoose the volume group. At the top of the new window it shows its total capacity.
If you have chosen the Default partitioning but are not content with the basic principle of creating Logical volumes for user data you have now to extend the existing root logical volume to accomodate your data. Cockpit provides an easy way for this. Choose Grow. Determine the new size as needed.
This is not a recommended procedure! Don’t complain in case of issues.
If you have decided for a stricter separation of system and user data and created a small Volume Group for system data, it is sometimes tricky to additionally create a large empty partition and Volume Group in Anacondas Custom interface. So you have to create it here.
Storagein Cockpit’s main menu and then your drive in the right column. Select
Create new partitionand fill in the upcomming form accordingly. In the box "Devices" select from the Menu "Create LVM2 volume group" and fill in the upcomming form accordingly.
The software monitors the log files for authentication errors. In case of multiple retries from the same IP address, the source IP gets blocked by the firewall. This is to prevent brute force methods for cracking passwords and bots checking for weak passwords. However, if an error occurs in the authentication process, a system administrator may also lock himself out.
If you disabled system users password Login in the previous step so sshd only allows keys, you may skip this section. There will be nothing to log in this regard anymore.
Installation of the software and the required Postfix
[…]# dnf install fail2ban postfix
Create and fill configuration file
[…]# vi /etc/fail2ban/jail.local # Jail configuration additions for local installation # Adjust the default configuration's default values [DEFAULT] # Optional enter an trusted IP never to ban #ignoreip = www.xxx.yyy.zzz/32 bantime = 6600 backend = auto # The main configuration file defines all services but # deactivates them by default. We have to activate those neeeded [sshd] enabled = true
[…]# systemctl enable postfix --now […]# systemctl enable fail2ban --now
Control in the log
[…]# tail -f /var/log/fail2ban.log
The software checks log files for anomalies and compiles a daily report that can optionally be sent to system administrators via email. It is something like a minimal defensive effort. More powerful, but much more involved would be the installation of a monitor software.
[…]# dnf install logwatch
The only configuration required is to enter a real email address for root, the recipient of the report. It is added at the end of the file.
[…]# vi /etc/aliases ... # Person who should get root's mail #root: marc root: email@example.com […]# newaliases
It is common sense among system administrators,that regular installation of bug fixes and closing of security vulnerabilities is essential, i.e. applying updates in a systremtatic way. An important step is to automate the process as much as it is reasonable.
Fedora includes a tool, dnf-automatic, which supports several modes of update automation: do not apply, notify admin, apply and notify admin, apply without notification. In particular, alternatives 2 and 3 are definitely worth considering. A general principle might be: Alternative 2 is the minimum choice for almost any system, alternative 3 is not at all suitable for critical systems that must not fail under any circumstances.
We recommend to install at least alternative 2:
[…]# dnf install dnf-automatic […]# vi /etc/dnf/automatic.conf ##emit_via = stdio emit_via = email ##email_from = firstname.lastname@example.org email_from = email@example.com ## […]# vi /etc/aliases ... # Person who should get root's mail #root: marc root: firstname.lastname@example.org […]# newaliases […]# systemctl enable --now dnf-automatic-notifyonly.timer
If you want to automatically install, activate the corresponding timer instead.
[…]# systemctl enable --now dnf-automatic-install.timer
With the shift to systemd-resolved, the
/etc/resolve.conf file is a link to a corresponding systemd runtime configuration. In Fedora 35, this must be fixed manually.
[…]# cd /etc […]# rm resolv.conf […]# ln -s ../run/systemd/resolve/stub-resolv.conf /etc/resolv.conf […]# cd […]# systemctl restart systemd-resolved
Now that secure administrative access is in place, it’s time to update the system and install some useful software. Of course, 'useful software' concretizes itself differently depending on the user and application context. Anyway, a good choice might be vim. With vimdiff e.g. a comparison of updates of configuration files (*.rpmnew) is very comfortable and straightforward.
[…]# dnf remove nano-default-editor […]# dnf install vim vim-default-editor […]# dnf update
Add to the software list as needed.
Finally reboot the server.