Time

Accurate system time is essential, both for ensuring the correct operation of a number of network protocols, and producing logs that can be correlated between systems and events.

Management of system time across debian.org is handled in several different ways.

Ganeti clusters

The hosts for some of our Ganeti clusters run NTP servers, which local guests can synchronise against, using systemd-timesyncd.

Hosts with broken/missing RTCs

Some hosts either do not have an RTC, or it does not work. Such systems use a combination of fake-hwclock and ntpdate to maintain the system clock.

"The rest"

Most physical hosts run a local NTPD, which in turns synchronises to a set of trusted NTP servers in our network. Communication to those servers is authenticated using the "autokey" mechanism, with public keys distributed by configuration management.

The keys are valid for a year, so must be regenerated prior to their expiry and updated in configuration management, and thus distributed to clients. Ideally renewal of the keys will be spaced out so that clients are always able to reach a trusted server.

Trusted servers have their keys monitored for upcoming expiry, so that we can replace them in plenty of time.

Renewing server keys

On the trusted NTP server:

RANDFILE=/dev/urandom /usr/sbin/ntp-keygen -T -I -H -c RSA-SHA256 -m 2048
RANDFILE=/dev/urandom ntp-keygen -q `hostname` -e | (
  read l; echo "$l";
  read l; echo "$l";
  echo;
  echo "# This is the public version of this 'private' key -";
  echo "# the private data has been replaced by 0x01.";
  echo "# (just ask 'openssl dsa -text < foo.pub')";
  echo;                 
  openssl dsa -passin `hostname` -passin pass:`hostname`
)  > ntpkey_iff_`hostname`.pub

The resulting .pub file should be then be added to configuration management, in place of the existing key.

Adding a new trusted server