Redhat Linux Sever Time Sync

Cloud Journey
6 min readOct 8, 2021

Overview

Real Flowers 2018 in San Francisco

Time sync is important for security and event correlation. Sometimes it is used for distributed transactions implementation. Time accuracy between multiple computer systems is achieved through synchronization.

In this blog post, I will

  • Inspect AWS EC2 Redhat 8 VM, to understand how time sync is configured.
  • Inspect Azure marketplace Redhat 8 VM, to understand how time sync is configured.
  • Configure or check time sync service for a physical Redhat server.

AWS EC2 Redhat 8

Provision EC2

For AWS EC2, SSM agent is pre-installed in Amazon Linux and Ubuntu, but not in Redhat, so if you plan to create the VM in a private subnet and use SSM to logon to the box, you will need to install SSM during VM launch.

Use user data during EC2 launch to install SSM agent. Below is user data example for us-east-2 region.

#!/bin/bash
sudo dnf install -y https://s3.us-east-2.amazonaws.com/amazon-ssm-us-east-2/latest/linux_amd64/amazon-ssm-agent.rpm
sudo systemctl enable amazon-ssm-agent
sudo systemctl start amazon-ssm-agent

chronyd

Check whether chrony is running.

sh-4.4$ sudo systemctl status chronyd
● chronyd.service - NTP client/server
Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2021-10-07 20:41:50 UTC; 3h 44min ago

Amazon NTP Server

The Amazon Time Sync Service is available through NTP at the 169.254.169.123 IPv4 address or the fd00:ec2::123 IPv6 address for any instance running in a VPC.

sh-4.4$ cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
pool 2.rhel.pool.ntp.org iburst
# Record the rate at which the system clock gains/losses time.
driftfile /var/lib/chrony/drift
# Allow the system clock to be stepped in the first three updates
# if its offset is larger than 1 second.
makestep 1.0 3
# Enable kernel synchronization of the real-time clock (RTC).
rtcsync
# Enable hardware timestamping on all interfaces that support it.
#hwtimestamp *
# Increase the minimum number of selectable sources required to adjust
# the system clock.
#minsources 2
# Allow NTP client access from local network.
#allow 192.168.0.0/16
# Serve time even if not synchronized to a time source.
#local stratum 10
# Specify file containing keys for NTP authentication.
keyfile /etc/chrony.keys
# Get TAI-UTC offset and leap seconds from the system tz database.
leapsectz right/UTC
# Specify directory for log files.
logdir /var/log/chrony
# Select which information is logged.
#log measurements statistics tracking
# Amazon Time Sync Service
server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4

Check if chrony is Synchronized

sh-4.4$ chronyc tracking|less
Reference ID : A9FEA97B (169.254.169.123)
Stratum : 4
Ref time (UTC) : Fri Oct 08 01:52:03 2021
System time : 0.000002206 seconds fast of NTP time
Last offset : +0.000000626 seconds
RMS offset : 0.000004415 seconds
Frequency : 11.512 ppm fast
Residual freq : +0.001 ppm
Skew : 0.104 ppm
Root delay : 0.000506683 seconds
Root dispersion : 0.000272471 seconds
Update interval : 16.1 seconds
Leap status : Normal
sh-4.4$ chronyc sources|less
210 Number of sources = 5
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 169.254.169.123 3 4 377 16 -26us[ -32us] +/- 511us
^- lofn.fancube.com 2 9 377 129 -1398us[-1403us] +/- 51ms
^- time.cloudflare.com 3 9 377 133 -1466us[-1467us] +/- 23ms
^- 38.229.52.9 2 8 377 128 +47ms[ +47ms] +/- 188ms
^- LAX.CALTICK.NET 2 7 377 64 -4541us[-4549us] +/- 50ms

Azure Redhat 8 VM

Azure hosts are synchronized to internal Microsoft time servers that take their time from Microsoft-owned Stratum 1 devices, with GPS antennas. Virtual machines in Azure can either depend on their host to pass the accurate time (host time) on to the VM or the VM can directly get time from a time server over internet.

In Azure VM Ubuntu 20, chrony is configured by default to point to hyperv PTP clock source.

In Azure VM Redhat 8, if refclock is not configured in the chrony.conf, you may manually add it. When chrony.conf is not configured with refclock, meaning chrony is not pointing to hyperv PTP clock source.

Check chrony.conf

$ cat chrony.conf
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
pool 2.rhel.pool.ntp.org iburst
# Record the rate at which the system clock gains/losses time.
driftfile /var/lib/chrony/drift
# Allow the system clock to be stepped in the first three updates
# if its offset is larger than 1 second.
makestep 1.0 3
# Enable kernel synchronization of the real-time clock (RTC).
rtcsync
# Enable hardware timestamping on all interfaces that support it.
#hwtimestamp *
# Increase the minimum number of selectable sources required to adjust
# the system clock.
#minsources 2
# Allow NTP client access from local network.
#allow 192.168.0.0/16
# Serve time even if not synchronized to a time source.
#local stratum 10
# Specify file containing keys for NTP authentication.
keyfile /etc/chrony.keys
# Get TAI-UTC offset and leap seconds from the system tz database.
leapsectz right/UTC
# Specify directory for log files.
logdir /var/log/chrony
# Select which information is logged.
#log measurements statistics tracking
# Azure hosts are synchronized to internal Microsoft time servers
# that take their time from Microsoft-owned Stratum 1 devices.
# The Hyper-V drivers surface this time source as a PTP-based
# time source in the guest. This configures chrony to use it.
refclock PHC /dev/ptp0 poll 3 dpoll -2 offset 0 stratum 2

Check if PTP clock source is present.

$ ls /sys/class/ptp
ptp0
$ cat /sys/class/ptp/ptp0/clock_name
hyperv

Check Time Sync Status

$ chronyc tracking
Reference ID : 50484330 (PHC0)
Stratum : 3
Ref time (UTC) : Fri Oct 08 03:09:41 2021
System time : 0.000003488 seconds fast of NTP time
Last offset : +0.000004436 seconds
RMS offset : 0.000015131 seconds
Frequency : 11.798 ppm slow
Residual freq : +0.286 ppm
Skew : 1.679 ppm
Root delay : 0.000000001 seconds
Root dispersion : 0.000048710 seconds
Update interval : 8.0 seconds
Leap status : Normal
$ chronyc sources
210 Number of sources = 5
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
#* PHC0 2 3 377 9 -13us[ -25us]
$ chronyc sourcestats
210 Number of sources = 5
Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev
==============================================================================
PHC0 6 4 37 +0.074 4.806 +491ns 20us

Redhat Physical Server

In physical server, there is no host or cloud provider NTP server, so we could use NTP server from pool.ntp.org (note: The 0, 1, 2 and 3.rhel.pool.ntp.org names point to a random set of NTP servers. so you will see random NTP server as the sources)

If you wish use any specific NTP server, edit /etc/chrony.conf file.

cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.rhel.pool.ntp.org iburst
server 1.rhel.pool.ntp.org iburst
server 2.rhel.pool.ntp.org iburst
server 3.rhel.pool.ntp.org iburst
#restart
systemctl restart chronyd

iburst

To improve the time taken for initial synchronization, add the following option to the end of a server command, this is now a default option in the configuration file.

There is no PTP cloud source in physical server:

# ls /sys/class/ptp
ls: cannot access /sys/class/ptp: No such file or directory
# cat /sys/class/ptp/ptp0/clock_name
cat: /sys/class/ptp/ptp0/clock_name: No such file or directory

Checking if chrony is Synchronized:

# chronyc tracking
Reference ID : 6C3D49F4 (108.61.73.244)
Stratum : 3
Ref time (UTC) : Fri Oct 08 03:25:35 2021
System time : 0.000088878 seconds fast of NTP time
Last offset : +0.000053425 seconds
RMS offset : 0.000176015 seconds
Frequency : 37.304 ppm slow
Residual freq : +0.001 ppm
Skew : 0.028 ppm
Root delay : 0.014527087 seconds
Root dispersion : 0.011323001 seconds
Update interval : 1029.4 seconds
Leap status : Normal
# chronyc sources
210 Number of sources = 4
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 108.61.73.244 2 10 377 1025 -177us[ -184us] +/- 18ms
^- 44.190.40.123 2 10 377 307 -5283us[-5283us] +/- 74ms
^+ hc-007-ntp1.weber.edu 1 10 377 549 +1699us[+1699us] +/- 36ms
^+ x.ns.gin.ntt.net 2 10 377 465 -1050us[-1050us] +/- 39ms

References

Chapter 18. Configuring NTP Using the chrony Suite Red Hat Enterprise Linux 7 | Red Hat Customer Portal

pool.ntp.org: the internet cluster of ntp servers (ntppool.org)

Set the time for your Linux instance — Amazon Elastic Compute Cloud

Manually install SSM Agent on Red Hat Enterprise Linux instances — AWS Systems Manager (amazon.com)

Log EC2 Linux user data and send it to the console log when running RHEL 7 or RHEL 8 (amazon.com)

Are the rhel.pool.ntp.org NTP servers supported by Red Hat? — Red Hat Customer Portal

19.17.11. Configuring the iburst Option Chapter 19. Configuring NTP Using ntpd Red Hat Enterprise Linux 7 | Red Hat Customer Portal

--

--

Cloud Journey

All blogs are strictly personal and do not reflect the views of my employer. https://github.com/Ronnie-personal