Domain Time II Configuration - Client/IEEE 1588-2008 (PTP)

IEEE 1588-2008 (PTP)

Domain Time II Client
Version 5.2

Information and configuration instructions for using the Precision Time Protocol (IEEE 1588-v2) with Domain Time II.

PTP Precision and Timestamping

A PTP network of hardware clocks on a well-regulated segment can usually synchronize to within a few tens or hundreds of nanoseconds. Software implementations can theoretically achieve the same results, but are subject to limitations of the host operating system. In particular, Windows and Linux are not real-time operating systems, and the connecting networks are often congested or have unpredictable or asymmetric performance characteristics. PTP implementations in these environments must estimate delays caused by servicing interrupts, stack traversal, latencies introduced by expiring time slice quanta, and vagaries in the network itself.

By default, when an application tells the operating system to place a packet on the wire, it can know only when the packet left the application, not when the packet finished traversing the operating system layers and was actually placed on the wire. Likewise, incoming packets may suffer delays in the network driver or the operating system before being delivered to the application. This degree of uncertainty produces “jitter,” or small variations in the measured time that are artifacts of the environment rather than the protocol. In addition, any procedure can be preempted, leading to erroneous measurements of elapsed time.

Some operating systems, with assistance from the network driver, can provide a hardware timestamp indicating when a particular packet left or entered the system. Using this information, the application can determine how much extra time was spent handling the packet between the application and the wire. The network socket options SO_TIMESTAMPING (hardware), SO_TIMESTAMPNS (software), and or SO_TIMESTAMP (low-resolution software) are used to enable this functionality on Linux. If the operating system and network driver support either hardware or software timestamps, the measurements of PTP can be significantly more precise than without such support. DTLinux will automatically detect and use the best type of timestamp support. SO_TIMESTAMPING requires a NIC capable of providing hardware timestamps.

Current Windows operating systems do not support SO_TIMESTAMPING, SO_TIMESTAMPNS, or SO_TIMESTAMP. However, as of Server 2019 and recent builds of Win10, they do offer NDIS software timestamping measurement of internal network stack delays. Domain Time can use this function, see KB2019.708 for more information.

The theoretical limit of precision for Domain Time on Windows using any protocol is the hectonanosecond. The limit of precision on Linux is the nanosecond. A hectonanosecond is 100 nanoseconds, or 1/10th of a microsecond. The real-world behavior of Domain Time’s implementation of PTP achieves results as good or better than any other supported protocol.

Introduction

Optimum Environment for PTP

PTP Profiles

Configure Domain Time for Windows as a PTP Slave

Configure Domain Time Client for Linux (DTLinux) as a PTP Slave

PTP Monitor/Audit Server

PTPCheck

Introduction

PTP, like other time protocols, is used to determine the current time offset and discipline the local clock. PTP typically achieves a much higher degree of accuracy than other types of time synchronization. In most cases, the accuracy is several orders of magnitude better than NTP or DT2 alone.

Domain Time II supports Precision Time Protocol version 2.0 (IEEE 1588-2008) and version 2.1 (IEEE 1588-2019). PTPv1 is not supported.
Domain Time Servers and Clients can be configured to run in slave-only PTP mode using the Default (multicast or hybrid), Enterprise, or (as of v5.2.b.20180606) the Telecom PTP profile (see PTP Profiles below).
Domain Time II Server can assume the master clock role using either the Default or Enterprise (Hybrid) PTP Profiles. As of v5.2.b.20180801, Domain Time Server can also act as a Telecom master. This option should only be enabled if you have no hardware-based PTP grandmaster, or if you need a software-based backup to your grandmaster. PTP is designed to work with hardware-based clocks. Software-based clocks cannot offer the same degree of precision or accuracy as hardware-based clocks.
Using the Default or Enterprise profiles, Domain Time attempts to automatically detect and synchronizes to the best master on the local segment. When using the Telecom profile, you must must manually select the Telecom profile and configure a list of acceptable masters.
PTP has two delay-measurement mechanisms: End-to-End and Peer-to-Peer. Most master clocks can support either, but only one at a time. By default, Domain Time auto-detects the delay mechanism and uses whichever one the master supports.
Domain Time is compatible with masters providing both one-step Syncs and two-step Syncs (Sync + Followup). When running as a Master, Domain Time Server defaults to acting as a one-step clock, but you may change it to two-step.
PTP is a “chatty” protocol. The master clock sends announce, sync, and other messages on a regular basis, and slaves send queries and responses of their own. The default sync frequency is one per second, but most master clocks allow this number to be adjusted, and administrators may choose a higher number. Theoretically, the higher the rate of messages, the more accurately the slave can measure the master's information, but this is primarily true only for hardware appliances. Domain Time is heavily optimized to perform best at one sync packet per second, so increasing the sync frequency can actually lead to worse performance.
A standard-compliant PTP node using the Default Profile operates using only multicast messages. This means that not only does every node see the master’s messages, but every node sees every request made by every slave, and every reply. Domain Time II tries to reduce this traffic by sending unicast delay requests (Hybrid Mode). The Enterprise Profile also uses unicast delay requests. If the master clock supports multicast with unicast replies, Domain Time II will continue using only unicast for delay measurement with that clock. If the master does not support unicast for delay requests, Domain Time II will fall back to using multicast. Domain Time can also operate as a Telecom profile slave (as of v5.2.b.20180606) or master (as of v5.2.b.20180801). Telecom masters and slaves communicate solely over unicast.
When PTP is enabled on Domain Time, it runs continuously in the background, exchanging messages with the master and collecting statistics. You can see this activity using the logs and graphs from the IEEE 1588 Status and Graph links on the Obtain the Time property page of the Windows control panel applet, or the Graphs and Statistics menu of the DTLinux control panel app in Domain Time II Manager.
Domain Time does not support the experimental PTP authentication mechanisms described in Annex K of 1588-2008. Domain Time supports PTP I588-2019 v2.1 authentication as of version 5.2.b.20190331. See KB article 2019.331 for more information.

Optimum Environment for PTP

The hardware, software, and network environment is critical for PTP synchronization at the sub-millisecond level. This section will help customers hoping to achieve sub-millisecond synchronization across your network of machines:

CPU
The best processors for time synchronization are Intel’s Core i7 line (or later) or Xeon E7 line (or later). Earlier chips are not as stable or as precise as the newer models. The newer processors also have an invariant timestamp counter, which allows Domain Time II to measure the passage of time accurately regardless of SpeedStep or other power-saving mechanisms. On Windows machines, issuing dtcheck -cpuid from the command-line will show you whether or not your processor supports an invariant TSC. On Linux machines, this information is included in the startup banner of the text log.
Operating System
Win8-Win10, or Server 2012-2019 are preferred and are more predictable than Vista, Windows 7, Windows 2008, or Windows 2008r2 for high-accuracy timing. The older XP/Server 2003 platform is also more stable than the problematic Vista/Win7/2008/2008r2 versions.
Domain Time Version
PTP is a loose and evolving standard. You should always use the most current version of Domain Time Server or Client to be sure you are getting optimal performance and compatibility. Check the Changelog to see details of the current version.
Network Switches
The lower the latency in your network switches or hubs, the better. Switches typically use store-and-forward technology, which means that packets can be delayed even under ideal circumstances. During times of high traffic, the delay time can quickly soar to unacceptable levels. Even a small delay will affect timing, even though other applications may not even notice it. If you have managed switches, examine the statistics to determine if packets are being delayed. You may also set up a test environment and compare the synchronization accuracy in your test environment vs. your production environment. If the performance suffers in your production environment with identical equipment, suspect underpowered switches.
Machines using multicast profiles of PTP require IGMP support from either network routers or switches to join multicast groups. Be sure a device acting as an IGMP Querier (often provided with IGMP Snooping features) is present on each of your network segments.
Some switches and routers have PTP-awareness (such those capable of operating as boundary clocks) which can intercept or interfere with PTP communications. Be sure you have updated all of these to the latest firmware. Many implementations of these products have had significant bugs, such as preventing PTP management messages from being propagated correctly.
If your managed switch supports QoS, you should set it so that PTP traffic (to/from UDP ports 319 and 320) have the highest possibly priority.
Other Applications
Most user-mode applications do not affect timing significantly. However, programs that change the system clock resolution, or that overtax the hardware, can degrade performance. If possible, avoid Java-based applications, Flash applications, or any program that exercises the CPU excessively.
Miscellaneous Hardware and Drivers
Drivers for NICs, video, and disk subsystems can cause unpredictable delays in servicing interrupts. Windows timing relies on the regularity and predictability of interrupts. If sub-millisecond timing is critical for your environment, use only the best and fastest hardware. Some Linux distros benefit from changing the NOHZ setting (see Linux Kernel Documentation for more information).
Some switches and routers have PTP-awareness (such those capable of operating as boundary clocks) which can intercept or interfere with PTP communications. Be sure you have updated all of these to the latest firmware. Many implementations of these products have had significant bugs, such as preventing PTP management messages from being propagated correctly.
Virtualization
Any machine running either Hyper-V or VMWare will not perform as well as a stand-alone machine, although modern versions of Hyper-V (as of Server 2012) offer dramatically improved timing performance. This applies to both the host operating system and any guests, using any time synchronization protocol. On older hardware or systems with high resource use/low memory, the extra overhead of handling the continual PTP network activity may actually result in more interrupt delays among guests, resulting in additional clock drift.

Domain Time's PTP support was developed using the Microsemi SyncServer S600
[Click for larger size]
Use a hardware clock as your master, if possible
PTP works best with hardware-based grandmaster clocks. Although you may design a network using only software components, your accuracy will be somewhat diminished. We recommend careful planning to ensure you have hardware clocks available, and that their capacity is commensurate with your expected load. Your grandmaster manufacturer can help you plan your network.

PTP Profiles
The Precision Time Protocol IEEE1588-2008 (PTP) specification includes definition of several operating profiles. A profile is simply a selection of settings and attributes optimized for different environments or purposes. The idea behind profiles is that, as long as all nodes use the same profile, devices can interoperate with a minimum of user configuration.

The default profiles

Although the default profiles specify values for certain parameters, administrators have the ability to change some of them. Many administrator-selectable values are distributed automatically, but several important ones are not. Domain Time automatically queries the master using management messages to discover important variables when possible, minimizing manual configuration. Other PTP devices or implementations may need to be manually configured for these items.

The Delay Request-Response Default PTP Profile (identifier code 00-1B-19-00-01-00) uses End-to-End (E2E) delay measurement. Slaves send periodic delay requests via multicast to the entire network. A master overhearing the request sends a multicast delay response packet with the slave request's sequence number and portIdentity. All nodes on the network hear each request and each response, even though delay requests are only answered by masters, and a particular delay response is only meaningful to the slave listening for that particular packet. The delay request frequency is normally controlled by the master when using E2E. The required interval is sent as part of the master’s regular packets. Domain Time II allows you to override this automatic request frequency and specify a fixed interval.
The Peer-to-Peer Default PTP Profile (identifier code 00-1B-19-00-02-00) uses Peer-to-Peer (P2P) for delay measurements. The operation is very similar to E2E, but is primarily used by Boundary Clocks, or between multiple appliance-type grandmasters, rather than by individual nodes. The P2P request packet is larger, and the response may be either one-step or two-step. Two-step responses are used by masters who can measure the exact departure time of the first reply, but cannot dynamically rewrite the contents of the first reply to include the correct time. The second reply contains a more precise timestamp. P2P uses the master’s announce frequency times as an implementation-dependent multiplier to derive the delay request frequency. Domain Time II allows you to override this automatic request frequency and specify a fixed interval. P2P suffers the same excessive bandwidth issue as E2E, in that all requests and responses are sent to the entire network via multicast.
Bandwidth reduction
Since using multicast for all PTP traffic is an inefficient use of network resources, methods have been introduced to reduce the amount of multicast traffic, primarily by using directed unicast packets for the delay measurement portion of the PTP conversation between masters and slaves.
- Hybrid Mode
  Hybrid Mode is not defined by IEEE1588-2008, but it is in common use. It does not have an identifier code of its own; rather it operates using either E2E or P2P with one important difference: Delay requests are sent directly to the master using unicast. Masters that support hybrid mode will send the reply by unicast directly back to the requesting slave. This allows messages that need to be seen by the entire network (a master's Announces and Syncs) to continue being multicast, but limits delay measurement exchanges to only the nodes that need to see them. Other nodes do not need to process and discard inapplicable delay measurement requests or replies. Domain Time, PTPd, and most other implementations of IEEE1588-2008, support hybrid mode. Most grandmaster appliances also support it. Appliances that do not support hybrid mode may respond to unicast requests with multicast responses, or may not respond at all (the behavior is manufacturer-defined). Domain Time automatically detects both which profile to use and whether or not hybrid mode is supported by the master.
- Unicast Negotiation
  Unicast negotiation is described in Table 34 of the IEEE1588-2008 standard (REQUEST_UNICAST_TRANSMISSION). Negotiated unicast works somewhat like DHCP, in that leases are requested, granted or denied, have expiry times, and must be renewed. This is primarily useful in networks that do not allow multicast, such as older telecommunication networks. Domain Time supports unicast negotiation as a slave (as of v5.2.b.20180606) or master (as of v5.2.b.20180801) using the Telecom Profile.
PTP Domains
IEEE 1588-2008 v2.0 specifies a "domain" to be a number in the range of 0-127 (default 0), that logically separates multiple timing networks on the same wire. Version 2.1 allows domain numbers to extend to 239. Each node is required to have an operating domain number, and to ignore messages originating from or addressed to a different domain number.
- Dynamic Domains
  Two equal grandmasters in the same domain will not both send Announce and Sync messages. Using the BMC algorithm, they will decide which is primary and which is secondary. The secondary grandmaster will enter passive mode, waiting for the primary to go offline or suffer a clock degradation, at which point the roles will switch. This mechanism works as expected as long as the grandmasters use the same domain number.
  Multiple grandmasters visible on the same wire may all act as primaries as long as they each use a different domain number. In this way, networks can be logically separated even if not physically segmented.
  Slaves, too, have an operating domain, and IEEE1588-2008 requires them to exercise the BMC algorithm only among masters advertising in the same domain. The Enterprise Profile (see below) provides for slaves to follow masters in more than one domain, although the mechanism and selection criteria are implementation-dependent. As of version 5.2.b.20161215, Domain Time supports the dynamic domain concept, whether or not the Enterprise Profile is specified. When dynamic domain is enabled, Domain Time slaves will choose the best master from among all visible masters in domains 0 through 127 (or through 239 if using v2.1), and change their operating domain to match the domain of the selected master.
- The "All Domains" Domain Number
  Domain numbers are conveyed in an unsigned 8-bit value. Ordinary v2.0 clocks are restricted to domains 0-127. Values from 128-239 are available for version 2.1 use. Other values are reserved.
  IEEE1588-2008 supports special values for clockIdentities and portNumbers that act as wildcards for management messages. A clockIdentity of all 1's is addressed to "all clocks," and a portNumber of all 1's is addressed to "all ports." Therefore a portIdentity of all 1's means "all ports of all clocks" but is limited to nodes using the same domain.
  Oddly, IEEE1588-2008 does not include a wildcard "all domains" value. An 8-bit value of all 1's is 255 (0xFF), and is reserved by the standard, so is the perfect candidate for a domain wildcard value. Domain Time will respond to management messages addressed to domain 255. The reply will contain the node's current operating domain. In theory, a management station could send a single multicast addressed to all clocks, all ports, all domains, and, by collecting the responses, discover the identities of each node, including all of the logical domains in use.
  Because the "all domains" domain wildcard is not in PTPv2.0 or v2.1, Domain Time does not place messages addressed to the wildcard domain on the wire, but will respond correctly to management messages addressed this way.

Other Profiles
Various additional profiles have been introduced to address specialized needs or industry requirements.

The Enterprise Profile (identifier code 00-00-5E-00-01-00, see draft 09 et sequelae) is an emerging standard that is already widely deployed. It is a specialized version of the default End-to-End Profile, and is interoperable with the default End-to-End Profile. It provides potentially tighter synchronization, allows for slaves to follow masters in multiple domains, and eliminates the need to query the masters for network variables.
Domain Time, in auto-detect mode, will use the values provided by the master, regardless of whether the master is using the End-to-End Default Profile or the Enterprise Profile. As of version 5.2.b.20170101, you may force Domain Time to use the Enterprise Profile.
When using the Enterprise Profile as a slave, Domain Time will not query the master; it always uses the values specified by the profile, and always uses hybrid mode. When using the Enterprise Profile as a master, Domain Time uses the values specified by the profile, will respond to multicast queries with multicast responses, and will respond to unicast queries with unicast responses.
Note that although the Enterprise Profile's basic goal is largely designed around codification of hybrid mode, it also sets several values intended to be fixed. As with the default profiles, manufacturers can vary some of these values. In particular, a master may send Sync messages more often than once per second, and it may provide, when queried, a different delay measurement interval for slaves to use.
The Telecom Profile (identifier code 00-19-A7-00-01-00) is not one of the default profiles and is not compatible with them. Each slave node must be configured with the IP address and domain number of the master(s) it is allowed to use. The slave then obtains separate leases for Announces, Syncs, and E2E delay measurements by exchanging unicast messages with the master(s). The Telecom Profile uses a custom BMC algorithm, as defined by ITU-T G.8265.1, using the master's clockClass, the master's priority 2 value, and finally the local priority. Local priority is based on the order in which you provision the list of masters. The Telecom Profile requires that a given node is either always a slave or always a master; this is why Domain Time, when configured to use the Telecom Profile as a slave, can never assume the master role. Domain Time supports unicast negotiation as a slave (as of v5.2.b.20180606) or master (as of v5.2.b.20180801).
Since you must provision a Telecom Profile slave with the master's IP and domain number, the Dynamic Domain checkbox will be grayed-out. Domain Time can follow up to sixteen Telecom masters in any combination of domains.
The 802.1AS Profile (identifier code 00-80-C2-00-01-00) is aimed primarily at the audio-video industry. This is a specialized version of the default Peer-to-Peer Profile using multicast for all messages, and different requirements for certain types of message contents. Its primary goal is to provide reliable timestamps across multiple nodes used for streaming source material. It is not, generally, interoperable with the default Peer-to-Peer Profile, and there are several mutually-incompatible versions in common use. Domain Time can usually slave successfully to an 802.1AS master, but compatibility must be determined on a trial-and-error basis due to the conflicting definitions.
The SMPTE ST-2059-2 Professional Broadcast Environment Profile (identifier code 68-97-E8-00-01-00) for video synchronization. The profile is similar to the Default Profile, but requires different scheduling of Announce and Sync messages, as well as using Domain 127. Domain Time should be able to synchronize as a slave using this profile.

Proceed to the Timings page

Back to the Obtain the Time page