Information and configuration instructions for using the Precision Time Protocol (IEEE 1588-v2) with Domain Time II.
A PTP network of hardware clocks on a well-regulated segment can usually synchronize to within a few tens or hundreds of nanoseconds. Software implementations can theoretically achieve the same results, but are subject to limitations of the host operating system. In particular, Windows, Linux, and BSD are not real-time operating systems, and the connecting networks are often congested or have unpredictable or asymmetric performance characteristics. PTP implementations in these environments must estimate delays caused by servicing interrupts, stack traversal, latencies introduced by expiring time slice quanta, and vagaries in the network itself.
By default, when an application tells the operating system to place a packet on the wire, it can know only when the packet left the application, not when the packet finished traversing the operating system layers and was actually placed on the wire. Likewise, incoming packets may suffer delays in the network driver or the operating system before being delivered to the application. This degree of uncertainty produces ďjitter,Ē or small variations in the measured time that are artifacts of the environment rather than the protocol. In addition, any procedure can be preempted, leading to erroneous measurements of elapsed time.
Some operating systems, with assistance from the network driver, can provide a timestamp indicating when a particular packet left or entered the system. Using this information, the application can determine how much extra time was spent handling the packet between the application and the wire. The BSD-style socket options SO_TIMESTAMP or SO_TIMESTAMPNS are used to enable this functionality on BSD or Linux. If the operating system and network driver support SO_TIMESTAMP or SO_TIMESTAMPNS, the measurements of PTP can be significantly more precise than without such support.
Current Windows operating systems do not support SO_TIMESTAMP or SO_TIMESTAMPNS. When Windows adds this support, and when NIC manufacturers add support in their Windows drivers, any application designed to take advantage of the extra information will benefit.
The theoretical limit of precision for Domain Time II using any protocol is the hectonanosecond. A hectonanosecond is 100 nanoseconds, or 1/10th of a microsecond. The real-world behavior of Domain Time IIís implementation of PTP achieves results as good or better than any other supported protocol, and has the potential for even better results when operating system and NIC support become available.
The presence of PTP on a network gives Domain Time II the opportunity to adjust the timing of the system up to 64 times per second. This is not possible with conventional request-response time protocols because the overhead of making requests are prohibitive.
Domain Time II supports Precision Time Protocol version 2 (IEEE 1588-2008). PTP is designed to be mostly configuration-free. Master clocks on the local segment exchange periodic messages with Domain Time II in order to determine the current time offset.
Domain Time II normally runs in slave-only mode, with PTP providing time samples which Domain Time II uses to manage the machineís clock. This applies to both Domain Time II Client and Domain Time II Server, each of which requires a reliable time source.
Domain Time II Server allows you to let Domain Time II assume the master clock role. This option should only be enabled if you have no hardware-based PTP grandmaster, or if you need a software-based backup to your grandmaster. PTP is designed to work with hardware-based clocks. Software-based clocks cannot offer the same degree of precision or accuracy as hardware-based clocks.
Domain Time II automatically detects and synchronizes to the best master on the local segment.
PTP has two delay-measurement mechanisms, End-to-End and Peer-to-Peer. Most master clocks can support either, but only one at a time. Domain Time II auto-detects the delay mechanism and uses whichever one the master supports.
PTP is a ďchattyĒ protocol. The master clock sends messages on a regular basis. The default sync frequency is one per second, but most master clocks allow this number to be adjusted, and many administrators choose a higher number. In general, the higher the rate of messages, the more accurately the slave can measure the masterís information but at the cost of increased network traffic and measurement artifacts. Typically, a sync frequency of 2 to 4 times per second provides optimal results.
A standard-compliant PTP node operates using only multicast messages. This means that not only does every node see the masterís messages, but every node sees every request made by every slave, and every reply. Domain Time II tries to reduce this traffic by sending unicast delay requests (Hybrid Mode). If the master clock supports unicast replies, Domain Time II will continue using only unicast with that clock. If the master does not support unicast for delay requests, Domain Time II will fall back to using multicast.
When PTP is enabled on Domain Time II, it runs continuously in the background, exchanging messages with the master and collecting statistics. You can see this activity using the Domain Time II control panel applet. At its next regularly scheduled timecheck, Domain Time II coalesces the filtered results of PTPís statistics as a single time sample. You may choose to use the samples without coalescing them, in which case they are filtered along with whatever time sources you have configured.
PTPv1 is not supported. PTPv1 and PTPv2 are incompatible. A boundary clock is required to separate v1 and v2 segments. Domain Time II will not recognize or respond to PTPv1 messages.
Domain Time II does not support the experimental PTP authentication mechanisms.
Optimum Environment for PTP
The hardware, software, and network environment is critical for PTP synchronization at the sub-millisecond level. This section will help customers hoping to achieve sub-millisecond synchronization across your network of machines:
The best processors for time synchronization are Intelís Core i7 line (or later) or Xeon E7 line (or later). Earlier chips are not as stable or as precise as the newer models. The newer processors also have an invariant timestamp counter, which allows Domain Time II to measure the passage of time accurately regardless of SpeedStep or other power-saving mechanisms. Issuing
DTCHECK /cpuid from the command-line will show you whether or not your processor supports an invariant TSC.
Win8/2012 or newer versions are preferred and are more predictable than Vista, Windows 7, or Windows 2008 for high-accuracy timing. The older XP/Server 2003 platform is also more stable than the problematic Vista/Win7/2008 versions.
The lower the latency in your network switches or hubs, the better. Switches use a store-and-forward technology, which means that packets can be delayed even under ideal circumstances. During times of high traffic, the delay time can quickly soar to unacceptable levels. Even a small delay will affect timing, even though other applications may not even notice it. If you have managed switches, examine the statistics to determine if packets are being delayed. You may also set up a test environment and compare the synchronization accuracy in your test environment vs. your production environment. If the performance suffers in your production environment with identical equipment, suspect underpowered switches.
Most user-mode applications do not affect timing significantly. However, programs that change the system clock resolution, or that overtax the hardware, can degrade performance. If possible, avoid Java-based applications, Flash applications, or any program that exercises the CPU excessively.
Miscellaneous Hardware and Drivers
Drivers for NICs, video, and disk subsystems can cause unpredictable delays in servicing interrupts. Windows timing relies on the regularity and predictability of interrupts. If sub-millisecond timing is critical for your environment, use only the best and fastest hardware.
Any machine running either Hyper-V or VMWare will not perform as well as a stand-alone machine, although modern versions of Hyper-V (as of Server 2012) offer dramatically improved timing performance. This applies to both the host operating system and any guests, using any time synchronization protocol.
On older hardware or systems with high resource use/low memory, the extra overhead of handling the continual PTP network activity may actually result in more interrupt delays among guests, resulting in additional clock drift.
Use a hardware clock as your master, if possible
PTP works best with hardware-based grandmaster clocks. Although you may design a network using only software components, your accuracy will be somewhat diminished. We recommend careful planning to ensure you have hardware clocks available, and that their capacity is commensurate with your expected load. Your grandmaster manufacturer can help you plan your network.
To Configure Domain Time as a PTP Slave:
In order to use Domain Time as a PTP Slave to an existing Master, you must make the following configuration choices on the Obtain the Time property page on the Domain Time applet:
Choose the Set this machine's clock by querying a list of servers radio-button.
On versions prior to 5.2.b.20150828, you must uncheck the Analyze time samples from all servers and choose the best checkbox to prevent skewing of the time by the additional sample analysis from sources other than PTP.
However, on current versions, you may leave this box checked, since non-PTP samples are now automatically excluded. This allows you to have better performance when falling back to DT2 or NTP sources without degrading the PTP performance in normal operation.
Uncheck the If all listed servers fail, try to discover sources automatically checkbox to prevent auto-discovery of NTP or DT2 time sources.
Specify at least one reliable non-PTP server (either NTP or DT2) in the List of backup time sources to use. field to act as fallback server(s). This way, when PTP hasnít synchronized yet or when it fails because the master clock goes offline, Domain Time II will have another source for time.
On the Timings property page, set both the Check Interval when able to get and correct the time and Check Interval when getting the time fails settings to use the same fixed period, i.e. every 1 minute.
Although these settings do not control PTP synchronization rates, it does control how often Domain Time synchronizes if falling back to DT2 or NTP, and also how often it records sync status in the sync logs and performs other alerting functions.
Note: PTP does not begin synchronizing immediately when starting. It must listen to the network to detect the master clock, and must then calibrate to it before usable time samples are available to the Domain Time II clock management algorithms.
This means that for a period after startup, or if PTP sync is lost, Domain Time II will not be able to use PTP as a time source. Having a backup NTP or DT2 time source ensures the system has a good source of time in the interim. Also,
PTP can take a fairly long time to correct large variances, so having a fallback server at startup avoids having a long training period for PTP to bring the clock into conformance.
Check the Enable IEEE 1588 Precision Time Protocol (PTP) checkbox. Domain Time will then start listening for Master announcements, and will automatically calibrate and start synchronizing with the Master.
You may check the activity of the PTP conversation by clicking the Status and Graph links.
Troubleshooting The default PTP settings should be sufficient for Domain Time to synchronize with most Master clocks. However, you may encounter some network issues or non-compliant Master servers that prevent synchronization. Here are some steps you can try:
Verify that multicasts are allowed on your network. See the Planning documentation for network requirements.
Ensure that there is an IGMP-enabled router attached to the same subnet as your PTP Master and Slaves. IGMP is required to join multicast groups. Unlike with unicast protocols like NTP, a direct cable connection to a clock with only a crossover cable will not work.
Enable network communication through firewalls. You may quickly open all relevant time protocol ports in the Windows Firewall using the following command as Administrator from a command-prompt (Version 5.2 and later):
NOTE: Certain versions of Windows block these ports by default regardless of whether the Windows Firewall service is running. The Windows Firewall service must be started to run the above command. If the service is disabled, you must re-enable it temporarily to run the command, then re-disable the service if you wish.
Ensure that the following checkboxes are checked on the Network property page:
Initiate rebind and resync if IP address changes Enumerate multicast interfaces during IPv4/IPv6 bind Reply to multicasts using incoming interface if possible
Be sure you have not excluded the IP address of your PTP Grandmaster using the Best Master Clock Options settings.
To easily see which PTP masters are visible on your network, issue the following command from a command-prompt:
If your machine is multihomed, you may need to restrict Domain Time to listen only on the proper subnet using the Listen only on these addresses: list box on the Network property page.
Try relaxing strict IEEE 1588 requirements. Click the Options link to display the IEEE Precision Time Protocol Options page.
Then, click the button to display the IEEE 1588 Precision Time Protocol Advanced Settings dialog page.
First, uncheck the Require strict IEEE 1588-2008 standard compliance (recommended) checkbox. Test to see if this configuration now allows synchronization.
Next, uncheck the Require ptpTimescale (IEEE 15878-2008 section 184.108.40.206) checkbox. Test to see if this configuration now allows synchronization.
Take care with this option. If unchecked, your Master clock may be using a different timescale than expected, resulting in the time being off. Check with your clock manufacturer to be sure the Master clock is using the correct timesecale.
Specify a PTP profile if one isn't being autodetected. On the IEEE Precision Time Protocol Options page, you may use the PTP Profile dropdown to specify the profile used by your Master clock.
Specify a defined Delay Transport option instead of "Auto-detect." Disable Unicast support if necessary.
Ensure the Clock Identity value is unique on your network. The default value is derived from the MAC address of your primary network adapter during installation.
You must change this value to a different guaranteed-unique value if the PTP master to which you want to sync is on the same NIC and/or has the same Clock Identity. Otherwise, do not change this value.
The Precision Time Protocol IEEE1588-2008 (PTP) specification includes definition of several operating profiles.
A profile is simply a selection of settings and attributes optimized for different environments or purposes.
The idea behind profiles is that, as long as all nodes use the same profile, devices can interoperate with a minimum of user configuration.
The default profiles
IEEE 1588-2008 Annex J specifies two main default profiles for use with PTP. The profiles are identical except for the method of delay measurement specified.
Both of the default profiles use the IEEE1588-2008 defined "Best Master Clock" (BMC) algorithm to determine which node on the network is master.
It is not unusual for a network to consist of nodes using either of the default protocols.
Although the default profiles specify values for certain parameters, administrators have the ability to change some of them.
Many administrator-selectable values are distributed automatically, but several important ones are not. Domain Time automatically
queries the master using management messages to discover important variables when possible, minimizing manual configuration.
Other PTP devices or implementations may need to be manually configured for these items.
The Delay Request-Response Default PTP Profile (identifier code 00-1B-19-00-01-00) uses End-to-End (E2E) delay measurement.
Slaves send periodic delay requests via multicast to the entire network. A master overhearing the request sends
a multicast delay response packet with the slave request's sequence number and portIdentity. All nodes on the network hear each
request and each response, even though delay requests are only answered by masters, and a particular delay response is only
meaningful to the slave listening for that particular packet. The delay request frequency is normally controlled by the
master when using E2E. The required interval is sent as part of the masterís regular packets. Domain Time II allows you to override
this automatic request frequency and specify a fixed interval.
The Peer-to-Peer Default PTP Profile (identifier code 00-1B-19-00-02-00) uses Peer-to-Peer (P2P) for delay measurements.
The operation is very similar to E2E, but is primarily used by Boundary Clocks, or between multiple appliance-type grandmasters,
rather than by individual nodes. The P2P request packet is larger, and the response may be either one-step or two-step.
Two-step responses are used by masters who can measure the exact departure time of the first reply, but cannot dynamically
rewrite the contents of the first reply to include the correct time. The second reply contains a more precise timestamp.
P2P uses the masterís announce frequency times as an implementation-dependent multiplier to derive the delay request frequency.
Domain Time II allows you to override this automatic request frequency and specify a fixed interval.
P2P suffers the same excessive bandwidth issue as E2E, in that all requests and responses are sent to the entire network via multicast.
Since using multicast for all PTP traffic is an inefficient use of network resources, methods have been introduced to
reduce the amount of multicast traffic, primarily by using directed unicast packets for the delay measurement portion of
the PTP conversation between masters and slaves.
Hybrid Mode is not defined by IEEE1588-2008, but it is in common use. It does not have an identifier code of its own;
rather it operates using either E2E or P2P with one important difference: Delay requests are sent directly to the master
using unicast. Masters that support hybrid mode will send the reply by unicast directly back to the requesting slave.
This allows messages that need to be seen by the entire network (a master's Announces and Syncs) to continue being multicast,
but limits delay measurement exchanges to only the nodes that need to see them. Other nodes do not need to process and discard
inapplicable delay measurement requests or replies. Domain Time, PTPd, and most other implementations of IEEE1588-2008,
support hybrid mode. Most grandmaster appliances also support it. Appliances that do not support hybrid mode may respond to
unicast requests with multicast responses, or may not respond at all (the behavior is manufacturer-defined).
Domain Time automatically detects both which profile to use and whether or not hybrid mode is supported by the master.
Unicast negotiation is described in Table 34 of the IEEE1588-2008 standard (REQUEST_UNICAST_TRANSMISSION).
Negotiated unicast works somewhat like DHCP, in that leases are requested, granted or denied, have expiry times, and
must be renewed. This is primarily useful in networks that do not allow multicast, such as older telecommunication networks.
Domain Time II does not use this method, since the management mechanism of the standard is awkward, liable to error, does
not scale well, and requires constant renegotiation for each slave with each master for each type of message.
IEEE1588-2008 specifies a "domain" to be a number in the range of 0-127 (default 0), that logically separates multiple timing
networks on the same wire. Each node is required to have an operating domain number, and to ignore messages originating from
or addressed to a different domain number.
Two equal grandmasters in the same domain will not both send Announce and Sync messages. Using the BMC algorithm, they will decide which
is primary and which is secondary. The secondary grandmaster will enter passive mode, waiting for the primary to go offline or suffer
a clock degradation, at which point the roles will switch. This mechanism works as expected as long as the grandmasters use the same
Multiple grandmasters visible on the same wire may all act as primaries as long as they each use a different domain
number. In this way, networks can be logically separated even if not physically segmented.
Slaves, too, have an operating domain, and IEEE1588-2008 requires them to exercise the BMC algorithm only among masters advertising
in the same domain. The Enterprise Profile (see below) provides for slaves to follow masters in more than
one domain, although the mechanism and selection criteria have yet to be determined. As of version 5.2.b.20161215, Domain
Time supports the dynamic domain concept, whether or not the Enterprise Profile is specified. When dynamic domain is enabled,
Domain Time slaves will choose the best master from among all visible masters in domains 0 through 127, and change
their operating domain to match the domain of the selected master.
The "All Domains" Domain Number
Domain numbers are conveyed in an unsigned 8-bit value. Ordinary clocks are restricted to domains 0-127. Values from 128-255 are
reserved, with some of them allocated by the forthcoming PTPv3 specification, but 255 is never used.
IEEE1588-2008 supports special values for clockIdentities and portNumbers that act as wildcards for management messages. A clockIdentity
of all 1's is addressed to "all clocks," and a portNumber of all 1's is addressed to "all ports." Therefore a portIdentity of all 1's
means "all ports of all clocks" but is limited to nodes using the same domain.
Oddly, IEEE1588-2008 does not include a wildcard "all domains" value. An 8-bit value of all 1's is 255 (0xFF), and is reserved by the
standard, so is the perfect candidate for a domain wildcard value. Domain Time will respond to management messages addressed to
domain 255. The reply will contain the node's current operating domain. In theory, a management station could send a single multicast
addressed to all clocks, all ports, all domains, and, by collecting the responses, discover the identities of each node, including
all of the logical domains in use.
Because the "all domains" domain wildcard is not in PTPv2, and PTPv3 is still under development, Domain Time does not place messages
addressed to the wildcard domain on the wire, but will respond correctly to management messages addressed this way.
Various additional profiles have been introduced to address specialized needs or industry requirements.
The Enterprise Profile (no profile identifier assigned yet) is an emerging standard that is already widely deployed.
It is a specialized version of the default End-to-End Profile, and is interoperable with the default End-to-End Profile.
It provides potentially tighter synchronization, allows for slaves to follow masters in multiple domains, and eliminates the
need to query the masters for network variables.
Domain Time, in auto-detect mode, will use the values provided by the master, regardless of whether the master is using the
End-to-End Default Profile or the Enterprise Profile. As of version 5.2.b.20170101, you may force Domain Time to use the
When using the Enterprise Profile as a slave, Domain Time will not query the master; it always uses the
values specified by the profile, and always uses hybrid mode. When using the Enterprise Profile as a master, Domain Time uses
the values specified by the profile, will respond to multicast queries with multicast responses, and will respond to unicast
queries with unicast responses.
Note that although the Enterprise Profile's basic goal is largely designed around codification of hybrid mode, it also sets
several values intended to be fixed. As with the default profiles, manufacturers can vary some of these values. In particular,
a master may send Sync messages more often than once per second, and it may provide, when queried, a different delay measurement
interval for slaves to use.
— This profile is a codification of hybrid mode, with fixed values for packet intervals
— Uses a fixed announce multiplier of 3 through the network
— Requires masters to use a fixed announce interval of 1
— Requires masters to use a fixed sync interval of 1
— Requires masters to use multicast for Announce and Sync messages
— Requires masters to respond to unicast delay requests with unicast delay responses
— Slaves use a fixed delay measurement interval of 1
— Slaves are forbidden from attempting unicast negotiation
— Slaves are required to use E2E instead of P2P for delay measurement
— Slaves are allowed to choose from masters in multiple domains
— Slaves are encouraged to use unicast for delay requests
— Nodes are not forbidden from using multicast for management messages
— Masters are not forbidden from responding to multicast messages via multicast
The Telecom Profile (identifier code 00-19-A7-00-01-00) is not one of the default profiles and is not compatible with them.
Each slave node must be configured with the IP address and portIdentity of the master(s) it is allowed to use. The slave then obtains
separate leases for Announces, Syncs, and delay measurements by exchanging unicast messages with the master(s). The BMC algorithm must be modified
to account for no advertisements from masters that haven't granted a lease, and normally assumes that a slave using the Telecom
Profile will never become a master itself. Although it uses unicast negotiation (described above), the overall traffic is not lessened,
since the various message types must be periodically renegotiated, and since the master must send separate Announces and Syncs to
each subscriber. Domain Time does not support unicast negotiation and is not compatible with the Telecom Profile.
The 802.1AS Profile (identifier code 00-80-C2-00-01-00) is aimed primarily at the audio-video industry.
This is a specialized version of the default Peer-to-Peer Profile using multicast for all messages, and different requirements for
certain types of message contents. Its primary goal is to provide reliable timestamps across multiple nodes used for streaming
source material. It is not, generally, interoperable with the default Peer-to-Peer Profile, and there are several mutually-incompatible
versions in common use. Domain Time can usually slave successfully to an 802.1AS master, but compatibility must be determined on a
trial-and-error basis due to the conflicting definitions.
While PTP is designed to operate on the local segment without any configuration by the administrator, Domain Time II nevertheless exposes a few options for customization.
Networks of clocks using PTP are separated into logical ďdomains,Ē or groups of clocks. Nodes will only respond to messages sent
to their domain. The domain is specified by a number, and the default domain is 0 (zero). Any number between 0 and 127 inclusive is a valid domain number.
If you change this setting on your master clock, you must change each Domain Time II machine to match.
As of version 5.2.b.20170101, you may use the Dynamic Domains feature by checking the Dynamic (allow any domain when slave) checkbox.
The default operation of Domain Time II is to detect the delay mechanism used by the master clock. The options are:
End-To-End Default Profile (00-1B-19-00-01-00)
End-To-End Enterprise Profile
Peer-to-Peer Default Profile (00-1B-19-00-02-00)
Disable link delay measurement
The default operation of Domain Time II is to test the master clock for unicast delay support and use unicast if possible. The options are:
Unicast (hybrid mode)
Multicast (standard mode)
The default operation of Domain Time IIís support for PTP over IPv6 is to listen and send on the site-local scope. You may override this to specify link-local scope instead.
Best Master Clock Options
Domain Time II is a software-based implementation of PTP. The default operation according to the standard is for Domain Time II to discover and synchronize with the best master clock available on the local segment. If that clock goes offline and another clock assumes the master role, or if another clock claims to be more authoritative, Domain Time II will begin synchronizing to the new master automatically. The standard behavior is sometimes undesirable. A user could bring up a new master clock on the network that, either by intention or mistake, assumes the master role without providing better time.
Domain Time II optionally allows you to override the automatic selection of master clock by specifying the IP address(es) or CIDR mask(s) of the master(s) to which you are willing to synchronize. When you override the automatic selection of master clock, Domain Time II only pays attention to the IP addresses you specify; messages from other IP addresses are silently discarded.
Note: Because PTP nodes automatically choose a master according to the algorithm specified in the standard, specifying a list of machines for synchronization may result in no server being available because the other nodes may elect a master that you have chosen to ignore. Use this option with caution.
For how to configure other available options, please see the IEEE 1588-2008 specification for explanations and recommended settings. Also, contact your vendor for device-specific recommendations/requirements for compatibility with their PTP products.