DAG Cards can capture and transmit packets at line rate. Using the DAG Card’s Playback function packets can be re-transmitted precisely as they were captured - using the captured packet time stamp to determine re-transmission timing in hardware. The traffic Playback rate can also be precisely scaled up or down or a specific packet rate or bandwidth can be selected.
Repeatedly playing back traffic, using the precise timing with which it was captured, lets analysts benchmark the performance impact of enabling a new traffic processing rule or changing a processing setting in their analytics tools. They can turn on the new rule, play back the traffic again, and measure the difference in throughput with the new rule activated. Doing this testing offline before modifying production systems optimizes tuning while removing or reducing the need for maintenance windows. In fact, many of our customers use DAG cards for exactly this type of performance testing – algorithmic tuning in the high-frequency trading industry is a good example.
Filtering and Classification
DAG Cards include sophisticated, hardware-based filtering and classification features that operate on captured packets at 100% line rate. Multiple filter rules can be applied to each packet. Fields that can be filtered on include IPv4 and IPv6 addresses or subnets, VLAN VIDs, MPLS labels, TCP/UDP ports and more.
The resulting per-packet classification value can be used to filter out unwanted traffic, or steer packets to per-application streams. Filtering out unwanted traffic in hardware reduces processing load on the receiving application, while also reducing the amount of memory and disk space needed to store recorded packets.
DAG Cards also generate bi-directional flow hashes using a configurable set of fields, allowing flow-safe load-balancing with zero CPU overhead. Filtering and load-balancing can operate simultaneously. For example, load-balancing flows across 31 individual application threads while duplicating all DNS traffic to a 32nd stream.
Precision timing is critical for accurate performance measurement, and for correlating events across a network. Increasingly, financial market regulations - such as MiFID 2 - are imposing strict compliance obligations on traders to ensure highly accurate time stamps are recorded alongside archived trade data. DAG Cards record a high precision time stamp for every packet received, on every port, in hardware. This eliminates the inaccuracies, inherent in the software time-stamping used by NICs, that can occur due to coalescing, interrupt latencies, and receive offloads. Additionally, hardware time-stamping is extremely precise, offering resolution to around 5ns.
Synchronized time stamps allow for easy correlation of network events, and measurement of network and application performance even across geographically distributed datacenters or even different continents. DAG cards can be synchronized to within 100 nanoseconds of UTC using external reference clocks connected to a GPS receiver. DAG Cards have a dedicated time-signal port and support 1PPS and IRIG-B signals, as well as IEEE 1588-2008 Precision Time Protocol (PTP) over Ethernet for long reach and wide compatibility.
For further discussion about PTP timing accuracy and a discussion of PTP time accuracy please refer to our PTP Timing Whitepaper.
Provenance™ is a mechanism for adding context to network packet capture data automatically as the packets are captured – like the meta data in digital images. It stores this context information in sequential records written into the packet stream as the packets are recorded – so the context always lives with the packet data.
Provenance is supported by both Endace DAG cards and EndaceProbe™ Network Analytics Platform. Like pcapng the next generation pcap format, Provenance provides a mechanism for adding context to network packet capture data. But it differs in a significant way from pcapng in its basic design.
Whereas pcapng allows metadata to be added in header blocks, Provenance writes contextual data into sequential records in the packet stream as the packets are recorded. This makes it ideal for recording data that can vary over the course of the packet capture - such as optical power levels or the amount of clock drift from a time signal source.
Provenance is written into recorded packet streams that use ERF (Extensible Record Format) format and Provenance records can be read directly in tools, such as Wireshark, that support ERF format packet capture files.
If you are interested in how Provenance™ can help high-speed traders meet regulatory time accuracy obligations, such as MiFID 2’s RTS-25 requirement, please refer to our Solution Brief “Using Provenance to satisfy MiFID 2 regulatory timing demands”.