Last edited 2 months ago

How to configure TSN endpoint

Applicable for STM32MP25x lines

1. Purpose[edit source]

This article describes how to configure TSN (Time-Sensitive Networking) endpoint
This article provides different steps to execute TSN scenario endpoint:

  • Need synchronization across the network. TSN requires precise time synchronization to ensure that time-critical data is delivered on time and in the correct order.
  • IEEE 802.1Qav
  • IEEE 802.1Qbv

2. IEEE 802.1AS-Rev or gPTP synchronization[edit source]

Testing using ptp4l tool from linuxptp project

To check the ptp clock adjustment with PTP protocol, a PTP with ordinary clock (client) and a PTP Grandmaster (server) applications are needed to run on separate devices. Open source application package linuxptp can be used.

Create file /etc/gPTP.cfg with content as below:

 [global]
 gmCapable		1
 priority1		248
 priority2		248
 logAnnounceInterval	0
 logSyncInterval		-3
 syncReceiptTimeout	3
 neighborPropDelayThresh	800
 min_neighbor_prop_delay	-20000000
 assume_two_step		1
 path_trace_enabled	1
 follow_up_info		1
 transportSpecific	0x1
 ptp_dst_mac		01:80:C2:00:00:0E
 network_transport	L2
 delay_mechanism		P2P
Warning white.png Warning
Be careful if you are using PHY which managed “low power mode” (i.e EEE)

Most of the time this feature is not compatible with TSN. To be considered “asCapable” the maximum acceptable value for neighborPropDelayThresh is 800ns


Hence TX timestamp generation can be delayed (especially with low speed links) the ptp4l “tx_timestamp_timeout” parameter need to be set for ptp4l to work. The default value for "tx_timestamp_timeout" is 5 seconds To change the value of "tx_timestamp_timeout", you can add or modify the following line in the configuration file:

tx_timestamp_timeout <timeout_value>

PTP synchro between two boards:

  ptp4l -i end0 -p /dev/ptp0 -m -f /etc/gPTP.cfg
Warning white.png Warning
if several Ethernet instances, ptp0 is not necessarily affected to end0, to verify which instance is affected to ptp0:

ls -l /sys/class/ptp/ptp0


PPS (Pulse Per Second) is a signal that is commonly used to check the synchronization of a system clock with a reference clock, such as a PTP grandmaster clock. To check the synchronization of the system clock with the reference clock using PPS, you can output signal PPS on gpio pin to control it on oscilloscope. More details in Ethernet_device_tree_configuration#How_to_configure_PTP_PPS_out_-3F

3. TSN 802.1Qav[edit source]

An example configuration with 4 reserved bandwidth given below:

  • Uses 4 Queues (Q0-Q3). Each Q has a bandwidth reserved in h/w. Maximum 4 Queues supported
  • Queue 0 has No Bandwidth Restriction, general traffic
  • Queue 1 has 10% of Maximum bandwidth reserved (100Mbps)
  • Queue 2 has 20% of Maximum bandwidth reserved (200Mbps)
  • Queue 3 has 30% of Maximum bandwidth reserved (300Mbps)
  • Q3 is the highest priority Queue and Q0 is the lowest priority
Alternate text
4 queues

Here are the steps to configure this setup.

Warning white.png Warning
If you want to perform CBS test, gPTP synchronization is not needed



Bring up end0 interface

ip link add link end0 name end0.200 type vlan id 200
ip link set end0.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
ifconfig end0.200 192.168.200.60 netmask 255.255.255.0

CBS is used to enforce a Quality of Service by limiting the data rate of a traffic class, to separate packets into traffic classes the user may choose mqprio(8) https://man7.org/linux/man-pages/man8/tc-mqprio.8.html, and configure it like this:

 tc qdisc add dev end0 clsact
 tc qdisc add dev end0 handle 100: parent root mqprio \
  num_tc 4 \
  map 0 0 1 1 2 2 3 3 3 3 3 3 3 3 3 3 \
  queues 1@0 1@1 1@2 1@3 hw 0

Configure Bandwidth allocated for each queue (https://man7.org/linux/man-pages/man8/tc-cbs.8.html)

 // Set bandwidth of queue 1 to be 100Mbps
 tc qdisc replace dev end0 parent 100:2 cbs \
  idleslope 100000 \
  sendslope -900000 \
  hicredit 150 \
  locredit -1350 \
  offload 1
 // Set bandwidth of queue 2 to be 200Mbps
 tc qdisc replace dev end0 parent 100:3 cbs \
  idleslope 200000 \
  sendslope -800000 \
  hicredit 300 \
  locredit -1200 \
  offload 1
 // Set bandwidth of queue 3 to be 300Mbps
 tc qdisc replace dev end0 parent 100:4 cbs \
  idleslope 300000 \
  sendslope -700000 \
  hicredit 450 \
  locredit -1050 \
  offload 1

So for queue 0 the bandwidth should be around 400Mbps (port_transmit_rate - sum of all bandwidth of each queue)

sendslope = idleslope - port_transmit_rate
hicredit = max_interference_size * (idleslope / port_transmit_rate)
locredit = max_frame_size * (sendslope / port_transmit_rate)
with max_interference_size = max_frame_size = 1500

For more details: https://en.wikipedia.org/wiki/Time-Sensitive_Networking#IEEE_802.1Qav_Forwarding_and_queuing_enhancements_for_time-sensitive_streams


Network core and Driver uses the skb priority to deliver frames to specific h/w queues. In the above case, priority 6 to 15 SKB (packet) goes to Q3 (4th entry in map in the tc qdisc command), priority 4 to 5 SKB goes to Q2 (3rd entry in map) priority 2 to 3 SKB goes to Q1 (2nd entry in map) and priority 0 to 1 SKB goes to Q0 (1st entry in map)

Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to end0 with IP address 192.168.200.30

iperf3 -s -p 5003&
iperf3 -s -p 5002&
iperf3 -s -p 5001&
iperf3 -s -p 5000&

At DUT, start transmission of stream using iperf3

//Stream with SKB priority 6  : TOS=0x14 == SO_PRIORITY=6
// https:man7.org/linux/man-pages/man8/tc-prio.8.html
iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4
iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
//Stream with SKB priority 2
iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort)
iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60  &

Result without tc rule apply :
For Traffic with priority 6:

[  5]   0.00-60.00  sec  1.40 GBytes   200 Mbits/sec  0.046 ms  11841/1031880 (1.1%)  receiver

For Traffic with priority 4:

[  5]   0.00-60.00  sec  40.1 MBytes  5.60 Mbits/sec  39.651 ms  936/29490 (3.2%)  receiver

For Traffic with priority 2:

[  5]   0.00-60.00  sec  4.94 GBytes   707 Mbits/sec  0.035 ms  164701/3768910 (4.4%)  receiver

For Traffic with priority 0 (Best effort):

[  5]   0.00-60.00  sec   106 MBytes  14.8 Mbits/sec  0.017 ms  1765/76956 (2.3%)  receiver

Without CBS TSN rules in place, multiple traffic streams may contend for network resources, causing congestion and possible packet loss. This can lead to poor network performance and lower quality of service for users. Furthermore, without CBS TSN rules, there no prioritization of important traffic, such as real-time audio or video streams, which can cause delays and interruptions in these applications. In summary, the absence of CBS TSN rules can create an unstable and undependable network environment.


Result with tc rule apply:
For Queue 3 with priority 6 and with BW of 300Mbps:

[  5]   0.00-60.00  sec  1.72 GBytes   246 Mbits/sec  0.052 ms  189322/1444777 (13%)  receiver

For Queue 2 with priority 4 and with BW of 200Mbps:

[  5]   0.00-60.00  sec  1.16 GBytes   167 Mbits/sec  0.064 ms  110342/959769 (11%)  receiver

For Queue 1 with priority 2 and with BW of 100Mbps:

[  5]   0.00-60.00  sec   641 MBytes  89.6 Mbits/sec  0.266 ms  32740/489287 (6.7%)  receiver

For Queue0 with priority 0 (Best effort):

[  5]   0.00-60.00  sec  2.31 GBytes   331 Mbits/sec  0.006 ms  234299/1921970 (12%)  receiver

When TSN rules are implemented, the network can provide guaranteed bandwidth allocation and prioritization for different traffic streams. CBS TSN rules allow for the allocation of credits to different traffic streams, which can be used to transmit data over the network. This ensures that critical traffic, such as real-time audio or video streams, receive the necessary bandwidth and are not affected by other traffic streams. CBS TSN rules also prevent congestion and packet loss by limiting the amount of data that can be transmitted by each traffic stream. This results in a more predictable and reliable network environment, with improved quality of service for users.

4. TSN 802.1Qbv[edit source]

An example configuration with 4 schedule entries given below:

  • Uses 4 Queues (Q0-Q3). Each Q has a Gate associated in h/w. Maximum 4 Queues/Gates supported
  • 3 higher priority Gates open for 125usec (Q1, Q2 and Q3)
  • Q3 is the higest priority Queue and Q0 is the lowest priority
  • 1 lower priority (Q0) opens remaining gates for 250 usec
Alternate text
4 queues

Here are the steps to configure this schedule.

Warning white.png Warning
Be careful if you want to perform QBV test, you need to start gPTP synchronization before



Bring up end0 interface

ip link add link end0 name end0.200 type vlan id 200
ip link set end0.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
ifconfig end0.200 192.168.200.60 netmask 255.255.255.0

Setup EST schedule with 4 Gates (Q0-Q3). For description of Command parameters, see manual page for taprio. https://man7.org/linux/man-pages/man8/tc-taprio.8.html TC0 <-> Q0, TC1 <-> Q1, TC2 <-> Q2 and TC3 <-> Q3

 tc qdisc replace dev end0 parent root handle 100 taprio \
  num_tc 4 \
  map 0 0 1 1 2 2 3 3 3 3 3 3 3 3 3 3 \
  queues 1@0 1@1 1@2 1@3 \
  base-time 0000 \
  sched-entry S 8 125000 \
  sched-entry S 4 125000 \
  sched-entry S 2 125000 \
  sched-entry S 1 250000 \
  sched-entry S 0 375000 \
  flags 2

Where num_tc is same as number of queues = 4, maps 16 priorities to one of 4 TCs, queues specify the Queue associated with each TC, TC0 - One queue @0, TC1 - One queue @1, TC2 - One queue @2 and TC3 - One queue @3

 sched-entry S 8 125000
 S - SetGateStates operation
 8 - Bit mask showing bit 4 set (Q3/TC3)
 125000 - 125000 nsecs (125 usecs ) duration of Gate open
 The cycle-time is 1 msec (which is the sum of the intervals of individual schedule)

Enable classifier. Classifier is used to mark the packet based on packet meta data.

 tc qdisc add dev end0 clsact

Network core and Driver uses the skb priority to deliver frames to specific h/w queues. In the above case, priority 6 to 15 SKB (packet) goes to Q3 (4th entry in map in the tc qdisc command), priority 4 to 5 SKB goes to Q2 (3rd entry in map) priority 2 to 3 SKB goes to Q1 (2nd entry in map) and priority 0 to 1 SKB goes to Q0 (1st entry in map)

Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to end0 with IP address 192.168.200.30

iperf3 -s -p 5003&
iperf3 -s -p 5002&
iperf3 -s -p 5001&
iperf3 -s -p 5000&

At DUT, start transmission of stream using iperf3

//Stream with SKB priority 6  : TOS=0x14 == SO_PRIORITY=6
// https:man7.org/linux/man-pages/man8/tc-prio.8.html
iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4
iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
//Stream with SKB priority 2
iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort)
iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60  &

Result without tc rule apply :

Alternate text
schedule without QBV rule: it's anarchy

Result with tc rule apply:

Alternate text
schedule with QBV rule: each traffic has its time frame

5. References[edit source]