How to configure TSN switch

Applicable for

STM32MP25x lines

---

1. Article purpose

This article describes how to configure TSN capabilities on the Ethernet switch IP
Throughout this article, we'll be mentioning IEEE standards. To get more information about it, please refer to our TSN introduction.

This article provides different steps to set up the TSN standards on Ethernet switch:

• IEEE 802.1AS-Rev
• IEEE 802.1Qav
• IEEE 802.1Qbv

2. Prerequisites

This article necessarily needs the X-LINUX-TSNSWCH Expansion Package installed on top of OpenSTLinux distribution. In this article explanations, we assume that you have this software expansion package installed.

3. IEEE 802.1AS-Rev or gPTP synchronization

3.1. PTP stack

gPTP synchronization is provided by the proprietary PTP stack (by TTTech).

The PTP stack:

• Start gPTP protocol
• synchronize system time to PTP clock

3.1.1. run the PTP stack

Deptp is started by default.

3.1.2. Configure the PTP stack

Basic configuration of the PTP stack is done via configuration file, which has XML format. See /etc/deptp/ptp_config.xml for stack config

3.2. PTP Management tool

PTP Management tool, called deptp tool, is to access PTP runtime data from Linux command line.

Useful commands:

// Show offset from master
watch -n1 deptp_tool --get-current-dataset
 
// Show PTP state of sw0p2
deptp_tool -t1 --get-port-dataset

// Show PTP state of sw0p3
deptp_tool -t2 --get-port-dataset

Warning

Be careful if you are using PHY which managed “low power mode” (i.e EEE), most of the time this feature is not compatible with TSN. To be considered “asCapable” the maximum acceptable value for neighborPropDelayThresh is 800ns (default value), if you need to tune this value you can add in your Interface describe in XML ptp_config file: <delay_thresh>800</delay_thresh> and update value

3.3. How to trace the PTP stack

Deptp can be launched in verbose mode to have more details about synchronization

Stop deptp:

systemctl stop deptp

Edit /etc/deptp/ptp_config.xml to change debug level of trace (by default 0)

<General>
   <debug>0</debug>
</General>

Possible value:

   0 ///< No debug
   1 ///< Error debug only
   2 ///< INFO level debug (easy to see that synchronization is working)
   3 ///< VERBOSE level debug, more information than in INFO mode
   4 ///< ALL - print all available debug messages. Please note, this may affect device performance

Start deptp:

systemctl start deptp

Log will be available with:

journalctl -u deptp 

or in /var/log/messages

4. IEEE 802.1Qav

Maximum 4 queues supported by GMAC and maximum 8 queues supported by switch.

An example of configuration with 4 reserved bandwidth across switch given below:

• Uses 4 Queues (Q0-Q3). Each Q has a bandwidth reserved in h/w.
  • Queue 0 (Yellow stream) has No Bandwidth Restriction, general traffic (Prio 0)
  • Queue 1 (Blue stream) has 40% of Maximum bandwidth reserved (300Mbps) (Prio 2)
  • Queue 2 (Green stream) has 30% of Maximum bandwidth reserved (200Mbps) (Prio 4)
  • Queue 3 (Red stream) has 20% of Maximum bandwidth reserved (100Mbps) (Prio 6)

Queue 3 is the highest priority Queue and Q0 is the lowest priority

Streams across the switch

4.1. How to configure the setup

Here are the steps to configure this setup.

Warning

If you want to perform CBS test, gPTP synchronization is not needed

---

Bring up end1/sw0ep interface

ip link add link sw0ep name sw0ep.200 type vlan id 200
ip link set sw0ep.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
ifconfig sw0ep.200 192.168.200.60 netmask 255.255.255.0
bridge vlan add vid 200 dev sw0p1
bridge vlan add vid 200 dev sw0p2
bridge vlan add vid 200 dev sw0p3

CBS is used to enforce a Quality of Service by limiting the data rate of a traffic class, to separate packets into traffic classes the user may choose mqprio(8) https://man7.org/linux/man-pages/man8/tc-mqprio.8.html, and configure it like this:

---

Network core and Driver uses the skb priority to deliver frames to specific h/w queues. In the above case, priority 6 (packet) goes to Q3, priority 4 goes to Q2, priority 2 goes to Q1 and priority 0 SKB goes to Q0.

 ./set-cbs-idleslope.sh sw0p3 6 100000000
 + tsntool fqtss tsa 6 cbs sw0p3
 + tsntool fqtss slope 6 100000000 sw0p3
 + tsntool fqtss show sw0p3
Traffic Class        Algorithm            operIdleSlope
      0              Strict prio          n/a
      1              Strict prio          n/a
      2              Strict prio          n/a
      3              Strict prio          n/a
      4              Strict prio          n/a
      5              Strict prio          n/a
      6              CBS                  100000000 bps
      7              Strict prio          n/a

 ./set-cbs-idleslope.sh sw0p3 4 200000000
 + tsntool fqtss tsa 4 cbs sw0p3
 + tsntool fqtss slope 4 200000000 sw0p3
 + tsntool fqtss show sw0p3
Traffic Class        Algorithm            operIdleSlope
      0              Strict prio          n/a
      1              Strict prio          n/a
      2              Strict prio          n/a
      3              Strict prio          n/a
      4              CBS                  200000000 bps
      5              Strict prio          n/a
      6              CBS                  100000000 bps
      7              Strict prio          n/a

 ./set-cbs-idleslope.sh sw0p3 2 300000000
 + tsntool fqtss tsa 2 cbs sw0p3
 + tsntool fqtss slope 2 300000000 sw0p3
 + tsntool fqtss show sw0p3
Traffic Class        Algorithm            operIdleSlope
      0              Strict prio          n/a
      1              Strict prio          n/a
      2              CBS                  300000000 bps
      3              Strict prio          n/a
      4              CBS                  200000000 bps
      5              Strict prio          n/a
      6              CBS                  100000000 bps
      7              Strict prio          n/a

With set-cbs-idleslope.sh :

#!/bin/bash

#implemented for tsntool 2.4.0


if [[ "$#" -ne 3 ]] ; then
    echo "Usage: $0 port traffic-class idleSlope"
    exit 1
fi
port=$1
trafficClass=$2
idleSlope=$3
set -x
tsntool fqtss tsa $trafficClass cbs $port
tsntool fqtss slope $trafficClass $idleSlope $port
tsntool fqtss show $port

---

Run 4 iperf sessions to generate several streams with different priorities. Remote PC or other device connected to sw0p3 (ETH1) with IP address 192.168.200.30, and other device/PC connected to sw0p2 (ETH3) to launch 2 x iperf3 client to generate streams.

Run 4 iperf sessions to generate several streams with different priorities

Remote PC or other device connected to sw0p3 with IP address 192.168.200.30

iperf3 -s -p 5003&
iperf3 -s -p 5002&
iperf3 -s -p 5001&
iperf3 -s -p 5000&

Remote PC or other device connected to sw0p2 with IP address 192.168.200.40

//Stream with SKB priority 6  : TOS=0x14 == SO_PRIORITY=6
// https:man7.org/linux/man-pages/man8/tc-prio.8.html
iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &

//Stream with SKB priority 4
iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &

At DUT, start transmission of stream using iperf3

//Stream with SKB priority 2
iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &

//Stream with SKB priority 0 (Best effort)
iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60  &

---

Result without set-cbs-idleslope.sh rule apply :
For Traffic with priority 6:

[  5]   0.00-60.85  sec  2.65 GBytes   374 Mbits/sec  0.196 ms  437454/2371320 (18%)  receiver

For Traffic with priority 4:

[  5]   0.00-60.21  sec  2.61 GBytes   372 Mbits/sec  0.046 ms  421125/2321380 (18%)  receiver

For Traffic with priority 2:

[  5]   0.00-60.00  sec   518 MBytes  72.4 Mbits/sec  0.047 ms  1826344/2195179 (83%)  receiver

For Traffic with priority 0 (Best effort):

[  5]   0.00-60.00  sec   831 MBytes   116 Mbits/sec  0.013 ms  2355974/2948076 (80%)  receiver

Without CBS TSN rules in place, multiple traffic streams may contend for network resources, causing congestion and possible packet loss. This can lead to poor network performance and lower quality of service for users. Furthermore, without CBS TSN rules, there no prioritization of important traffic, such as real-time audio or video streams, which can cause delays and interruptions in these applications. In summary, the absence of CBS TSN rules can create an unstable and undependable network environment.

---

Result with set-cbs-idleslope.sh rule apply:
For Queue 3 with priority 6 and with BW of 100Mbps:

[  5]   0.00-56.00  sec   647 MBytes  97.0 Mbits/sec  0.108 ms  3766160/4227385 (89%)  receiver

For Queue 2 with priority 4 and with BW of 200Mbps:

[  5]   0.00-60.00  sec  1.35 GBytes   193 Mbits/sec  0.066 ms  3438066/4422599 (78%)  receiver

For Queue 1 with priority 2 and with BW of 300Mbps:

[  5]   0.00-60.00  sec  2.02 GBytes   290 Mbits/sec  0.041 ms  2994134/4470767 (67%)  receiver

For Queue0 with priority 0 (Best effort):

[  5]   0.00-60.00  sec  1.11 GBytes   131 Mbits/sec  0.006 ms  134299/1921970 (12%)  receiver

When TSN rules are implemented, the network can provide guaranteed bandwidth allocation and prioritization for different traffic streams. CBS TSN rules allow for the allocation of credits to different traffic streams, which can be used to transmit data over the network. This ensures that critical traffic, such as real-time audio or video streams, receive the necessary bandwidth and are not affected by other traffic streams. CBS TSN rules also prevent congestion and packet loss by limiting the amount of data that can be transmitted by each traffic stream. This results in a more predictable and reliable network environment, with improved quality of service for users.

5. IEEE 802.1Qbv

An example configuration with 4 streams across Switch given below: (same topology as described in previous chapter)

• Uses 4 Queues (Q0-Q3). Each Q has a schedule reserved in h/w.
  • Queue 0 (Yellow stream) considered best effort which have gate open for 250usec (Prio 0)
  • Queue 1 (Blue stream) which have gate open for 125usec (Prio 2)
  • Queue 2 (Green stream) which have gate open for 125usec (Prio 4)
  • Queue 3 (Red stream) which have gate open for 125usec (Prio 6)

Q3 is the highest priority Queue and Q0 is the lowest priority

Here are the steps to configure this schedule.

Warning

Be careful if you want to perform QBV test, you need to start gPTP synchronization before, for switch it is deptp and it is started by default

---

Bring up end1/sw0ep interface

ip link add link sw0ep name sw0ep.200 type vlan id 200
ip link set sw0ep.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7
ifconfig sw0ep.200 192.168.200.60 netmask 255.255.255.0
bridge vlan add vid 200 dev sw0p1
bridge vlan add vid 200 dev sw0p2
bridge vlan add vid 200 dev sw0p3

Example Qbv configuration: sgs is used for the configuration of Qbv. Save the following configuration to qbv.cfg.

with qbv.cfg:

sgs 125000 0x40
sgs 125000 0x10
sgs 125000 0x4
sgs 250000 0x1
sgs 250000 0
sgs 125000 0 

This configuration sets

• set gate for traffic classes 6 for 125000 nanoseconds
• set gate for traffic classes 4 for 125000 nanoseconds
• set gate for traffic classes 2 for 125000 nanoseconds
• set gate for traffic classes 0 for 250000 nanoseconds
• all gates close for 375000 nanoseconds

Once the schedule is created, it should be written to the AdminControlList , before loading it to the OperControlList. For writing it to the AdminControlList, call the tool tsntool as follows:

 tsntool st wrcl sw0p3 qbv.cfg

To check that the configuration was written to the AdminControlList, the AdminControlList can be read as follows:
tsntool st rdacl <interface>

Example of printing to the terminal:

tsntool st rdacl sw0p3
sgs  125000  0x40
sgs  125000  0x10
sgs  125000  0x04
sgs  250000  0x01
sgs  250000  0x00
sgs  125000  0x00

To load this schedule to the OperControlList so that it gets taken into use, call this command:
tsntool st configure <basetime> <cycletime> <cycletime-extension> <interface>
The basetime is formatted as <seconds>.<nanoseconds>, as the current time of the switch. Note: The current time of the switch can be read as follows:
cat /sys/class/net/sw0p3/ieee8021ST/CurrentTime
The parameter cycletime is formatted as <numerator>/<denominator> in seconds.
The cycletime-extension is formatted as plain nanoseconds. Note: The cycletime-extension parameter has to be specified for legacy reasons. It is ignored by the driver.
An alternative is to use a relative time to apply the configuration. It’s in the same format, but with a ’+’ or a ’-’ just before.

 tsntool st configure 0.0 1/1000 0 sw0p3

A cycle time of 1/1000 seconds = 1 millisecond is a lot longer than the time needed to complete one schedule run-through. After the schedule has finished within a cycle, the gates remain in the last configured state until the end of the cycle.
After calling the above commands, all the necessary parameters are set to have a basic Qbv configuration up and running. It can be confirmed by running a command:

tsntool st show sw0p3
GatesEnabled:            YES
ConfigPending:           NO
ConfigChangeError:       1

AdminCycleTime:          1/1000 sec
AdminCycleTimeExtension: 0 nsec
AdminControlListLength:  6 entries
OperCycleTime:           1/1000 sec
OperCycleTimeExtension:  0 nsec
OperControlListLength:   6 entries

CurrentTime (TAI):       2023-03-03 10:41:46.737801550 (1677840106.737801550)
AdminBaseTime (TAI):     1970-01-01 00:00:00.000000000
OperBaseTime (TAI):      1970-01-01 00:00:00.000000000
ConfigChangeTime (TAI):  2023-03-03 10:00:04.278000000 

SupportedCycleMax:       335211525/1000000000 sec
SupportedIntervalMax:    327675 nsec
SupportedListMax:        1023 entries
TickGranularity:         100000 1/10 nsec

portNumTrafficClasses:   8

queueMaxSDUTable:
TrafficClass   queueMaxSDU   TransmissionOverruns
0                   1506   6961
1                   1506   6961
2                   1506   6961
3                   1506   6961
4                   1506   6961
5                   1506   6961
6                   1506   6961
7                   1506   6961

---

Run 4 iperf sessions to generate several streams with different priorities. Remote PC or other device connected to sw0p3 (ETH1) with IP address 192.168.200.30, and other device/PC connected to sw0p2 (ETH3) to launch 2 x iperf3 client to generate streams.

Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to sw0p3 with IP address 192.168.200.30

iperf3 -s -p 5003&
iperf3 -s -p 5002&
iperf3 -s -p 5001&
iperf3 -s -p 5000&

Remote PC or other device connected to sw0p2 with IP address 192.168.200.40

//Stream with SKB priority 6  : TOS=0x14 == SO_PRIORITY=6
// https:man7.org/linux/man-pages/man8/tc-prio.8.html
iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &

//Stream with SKB priority 4
iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &

At DUT, start transmission of stream using iperf3

//Stream with SKB priority 2
iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &

//Stream with SKB priority 0 (Best effort)
iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60  &

---

Result without tsntool rule apply :

schedule without QBV rule: it's anarchy

---

Result with tsntool rule apply:

schedule with QBV rule: each traffic has its time frame