1. Article purpose[edit | edit source]
This article describes how to configure TSN capabilities on the Ethernet switch internal peripheral.
Throughout this article, IEEE standards is mentioned. To get more information about it, refer to TSN standards introduction.
This article provides different steps to set up the TSN standards on Ethernet switch:
- PTP (Precision Time Protocol) [IEEE-1588, IEEE-802.1AS]
- FQTSS / CBS (Credit Based Shaper) [IEEE-802.1Qav]
- EST (Enhancements for Scheduled Traffic) [IEEE-802.1Qbv]
If you want to use the TSN switch as a simple Ethernet switch : How to configure Ethernet Switch
2. Prerequisites[edit | edit source]
This article necessarily needs the X-LINUX-TSNSWCH expansion package installed on top of OpenSTLinux distribution. In this article explanations, we assume that you have this software expansion package installed.
3. IEEE 802.1AS-Rev or gPTP synchronization[edit | edit source]
On Ethernet Switch the protocol stack used to perform synchronisation is called dePTP stack (proprietary of TTTech).
The dePTP stack:
- Starts gPTP protocol,
- synchronizes system time to PTP clock.
3.1. Run the PTP stack[edit | edit source]
dePTP stackis started by default.
3.2. Configure the PTP stack[edit | edit source]
Basic configuration of the PTP stack is done via configuration file (ptp_config.xml), which has XML format.
cat /etc/deptp/ptp_config.xml
3.1. PTP Management tool[edit | edit source]
PTP Management tool, called deptp tool, is to access PTP runtime data from Linux command line.
Useful commands:
// Show offset from master watch -n1 deptp_tool --get-current-dataset // Show PTP state of sw0p2 deptp_tool -t1 --get-port-dataset // Show PTP state of sw0p3 deptp_tool -t2 --get-port-dataset
3.2. How to trace the PTP stack[edit | edit source]
Deptp can be launched in verbose mode to have more details about synchronization.
Stop deptp:
systemctl stop deptp
Edit /etc/deptp/ptp_config.xml to change debug level of trace (by default 0)
<General> <debug>0</debug> </General>
Possible value:
0 ///< No debug 1 ///< Error debug only 2 ///< INFO level debug (easy to see that synchronization is working) 3 ///< VERBOSE level debug, more information than in INFO mode 4 ///< ALL - print all available debug messages. Please note, this may affect device performance
Start deptp:
systemctl start deptp
Log will be available with:
journalctl -u deptp
or in /var/log/messages
4. CBS (Credit Based Shaper) [IEEE-802.1Qav][edit | edit source]
TSN 802.1Qav is an IEEE specification that allows stream reservation for networking traffic queues. It adds enhancements to traffic shaping with the introduction of stream reservation. This article demonstrates how TSN 802.1Qav enhances traffic shaping, using Ethernet Switch.
This is an example of configuration with 4 reserved bandwidth across switch given below:
- Uses 4 Queues (Q0-Q3). Each Q has a bandwidth reserved in h/w.
- Queue 0 (Yellow stream) has No Bandwidth Restriction, general traffic (Prio 0)
- Queue 1 (Blue stream) has 40% of Maximum bandwidth reserved (300Mbps) (Prio 2)
- Queue 2 (Green stream) has 30% of Maximum bandwidth reserved (200Mbps) (Prio 4)
- Queue 3 (Red stream) has 20% of Maximum bandwidth reserved (100Mbps) (Prio 6)
Queue 3 is the highest priority Queue and Q0 is the lowest priority
4.1. How to setup this use case[edit | edit source]
Here are the steps to configure this use case.
Bring up end1/sw0ep interface
ip link add link sw0ep name sw0ep.200 type vlan id 200 ip link set sw0ep.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7 ifconfig sw0ep.200 192.168.200.60 netmask 255.255.255.0 bridge vlan add vid 200 dev sw0p1 bridge vlan add vid 200 dev sw0p2 bridge vlan add vid 200 dev sw0p3
CBS is used to enforce a Quality of Service by limiting the data rate of a traffic class, to separate packets into traffic classes the user may choose mqprio(8) https://man7.org/linux/man-pages/man8/tc-mqprio.8.html, and configure it like this:
Network core and Driver uses the skb priority to deliver frames to specific h/w queues. In the above case, priority 6 (packet) goes to Q3, priority 4 goes to Q2, priority 2 goes to Q1 and priority 0 SKB goes to Q0.
./set-cbs-idleslope.sh sw0p3 6 100000000
+ tsntool fqtss tsa 6 cbs sw0p3
+ tsntool fqtss slope 6 100000000 sw0p3
+ tsntool fqtss show sw0p3
Traffic Class Algorithm operIdleSlope
0 Strict prio n/a
1 Strict prio n/a
2 Strict prio n/a
3 Strict prio n/a
4 Strict prio n/a
5 Strict prio n/a
6 CBS 100000000 bps
7 Strict prio n/a
./set-cbs-idleslope.sh sw0p3 4 200000000
+ tsntool fqtss tsa 4 cbs sw0p3
+ tsntool fqtss slope 4 200000000 sw0p3
+ tsntool fqtss show sw0p3
Traffic Class Algorithm operIdleSlope
0 Strict prio n/a
1 Strict prio n/a
2 Strict prio n/a
3 Strict prio n/a
4 CBS 200000000 bps
5 Strict prio n/a
6 CBS 100000000 bps
7 Strict prio n/a
./set-cbs-idleslope.sh sw0p3 2 300000000
+ tsntool fqtss tsa 2 cbs sw0p3
+ tsntool fqtss slope 2 300000000 sw0p3
+ tsntool fqtss show sw0p3
Traffic Class Algorithm operIdleSlope
0 Strict prio n/a
1 Strict prio n/a
2 CBS 300000000 bps
3 Strict prio n/a
4 CBS 200000000 bps
5 Strict prio n/a
6 CBS 100000000 bps
7 Strict prio n/a
With set-cbs-idleslope.sh :
#!/bin/bash
#implemented for tsntool 2.4.0
if [[ "$#" -ne 3 ]] ; then
echo "Usage: $0 port traffic-class idleSlope"
exit 1
fi
port=$1
trafficClass=$2
idleSlope=$3
set -x
tsntool fqtss tsa $trafficClass cbs $port
tsntool fqtss slope $trafficClass $idleSlope $port
tsntool fqtss show $port
Run 4 iperf sessions to generate several streams with different priorities. Remote PC or other device connected to sw0p3 (ETH1) with for example IP address 192.168.200.30, and other device/PC connected to sw0p2 (ETH3) to launch 2 x iperf3 client to generate streams.
Run 4 iperf sessions to generate several streams with different priorities
Remote PC or other device connected to sw0p3 with IP address 192.168.200.30
iperf3 -s -p 5003& iperf3 -s -p 5002& iperf3 -s -p 5001& iperf3 -s -p 5000&
Remote PC or other device connected to sw0p2 with for example IP address 192.168.200.40
//Stream with SKB priority 6 : TOS=0x14 == SO_PRIORITY=6 // https:man7.org/linux/man-pages/man8/tc-prio.8.html iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4 iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
At device under test (DUT), start transmission of stream using iperf3
//Stream with SKB priority 2 iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort) iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60 &
Result without set-cbs-idleslope.sh rule apply :
For Traffic with priority 6:
[ 5] 0.00-60.85 sec 2.65 GBytes 374 Mbits/sec 0.196 ms 437454/2371320 (18%) receiver
For Traffic with priority 4:
[ 5] 0.00-60.21 sec 2.61 GBytes 372 Mbits/sec 0.046 ms 421125/2321380 (18%) receiver
For Traffic with priority 2:
[ 5] 0.00-60.00 sec 518 MBytes 72.4 Mbits/sec 0.047 ms 1826344/2195179 (83%) receiver
For Traffic with priority 0 (Best effort):
[ 5] 0.00-60.00 sec 831 MBytes 116 Mbits/sec 0.013 ms 2355974/2948076 (80%) receiver
Without CBS TSN rules in place, multiple traffic streams may contend for network resources, causing congestion and possible packet loss. This can lead to poor network performance and lower quality of service for users. Furthermore, without CBS TSN rules, there no prioritization of important traffic, such as real-time audio or video streams, which can cause delays and interruptions in these applications. In summary, the absence of CBS TSN rules can create an unstable and undependable network environment.
Result with set-cbs-idleslope.sh rule apply:
For Queue 3 with priority 6 and with BW of 100Mbps:
[ 5] 0.00-56.00 sec 647 MBytes 97.0 Mbits/sec 0.108 ms 3766160/4227385 (89%) receiver
For Queue 2 with priority 4 and with BW of 200Mbps:
[ 5] 0.00-60.00 sec 1.35 GBytes 193 Mbits/sec 0.066 ms 3438066/4422599 (78%) receiver
For Queue 1 with priority 2 and with BW of 300Mbps:
[ 5] 0.00-60.00 sec 2.02 GBytes 290 Mbits/sec 0.041 ms 2994134/4470767 (67%) receiver
For Queue0 with priority 0 (Best effort):
[ 5] 0.00-60.00 sec 1.11 GBytes 131 Mbits/sec 0.006 ms 134299/1921970 (12%) receiver
When TSN rules are implemented, the network can provide guaranteed bandwidth allocation and prioritization for different traffic streams. CBS TSN rules allow for the allocation of credits to different traffic streams, which can be used to transmit data over the network. This ensures that critical traffic, such as real-time audio or video streams, receive the necessary bandwidth and are not affected by other traffic streams. CBS TSN rules also prevent congestion and packet loss by limiting the amount of data that can be transmitted by each traffic stream. This results in a more predictable and reliable network environment, with improved quality of service for users.
5. EST (Enhancements for Scheduled Traffic) [IEEE-802.1Qbv][edit | edit source]
TSN 802.1Qbv is an IEEE specification that allows to define transmission time slots for networking traffic queues. It adds enhancements to traffic scheduling with the introduction of time-aware shaper. This article demonstrates how TSN 802.1Qbv enhances traffic scheduling, using Ethernet Switch.
An example configuration with 4 streams across Switch given below: (same topology as described in previous chapter)
- Uses 4 Queues (Q0-Q3). Each Q has a schedule reserved in h/w.
- Queue 0 (Yellow stream) considered best effort which have gate open for 250usec (Prio 0)
- Queue 1 (Blue stream) which have gate open for 125usec (Prio 2)
- Queue 2 (Green stream) which have gate open for 125usec (Prio 4)
- Queue 3 (Red stream) which have gate open for 125usec (Prio 6)
Q3 is the highest priority Queue and Q0 is the lowest priority
Here are the steps to configure this schedule.
Bring up end1/sw0ep interface
ip link add link sw0ep name sw0ep.200 type vlan id 200 ip link set sw0ep.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7 ifconfig sw0ep.200 192.168.200.60 netmask 255.255.255.0 bridge vlan add vid 200 dev sw0p1 bridge vlan add vid 200 dev sw0p2 bridge vlan add vid 200 dev sw0p3
Example Qbv configuration: sgs is used for the configuration of Qbv. Save the following configuration to qbv.cfg.
with qbv.cfg:
sgs 125000 0x40 sgs 125000 0x10 sgs 125000 0x4 sgs 250000 0x1 sgs 250000 0 sgs 125000 0
This configuration sets
- set gate for traffic classes 6 for 125000 nanoseconds
- set gate for traffic classes 4 for 125000 nanoseconds
- set gate for traffic classes 2 for 125000 nanoseconds
- set gate for traffic classes 0 for 250000 nanoseconds
- all gates close for 375000 nanoseconds
Once the schedule is created, it should be written to the AdminControlList , before loading it to the OperControlList. For writing it to the AdminControlList, call the tool tsntool as follows:
tsntool st wrcl sw0p3 qbv.cfg
To check that the configuration was written to the AdminControlList, the AdminControlList can be read as follows:
tsntool st rdacl <interface>
Example of printing to the terminal:
tsntool st rdacl sw0p3 sgs 125000 0x40 sgs 125000 0x10 sgs 125000 0x04 sgs 250000 0x01 sgs 250000 0x00 sgs 125000 0x00
To load this schedule to the OperControlList so that it gets taken into use, call this command:
tsntool st configure <basetime> <cycletime> <cycletime-extension> <interface>
The basetime is formatted as <seconds>.<nanoseconds>, as the current time of the switch. Note: The current
time of the switch can be read as follows:
cat /sys/class/net/sw0p3/ieee8021ST/CurrentTime
The parameter cycletime is formatted as <numerator>/<denominator> in seconds.
The cycletime-extension is formatted as plain nanoseconds. Note: The cycletime-extension parameter has to be
specified for legacy reasons. It is ignored by the driver.
An alternative is to use a relative time to apply the configuration. It’s in the same format, but with a ’+’ or a
’-’ just before.
tsntool st configure 0.0 1/1000 0 sw0p3
A cycle time of 1/1000 seconds = 1 millisecond is a lot longer than the time needed to complete one schedule
run-through. After the schedule has finished within a cycle, the gates remain in the last configured state until
the end of the cycle.
After calling the above commands, all the necessary parameters are set to have a basic Qbv configuration up
and running. It can be confirmed by running a command:
tsntool st show sw0p3 GatesEnabled: YES ConfigPending: NO ConfigChangeError: 1 AdminCycleTime: 1/1000 sec AdminCycleTimeExtension: 0 nsec AdminControlListLength: 6 entries OperCycleTime: 1/1000 sec OperCycleTimeExtension: 0 nsec OperControlListLength: 6 entries CurrentTime (TAI): 2023-03-03 10:41:46.737801550 (1677840106.737801550) AdminBaseTime (TAI): 1970-01-01 00:00:00.000000000 OperBaseTime (TAI): 1970-01-01 00:00:00.000000000 ConfigChangeTime (TAI): 2023-03-03 10:00:04.278000000 SupportedCycleMax: 335211525/1000000000 sec SupportedIntervalMax: 327675 nsec SupportedListMax: 1023 entries TickGranularity: 100000 1/10 nsec portNumTrafficClasses: 8 queueMaxSDUTable: TrafficClass queueMaxSDU TransmissionOverruns 0 1506 6961 1 1506 6961 2 1506 6961 3 1506 6961 4 1506 6961 5 1506 6961 6 1506 6961 7 1506 6961
Run 4 iperf sessions to generate several streams with different priorities. Remote PC or other device connected to sw0p3 (ETH1) with IP address 192.168.200.30, and other device/PC connected to sw0p2 (ETH3) to launch 2 x iperf3 client to generate streams.
Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to sw0p3 with IP address 192.168.200.30
iperf3 -s -p 5003& iperf3 -s -p 5002& iperf3 -s -p 5001& iperf3 -s -p 5000&
Remote PC or other device connected to sw0p2 with IP address 192.168.200.40
//Stream with SKB priority 6 : TOS=0x14 == SO_PRIORITY=6 // https:man7.org/linux/man-pages/man8/tc-prio.8.html iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4 iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
At DUT, start transmission of stream using iperf3
//Stream with SKB priority 2 iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort) iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60 &
Result without tsntool rule apply :
Result with tsntool rule apply:
6. References[edit | edit source]