Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note

It is important to note that though its name, P4Emu/pcap is not emulating V1Model. P4Emu is emulating router.p4 packet processing logic and uses a packet forwarding library to effectively transmit packets at specific ingress port to the right egress port defined by freeRouter control plane message. 


Diagram

Image RemovedImage Added

[ #005 ] - Cookbook

...

Expand
titleInstall FreeRouter as per #001 article: "Create freeRouter environment"


Code Block
languagebash
themeMidnight
titleflush enp0s3 so that it can be controlled by dpdk
ip addr flush enp0s3


Info
titleSource Wikipedia:
You can add a second Host-only interface  (enp0s8) in VirtualBox in order to connect the ubuntu focal VM guest as you might lose connection when you flushed enp0s3.


Code Block
languagebash
themeMidnight
mkdir -p ~/freeRouter/bin ~/freeRouter/lib ~/freeRouter/etc ~/freeRouter/log
cd ~/freeRouter/lib
wget http://freerouter.nop.hu/rtr.jar


Code Block
languagebash
themeMidnight
titleUpdate & Upgrade system
tree freeRouter
freeRouter
├── bin   # binary files      
├── etc   # configuration files      
├── lib   # library files      
└── log   # log files      


...

Expand
titleCreate configuration files for router: freerouter

FreeRouter uses 2 configuration files in order to run, let's write these configuration files for R1 in ~/freeRouter/etc

Code Block
languagebash
themeMidnight
titlefreeRouter hardware configuration file: p4-freerouter-hw.txt
int eth0 eth 0000.1111.00fb 127.0.0.1 22710 127.0.0.1 22709
tcp2vrf 2323 v1 23
tcp2vrf 9080 v1 9080


Code Block
languagebash
themeMidnight
titlefreeRouter software configuration file: p4-freerouter-sw.txtconfiguration file: p4-freerouter-sw.txt
hostname pcap-freerouter
buggyhostname p4-freerouter
buggy
!
!
prefix-list p4
 sequence 10 permit 0.0.0.0/0 ge 0 le 0
 exit
!
prefix-list p6
 sequence 10 permit ::/0 ge 0 le 0
 exit
!
vrf definition v1
 exit
!
interface ethernet0
 description freerouter@P4_CPU_PORT[veth251]
 no shutdown
 no log-link-change
 exit
!
interface sdn1
 description freerouter@sdn1[enp0s3]
 mtu 9000
 macaddr 0072.3e18.1b6f
 vrf forwarding v1
 ipv4 address 192.168.0.131 255.255.255.0
 ipv6 address 2a01:e0a:159:2850::666 ffff:ffff:ffff:ffff::
 ipv6 enable
 no shutdown
 no log-link-change
 exit
!
!
!
!
!
!
!
!
!
!
!
!
!
!
server telnet tel
 security protocol telnet
 no exec authorization
 no login authentication
 vrf v1
 exit
!
server p4lang p4
 export-vrf v1 1
 export-port sdn1 1 0
 interconnect ethernet0
 vrf v1
 exit
!
end


...

Expand
titleLaunch freeRouter control plane alongside P4Emu/pcap dataplane


Code Block
languagebash
themeMidnight
titleSetup P4Emu dataplane communication channel via veth pair and interface adjustment (disable IPv6 at VM guest level, MTU 10240, disable TCP offload etc.)
echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6
echo 1 > /proc/sys/net/ipv6/conf/default/disable_ipv6

ip link add veth251 type veth peer name veth250
ip link set veth250  up 
ip link set veth251  up 

ifconfig enp0s3 promisc
ifconfig veth250 promisc
ifconfig veth251 promisc

ip link set dev veth250 up mtu 10240
ip link set dev veth251 up mtu 10240
ip link set dev enp0s3 up mtu 10240
export TOE_OPTIONS="rx tx sg tso ufo gso gro lro rxvlan txvlan rxhash"

for TOE_OPTION in $TOE_OPTIONS; do
    /sbin/ethtool --offload veth250 "$TOE_OPTION" off &> /dev/null
    /sbin/ethtool --offload veth251 "$TOE_OPTION" off &> /dev/null
    /sbin/ethtool --offload enp0s3 "$TOE_OPTION" off &> /dev/null
done


Code Block
languagebash
themeMidnight
titlefreeRouter launch with supplied p4-freerouter-hw.txt and p4-freerouter-sw.txt with a console prompt
java -jar lib/rtr.jar routersc etc/p4-freerouter-hw.txt etc/p4-freerouter-sw.txt
info cfg.cfgInit.doInit:cfgInit.java:556 booting
info cfg.cfgInit.doInit:cfgInit.java:680 initializing hardware
info cfg.cfgInit.doInit:cfgInit.java:687 applying defaults
info cfg.cfgInit.doInit:cfgInit.java:695 applying configuration
info cfg.cfgInit.doInit:cfgInit.java:721 done
welcome
line ready
pcap-freerouter#                   


Code Block
languagebash
themeMidnight
titlelaunch freeRouter pcapInt in order to stitch control plane and P4Emu/pcap dataplane communication
cd ~/freeRouter/bin
./pcapInt.bin veth251 22709 127.0.0.1 22710 127.0.0.1
binded to local port 127.0.0.1 22709.
will send to 127.0.0.1 22710.
pcap version: libpcap version 1.8.1
opening interface veth251 with pcap1.x api
serving others
> 


Code Block
languagebash
themeMidnight
titleLaunch P4Emu/pcap software dataplane
sudo ./p4emu.bin  127.0.0.1 9080 0 veth250 enp0s3
cpu port is #0 of 2...
pcap version: libpcap version 1.8.1
connecting 127.0.0.1 9080.
opening interface veth250.
opening interface enp0s3.
rx: 'myaddr4' 'add' '224.0.0.0' '4' '0' '1' '' 
rx: 'myaddr4' 'add' '255.255.255.255' '32' '0' '1' '' 
rx: 'myaddr6' 'add' 'ff00::' '8' '0' '1' '' 
rx: 'myaddr4' 'add' '192.168.0.0' '24' '-1' '1' '' 
rx: 'myaddr4' 'add' '192.168.0.131' '32' '-1' '1' '' 
rx: 'myaddr6' 'add' '2a01:e0a:159:2850::' '64' '-1' '1' '' 
rx: 'myaddr6' 'add' '2a01:e0a:159:2850::666' '128' '-1' '1' '' 
rx: 'myaddr6' 'add' 'fe80::' '64' '-1' '1' '' 
rx: 'mylabel4' 'add' '615589' '1' '' 
rx: 'mylabel6' 'add' '1036348' '1' '' 
rx: 'state' '1' '1' '0' '' 
rx: 'mtu' '1' '9000' '' 
rx: 'portvrf' 'add' '1' '1' '' 
rx: 'keepalive' '' 
rx: 'keepalive' '' 
rx: 'neigh6' 'add' '11120' 'fe80::224:d4ff:fea0:cd3' '00:24:d4:a0:0c:d3' '1' '00:72:3e:18:1b:6f' '1' '' 
rx: 'keepalive' '' 
rx: 'keepalive' '' 
rx: 'keepalive' '' 
rx: 'neigh4' 'add' '29738' '192.168.0.254' '00:24:d4:a0:0c:d3' '1' '00:72:3e:18:1b:6f' '1' '' 
rx: 'keepalive' '' 
rx: 'neigh4' 'add' '40470' '192.168.0.62' 'e0:3f:49:6d:18:99' '1' '00:72:3e:18:1b:6f' '1' '' 
rx: 'keepalive' '' 
rx: 'keepalive' '' 
rx: 'keepalive' '' 
rx: 'keepalive' '' 
rx: 'keepalive' '' 
rx: 'neigh6' 'add' '45820' '2a01:e0a:159:2850:e23f:49ff:fe6d:1899' 'e0:3f:49:6d:18:99' '1' '00:72:3e:18:1b:6f' '1' '' 
rx: 'keepalive' '' 
rx: 'neigh6' 'add' '49055' 'fe80::e23f:49ff:fe6d:1899' 'e0:3f:49:6d:18:99' '1' '00:72:3e:18:1b:6f' '1' '' 
rx: 'neigh6' 'add' '33334' '2a01:e0a:159:2850::
...


...

Expand
titleCheck telnet access for freerouter@2323


Code Block
languagebash
themeMidnight
titleFreeRouter telnet access from Virtualbox VM guest via port 2323
telnet localhost 2323
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
welcome
line ready
pcap-freerouter#



Expand
titleCheck running-config


Code Block
languagebash
themeMidnight
titlefreerouter running configuration
p4pcap-freerouter#term len 0                                                       
p4pcap-freerouter#sh run                                                           
hostname p4pcap-freerouter
buggy
!
vrf definition v1
 exit
!
interface ethernet0
 description freerouter@P4_CPU_PORT[veth251]
 no shutdown
 no log-link-change
 exit
!
interface sdn1
 description freerouter@sdn1[enp0s3]
 mtu 9000
 macaddr 0072.3e18.1b6f
 vrf forwarding v1
 ipv4 address 192.168.0.131 255.255.255.0
 ipv6 address 2a01:e0a:159:2850::666 ffff:ffff:ffff:ffff::
 ipv6 enable
 no shutdown
 no log-link-change
 exit
!
!
!
!
!
!
!
!
!
!
!
!
!
!
server telnet tel
 security protocol telnet
 no exec authorization
 no login authentication
 vrf v1
 exit
!
server p4lang p4
 export-vrf v1 1
 export-port sdn1 1 0
 interconnect ethernet0
 vrf v1
 exit
!
end


Code Block
languagebash
themeMidnight
titleCheck control plane is communicating with bf_switchd p4 dataplane
p4pcap-freerouter#show interfaces summary                                          
interface  state  tx    rx      drop
ethernet0  up     8739  404545  0
sdn1       up     8535  400013  0


...

Expand
titleConnectivity IPv4/IPv6 check for freeRouter


Code Block
languagebash
themeMidnight
titlePing IPv4 from freerouter -> LAN router gateway
p4pcap-freerouter#ping 192.168.0.254 /vrf v1                                       
pinging 192.168.0.254, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
.!!!!
result=80%, recv/sent/lost=4/5/1, rtt min/avg/max/total=1/1/2/1011


Code Block
languagebash
themeMidnight
titlePing IPv4 from freerouter -> LAN server
p4pcap-freerouter#ping 192.168.0.62 /vrf v1                                        
pinging 192.168.0.62, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
.!!!!
result=80%, recv/sent/lost=4/5/1, rtt min/avg/max/total=1/1/2/1005 


Note

Please observe the 1st ICMP packet loss that triggered \ ARP learning for respectively 192.168.0.254 and 192.168.0.62.


Code Block
languagebash
themeMidnight
titleIPv4 arp check
p4pcap-freerouter#sh ipv4 arp sdn1                                                 
mac             address        time      static
e03f.496d.1899  192.168.0.62   00:00:57  false    <----- Host server
0024.d4a0.0cd3  192.168.0.254  00:00:57  false    <----- LAN gateway


Code Block
languagebash
themeMidnight
titlePing IPv6 from freerouter -> LAN router
p4pcap-freerouter#ping 2a01:e0a:159:2850::1  /vrf v1                               
pinging 2a01:e0a:159:2850::1, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
.!!!!
result=80%, recv/sent/lost=4/5/1, rtt min/avg/max/total=0/1/2/1004


Code Block
languagebash
themeMidnight
titlePing IPv6 from freerouter -> Host server and SSH connection test
p4pcap-freerouter#ping 2a01:e0a:159:2850:e23f:49ff:fe6d:1899  /vrf v1              
pinging 2a01:e0a:159:2850:e23f:49ff:fe6d:1899, src=null, cnt=5, len=64, tim=1000, ttl=255, tos=0, sweep=false
.!!!!
result=80%, recv/sent/lost=4/5/1, rtt min/avg/max/total=1/1/1/1006


Note

Please observe the 1st ICMP packet loss that triggered IPv6 neighbor discovery for respectively 2a01:e0a:159:2850::1 and 2a01:e0a:159:2850:e23f:49ff:fe6d:1899


Code Block
languagebash
themeMidnight
titleIPv6 neighbor discovery check
p4pcap-freerouter#show ipv6 neighbors sdn1                                         
mac             address                                time      static  router
0024.d4a0.0cd3  2a01:e0a:159:2850::1                   00:00:53  false   false
e03f.496d.1899  2a01:e0a:159:2850:e23f:49ff:fe6d:1899  00:00:53  false   false
0024.d4a0.0cd3  fe80::224:d4ff:fea0:cd3                00:00:53  false   false
e03f.496d.1899  fe80::e23f:49ff:fe6d:1899              00:00:53  false   false


Code Block
languagebash
themeMidnight
titleInitiate IPv4 ssh from freerouter -> LAN router gateway
p4pcap-freerouter#ssh 192.168.0.62 /vrf v1 /user my-nas                           
 - connecting to 192.168.0.62 22
password: *******
                
 - securing connection

Last login: Mon Jul  6 15:05:38 2020 from 192.168.0.77
FreeBSD 11.3-RELEASE-p9 (FreeNAS.amd64) #0 r325575+588899735f7(HEAD): Mon Jun  1 15:04:31 EDT 2020

        FreeNAS (c) 2009-2020, The FreeNAS Development Team
        All rights reserved.
        FreeNAS is released under the modified BSD license.

        For more information, documentation, help or support, go here:
        http://freenas.org
Welcome to FreeNAS
MY-NAS% 


Code Block
languagebash
themeMidnight
titleInitiate IPv6 ssh from freerouter -> LAN router gateway
p4pcap-freerouter#ssh 2a01:e0a:159:2850:e23f:49ff:fe6d:1899  /vrf v1 /user my-nas 
 - connecting to 2a01:e0a:159:2850:e23f:49ff:fe6d:1899 22
password: *******
                
 - securing connection

Last login: Tue Jul  7 16:01:54 2020 from 2a01:e0a:159:2850::666
FreeBSD 11.3-RELEASE-p9 (FreeNAS.amd64) #0 r325575+588899735f7(HEAD): Mon Jun  1 15:04:31 EDT 2020

        FreeNAS (c) 2009-2020, The FreeNAS Development Team
        All rights reserved.
        FreeNAS is released under the modified BSD license.

        For more information, documentation, help or support, go here:
        http://freenas.org
Welcome to FreeNAS
MY-NAS% 


...

Tip
title[ #005 ] RARE/FreeRouter-101 - key take-away
  • FreeRouter is using UNIX socket in order to forward packet dedicated to control plane + dataplane communication.

This essential paradigm is used to ensure communication between freeRouter and P4Emu/pcap dataplane. It is ensured by pcapInt binary from freeRouter net-tools that will bind freeRouter socket (veth251@locathost:22710) to a virtual network interface (veth250@localhost:22709)  connected to CPU_PORT 64 0.

  • freeRouter is the control plane for P4Emu/pcap dataplane

freeRouter is doing all the control plane route computation and write/modify/remove message entry P4 entries are created/modified/removed accordingly from P4Emu/pcap tables. Although the name is P4Emu, it does not emulate BMv2 V1Model.p4, but rather router.p4

  • In this setup the combination of freeRouter/pcap deliver a solution for SOHO network having 1GE links

However, 1GE traffic rate require 50% of one CPU thread. Nevertheless, traffic rate achieved is higher with P4Emu/pcap than freeRouter native software packet forwarding software.

Note

In subsequent article we will see how we can improve the latter requirement implied by P4Emu/pcap.


...