[P4-dev] INT demo issue - Bad UDP checksum for packets headed to monitor

Rakesh Kumar kumar19 at illinois.edu
Sat May 13 16:59:44 EDT 2017


Okay. The solution to this lies in the quirks of how linux bridges get
filtered. Essentially, you need to go in disable all the filters so that
the UDP packets can get through. Details here:
https://wiki.linuxfoundation.org/networking/bridge

But essentially:

 # cd /proc/sys/net/bridge
 # for f in bridge-nf-*; do echo 0 > $f; done

See attached screenshot of the working INT web UI.


-
rakesh

On Thu, May 11, 2017 at 11:26 PM, Rakesh Kumar <kumar19 at illinois.edu> wrote:

> Couple of quick question Rakesh.
>> 1) Did you install the VXLAN_GPE driver and the other dependencies
>> mentioned in this readme?
>>
>
> Yes, I verified with dmesg/lsmod.
>
>
>> 2) Once you run the mininet scripts you should see a few log files
>> (prefixed with tmp_) generated in the path <p4factory>/mininet. Do they
>> report any error?
>>
>>
> No errors: A warning line saying "No route found for  IPv6 destination ::
> " in each one of the host files(i.e. tmp_h$).
>
>
>> 1) Run the mininet script
>> 2) Start a long running iperf session say, h1 perf -c 10.2.1.4 -t 100
>> 3) Open another window and console into
>>      a) switch leaf1 (since it is connected to the source h1).
>> Collect tcpdump on the interface leaf1-eth1 (interface connected to h1).
>> The host h1 is supposed to add the INT shim header. Verify if the INT
>> header is added.
>>
>      b) switch leaf1. Collect tcpdump on the interface leaf1-eth3
>> or leaf1-eth4 depending upon which interface the packets go out of.
>> leaf1 should have appended INT metadata.
>>      c) switch leaf2 (since it is connected to the destination h4).
>> Collect tcpdump on the interface leaf2-eth2 (interface connected to h4).
>> 4) Do refer the INT spec when verifying the tcpdump.
>>
>
> So, I did this with a  ping -c1 10.2.1.4  at h1 (instead of iperf). See
> attached screenshots for a captured icmp request and reply packet pair at
> leaf1-eth1, leaf1-eth3 and leaf2-eth2. I think it seems to be working, i.e.
> metadata is being added, the hop count is increasing and the length is
> increasing. I see this as UDP port 4790 traffic which as I understand is
> supposed to be carrying VXLAN-GPE...
>
>
>
>> 5) If every hop is appending it's metadata information, then proceed with
>> checking the monitor.
>>
> 6) Make sure you have all the dependencies for the monitor installed.
>> Refer readme.
>>
>
>
>>
>> Let me know if you have any more questions.
>>
>>
> From preprocessor.py, I can see UDP packets being sent via the socket to
> monitor. However, as I said in the previous email, these packets have bad
> checksum and never arrive at the interfaces named veth-t1/veth-t2 (verified
> this by tcpdump) on the host machine. Any ideas why these packets are not
> getting through the testbr1 and why they might have a bad checksum?
>
>
>
> PS: You need to build switch with switchlink.
>>
>>
> I did. Used this configuration as mentioned in the README (./configure
> --with-bmv2 --with-switchlink --enable-thrift --prefix=$HOME/install)
>
> -
> rakesh
>
> On Wed, May 10, 2017 at 11:54 AM, Avinash Herle <avinash.herle@
> barefootnetworks.com> wrote:
>
>> Couple of quick question Rakesh.
>> 1) Did you install the VXLAN_GPE driver and the other dependencies
>> mentioned in this readme?
>> 2) Once you run the mininet scripts you should see a few log files
>> (prefixed with tmp_) generated in the path <p4factory>/mininet. Do they
>> report any error?
>>
>> Please try the following
>>
>> 1) Run the mininet script
>> 2) Start a long running iperf session say, h1 perf -c 10.2.1.4 -t 100
>> 3) Open another window and console into
>>      a) switch leaf1 (since it is connected to the source h1).
>> Collect tcpdump on the interface leaf1-eth1 (interface connected to h1).
>> The host h1 is supposed to add the INT shim header. Verify if the INT
>> header is added.
>>      b) switch leaf1. Collect tcpdump on the interface leaf1-eth3
>> or leaf1-eth4 depending upon which interface the packets go out of.
>> leaf1 should have appended INT metadata.
>>      c) switch leaf2 (since it is connected to the destination h4).
>> Collect tcpdump on the interface leaf2-eth2 (interface connected to h4).
>> 4) Do refer the INT spec when verifying the tcpdump.
>> 5) If every hop is appending it's metadata information, then proceed with
>> checking the monitor.
>> 6) Make sure you have all the dependencies for the monitor installed.
>> Refer readme.
>>
>> Let me know if you have any more questions.
>>
>> Also, do not reload/refresh the web client when the application is
>> running. You might have to re-run the application in this case. The Readme
>> mentions a few known monitor issues and troubleshooting tips.
>>
>> PS: You need to build switch with switchlink.
>>
>> Thanks,
>> Avinash
>>
>> On Wed, May 10, 2017 at 12:01 AM, Yongfeng Sun <sunnogo at gmail.com> wrote:
>>
>>> I encountered this issue, too. But it's a bit different.
>>>
>>> I find that vm-eth22 got the packet after preprocessor.py which strips
>>> the INT body from vxlan-gpe body. However, web client doesn't show the flow.
>>>
>>> Use tshark to capture the packet:
>>>
>>>  sunyongfeng  ~  workshop  p4factory  mininet  sudo tshark -i
>>> vm-eth22
>>> [sudo] password for sunyongfeng:
>>> Running as user "root" and group "root". This could be dangerous.
>>> tshark: Lua: Error during loading:
>>>  [string "/usr/share/wireshark/init.lua"]:44: dofile has been disabled
>>> due to running Wireshark as superuser. See
>>> https://wiki.wireshark.org/CaptureSetup/CapturePrivileges for help in
>>> running Wireshark as an unprivileged user.
>>> Capturing on 'vm-eth22'
>>>   1 0.000000000 192.168.1.44 -> 192.168.1.100 UDP 62 47895 → 54321
>>>  Len=20
>>> ...
>>> 118 packets captured
>>>  sunyongfeng  ~  workshop  p4factory  mininet  sudo tshark -i
>>> vm-eth22 -Vc 1
>>> Running as user "root" and group "root". This could be dangerous.
>>> tshark: Lua: Error during loading:
>>>  [string "/usr/share/wireshark/init.lua"]:44: dofile has been disabled
>>> due to running Wireshark as superuser. See
>>> https://wiki.wireshark.org/CaptureSetup/CapturePrivileges for help in
>>> running Wireshark as an unprivileged user.
>>> Capturing on 'vm-eth22'
>>> Frame 1: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) on
>>> interface 0
>>>     Interface id: 0 (vm-eth22)
>>>     Encapsulation type: Ethernet (1)
>>>     Arrival Time: May 10, 2017 14:55:59.203745483 CST
>>>     [Time shift for this packet: 0.000000000 seconds]
>>>     Epoch Time: 1494399359.203745483 seconds
>>>     [Time delta from previous captured frame: 0.000000000 seconds]
>>>     [Time delta from previous displayed frame: 0.000000000 seconds]
>>>     [Time since reference or first frame: 0.000000000 seconds]
>>>     Frame Number: 1
>>>     Frame Length: 62 bytes (496 bits)
>>>     Capture Length: 62 bytes (496 bits)
>>>     [Frame is marked: False]
>>>     [Frame is ignored: False]
>>>     [Protocols in frame: eth:ethertype:ip:udp:data]
>>> Ethernet II, Src: 52:c7:b0:b9:09:a0 (52:c7:b0:b9:09:a0), Dst:
>>> 66:64:01:1c:df:8a (66:64:01:1c:df:8a)
>>>     Destination: 66:64:01:1c:df:8a (66:64:01:1c:df:8a)
>>>         Address: 66:64:01:1c:df:8a (66:64:01:1c:df:8a)
>>>         .... ..1. .... .... .... .... = LG bit: Locally administered
>>> address (this is NOT the factory default)
>>>         .... ...0 .... .... .... .... = IG bit: Individual address
>>> (unicast)
>>>     Source: 52:c7:b0:b9:09:a0 (52:c7:b0:b9:09:a0)
>>>         Address: 52:c7:b0:b9:09:a0 (52:c7:b0:b9:09:a0)
>>>         .... ..1. .... .... .... .... = LG bit: Locally administered
>>> address (this is NOT the factory default)
>>>         .... ...0 .... .... .... .... = IG bit: Individual address
>>> (unicast)
>>>     Type: IPv4 (0x0800)
>>> Internet Protocol Version 4, Src: 192.168.1.44, Dst: 192.168.1.100
>>>     0100 .... = Version: 4
>>>     .... 0101 = Header Length: 20 bytes
>>>     Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
>>>         0000 00.. = Differentiated Services Codepoint: Default (0)
>>>         .... ..00 = Explicit Congestion Notification: Not ECN-Capable
>>> Transport (0)
>>>     Total Length: 48
>>>     Identification: 0x0b77 (2935)
>>>     Flags: 0x02 (Don't Fragment)
>>>         0... .... = Reserved bit: Not set
>>>         .1.. .... = Don't fragment: Set
>>>         ..0. .... = More fragments: Not set
>>>     Fragment offset: 0
>>>     Time to live: 64
>>>     Protocol: UDP (17)
>>>     Header checksum: 0xab65 [validation disabled]
>>>         [Good: False]
>>>         [Bad: False]
>>>     Source: 192.168.1.44
>>>     Destination: 192.168.1.100
>>>     [Source GeoIP: Unknown]
>>>     [Destination GeoIP: Unknown]
>>> User Datagram Protocol, Src Port: 47895 (47895), Dst Port: 54321 (54321)
>>>     Source Port: 47895
>>>     Destination Port: 54321
>>>     Length: 28
>>>     Checksum: 0x840e [validation disabled]
>>>         [Good Checksum: False]
>>>         [Bad Checksum: False]
>>>     [Stream index: 0]
>>> Data (20 bytes)
>>>
>>> 0000  0a 02 01 04 0a 02 01 03 08 00 78 bc 00 00 0a 01   ..........x.....
>>> 0010  00 03 20 00                                       .. .
>>>     Data: 0a0201040a020103080078bc00000a0100032000
>>>     [Length: 20]
>>>
>>> 1 packet captured
>>>
>>>
>>> Rakesh Kumar <kumar19 at illinois.edu>于2017年5月10日周三 上午10:14写道:
>>>
>>>> Hey All,
>>>>
>>>> I was trying to recreate the INT demo on a Ubuntu 14.04 LTS VM and
>>>> followed the instructions here: https://github.com/p4lan
>>>> g/p4factory/tree/master/apps/int
>>>>
>>>> The network comes up, the hosts are able to connect to each other (i.e.
>>>> pings and iperfs between them work), but the web application does not see
>>>> any counter updates (i.e. the time series all sit at 0).
>>>>
>>>> So, I tried debugging it. Ultimately, I used tcpdump on each host and
>>>> verified that there are UDP packets being sent to the monitor at
>>>> 192.168.1.100 (IP address for the interface on the native host). These
>>>> packets never make it to the destination interface and have a bad UDP
>>>> checksum. This explains why the counters are zero, because of course there
>>>> are no updates reaching the monitor. But I don't have many ideas about how
>>>> to fix this?
>>>>
>>>> Any ideas on what might be causing this? Is it normal/expected behavior
>>>> due to some header modifications?
>>>>
>>>>
>>>> Thanks and Regards,
>>>>
>>>>
>>>> -
>>>> rakesh
>>>>
>>>> P.S: Is there a slack where devs hangout if I needed more interactive
>>>> help about anything?
>>>> _______________________________________________
>>>> P4-dev mailing list
>>>> P4-dev at lists.p4.org
>>>> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
>>>
>>>
>>> _______________________________________________
>>> P4-dev mailing list
>>> P4-dev at lists.p4.org
>>> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p4.org/pipermail/p4-dev_lists.p4.org/attachments/20170513/780a6f4e/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2017-05-13 at 3.58.32 PM.png
Type: image/png
Size: 1504478 bytes
Desc: not available
URL: <http://lists.p4.org/pipermail/p4-dev_lists.p4.org/attachments/20170513/780a6f4e/attachment.png>


More information about the P4-dev mailing list