[P4-dev] Parse a sflow packet which contain N numbers of samples (header + payload)

Vladimir Gurevich vladimir.gurevich at barefootnetworks.com
Thu Dec 27 20:21:36 EST 2018

Hello Lee,

If all you need to do it to process a pcap file and you do not need to do
it in real-time, you might have a lot more success using Scapy. Seriously.
It has the built-in ability to read/write PCAP files and it has a lot more
powerful parsing engine. Moreover, it gives you the full power of the
Python language to process your packets. In fact, you can even  use the
same code to receive packets on one interface and send them out of another,
as long as the packet rate is fairly slow.

If you want to process your packets at a relatively modest, "software" rate
and at real-time, then, of course you can use BMv2 with P4 and follow
Andy's excellent suggestions.

However, if you want to eventually process these packets at much higher
rates on the real chips, capable of processing billions packets per second
you might have a problem. The reason is that in these pipelines packets are
typically processed at a constant (or very well bounded) amount of time,
meaning that loops should either not be present in the code, be unrolled
or, at least, kept to an absolute (and well-bounded) minimum. For that same
reason, these pipelines cannot process unlimited amounts of payload (and
there are other reasons for that as well). Because P4 was designed to be
efficiently implementable on this kind of high-speed packet processing
pipelines, it also lacks these features which makes it really difficult to
process the kinds of packets you want, which is pretty much what you

Happy hacking,
*Vladimir Gurevich*

*Barefoot Networks*
*Technical Lead, Customer Engineering*
Email: vag at barefootnetworks.com
Phone: (408) 833-4505

On Thu, Dec 27, 2018 at 10:53 AM Andy Fingerhut <andy.fingerhut at gmail.com>

> I cannot think of any approaches that you have not tried or described
> already.
> Processing deep into a packet is not one of the strengths of P4.  A P4
> implementation that lets you parse an entire payload of a large packet,
> without encountering some kind of out of resources error, could do it.  I
> would expect that most P4 software switch implementations like BMv2 could
> do what you want, either as-is or with very small changes to the source
> code to increase some maximum resource limit somewhere.
> Was it BMv2 that you were using when you experienced the "not enough local
> memory for thread local, failed to allocate local memory" error you
> mentioned?  Are you interested in the possibility of small changes to BMv2
> source code that would enable your program to run successfully?  Even if
> that means your P4 program will work only on the probably small fraction of
> implementations that allow such deep parsing to be performed successfully?
> Andy
> On Wed, Dec 26, 2018 at 8:59 PM lee cyrus <LEE_LOKYIN_c2 at lab.ntt.co.jp>
> wrote:
>> Hello,
>> A little background a what I am working/investigating on. I have a pcap
>> file filled with sflow packets. Each packet is formed with Outer headers
>> + Samples, and the Samples are bunch of Inner header + Payload. The
>> picture below is the structure of a sflow packet that I am trying to
>> parse. What I am trying to do is to remove some header inside each Inner
>> Header section.
>> | Outer Headers  (4 headers)
>> | Inner Headers     (9 headers)
>> | Payload
>> | Inner Headers     (9 headers)
>> | Payload
>> | ...... (Inner header + payload can stack up to 10 sets)
>> Since I only want to modify the Inner Header section, I want to parse
>> the outer header and inner header but not the payload. But as the
>> picture above shown that each inner header is separated by a payload and
>> I need to parse that payload in order to parse the next inner header. I
>> did create a header that will extract the payload bits but because each
>> payload is quite large so eventually I ended up with a compile error
>> said "not enough local memory for thread local, failed to allocate local
>> memory". So I got two work around to this problem but both of them don't
>> seems to be feasible.
>> 1)    The first work around is to define only two sets of inner header +
>> payload and use recirculate() to iterate all the sample. But there is
>> another problem. There isn't any function regarding to skipping bits.
>> Obviously during the second iteration I don't want to parse the inner
>> headers that has been modified before. I read other thread that was
>> posted in 2015, they said P4 didn't have explicit "skip" function in the
>> parser. I also read the latest P4_14 spec and it doesn't said anything
>> about skipping as well. So I wonder has P4 implement anything similar
>> like that now.
>> 2)    The second work around is remove and add the modified inner
>> headers with its payload to the bottom of the sample stack. But I found
>> the location where the add header function will add to is right after
>> the last parsed header. Which means I will have to parse everything
>> inside the packet in order to add to the bottom of the stack. But this
>> is impossible because of the memory error.
>> To conclude my problem, I want to know how to parse a packet with this
>> sort of structure where it has payload in between each inner headers.
>> There aren't any post regarding to parsing sflow packet so I don't know
>> what is the recommended way to do so. If anyone have any ideas or
>> experience with sflow packet. Please share your information with me. I
>> will deeply appreciate any inputs.
>> Thanks in advance,
>> Cyrus Lee
>> _______________________________________________
>> P4-dev mailing list
>> P4-dev at lists.p4.org
>> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
> _______________________________________________
> P4-dev mailing list
> P4-dev at lists.p4.org
> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p4.org/pipermail/p4-dev_lists.p4.org/attachments/20181227/47d7fd15/attachment.html>

More information about the P4-dev mailing list