[P4-dev] complex function like pattern matching

Antonin Bas antonin at barefootnetworks.com
Wed Apr 19 12:39:17 EDT 2017


Adding back the list, as others may run into the same issue.

This error comes from a stupid optimization idea I had at the very
beginning of the bmv2 implementation: arithmetic operations would only be
enabled when a header field actually needs to be able to do arithmetic. I
expect it does save a few cycles, but the impact on overall is likely to be
negligible, and it has caused quite a few issues over time; which is why I
have been considering getting rid of it altogether.
The way it works is the bmv2 JSON parser detects which fields are never
used for arithmetic (in the P4 code) and turns off arithmetic on them. Here
you only access the fields from inside your extern and so the arithmetic is
disabled. Calling set_arith here will avoid the switch crashing but it will
not produce the correct result (get_uint will return an invalid value). I
would recommend that you call get_bytes() instead and work with the
ByteContainer object. That should not be a problem for you as your fields
seem to have a single byte (otherwise that is a very dubious assignment
that you are doing at the end):

auto &bc = hdr_instance.get_bytes();
assert(bc.size() == 1);
text[i] = bc.at(0);

On Wed, Apr 19, 2017 at 3:52 AM, yunchen chang <y2924uki at gmail.com> wrote:

> Hello Antonin:
>
> It works fine now.
> I am sorry for my mistake that I didn't remove the old version of p4-hlir
> yesterday.
>
> I still have a question.
>
> void operator ()(Field & result, HeaderStack &data) {
>
> char text[100] ;
>
> memset(text, '\0', 100);
>
> for (size_t i = 0; i < data.get_count(); i++) {
>
> auto &hdr_instance = data.at(i);
>
> assert(hdr_instance.is_valid());
>
> hdr_instance.get_field(0).set_arith(1); // here
>
> text[i] = hdr_instance.get_field(0).get_uint() ;
>
> }
>
>   }
>
> If I comment the code : "hdr_instance.get_field(0).set_arith(1);"
> There will be error message : lt-simple_switch:
> ../../include/bm/bm_sim/data.h:188: unsigned int bm::Data::get_uint()
> const: Assertion `arith' failed.
>
> Will it have effect if I uncomment the code? And what is the reason that
> cause the error?
>
> Thanks you so much.
>
> Best regards,
> Abbie
>
>
> On Tue, Apr 18, 2017 at 11:34 PM, Antonin Bas <
> antonin at barefootnetworks.com> wrote:
>
>> Please see inline
>>
>> On Tue, Apr 18, 2017 at 6:44 AM, yunchen chang <y2924uki at gmail.com>
>> wrote:
>>
>>> Hello Antonin:
>>>
>>> Thanks for the fix. But it still fail after I update it.
>>>
>>> But I make some change in p4c-bm(https://github.com/p
>>> 4lang/p4c-bm/blob/master/p4c_bm/gen_json.py#L1137).
>>> I change it to " if is_stack_ref(call_idx, arg_idx, primitive_name): ",
>>> then it works.
>>>
>>
>> This is not normal. I tested it this morning one more time. Maybe it
>> would help if you sent your P4 program and primitives.json file.
>>
>>
>>>
>>> ( Does it mean that I am using the version that has been abandoned? If
>>> yes, what is the affect if I keep using this version?)
>>>
>>
>> That would explain it, but based on your P4 snippets in this thread, you
>> are not using p4v1.1. Also it would require you to pass --p4v1.1 as a
>> command-line flag to p4c-bmv2 and I think you'd know if you were doing
>> that...
>>
>>
>>>
>>> After it work:
>>> I try to access the header_instance in header_stack.
>>>
>>> class pattern_match : public ActionPrimitive<HeaderStack &> {
>>>     void operator ()(HeaderStack &data) {
>>>         BMLOG_DEBUG("Header num : {}\n", data.get_count()  );
>>>         BMLOG_DEBUG("Header instance valid : {}\n",
>>> (data.get_next()).is_valid()  );
>>>     }
>>> }
>>>
>>> the result :
>>> > Header num : 17
>>> > Header instance valid : false
>>>
>>> I want to know why I can't access the header_instance.
>>>
>>
>> the "next" element in a header stack is never valid by definition...
>> here get_next() returns the 18th element in the stack, which is not valid
>> you can use get_last() to access the last valid element in the stack (in
>> this case the 17th element). You can also loop through the valid elements:
>> for (size_t i = 0; i < data.get_count(); i++) {
>>   auto &hdr_instance = data.at(i);
>>   assert(hdr_instance.is_valid())'
>> }
>>
>>
>>>
>>>
>>> I used to combine the header_instance to metadata in parser.
>>> But it can't access instance in parser too(it can only extract).
>>> And I know that it can't have loop in tables or actions.
>>> Are there other suggestions to combine the header_instances before I
>>> pass it to function(then I can avoid using header_stack)?
>>>
>>
>> I don't understand the question.
>>
>>
>>>
>>>
>>> more little question:
>>> I just installed p4c-bm and bmv2 before, and everything works fine.
>>> Do I must install the p4-hlir?
>>>
>>
>> p4-hlir is installed automatically when you install p4c-bm. However,
>> since I pushed an update to p4-hlir yesterday, you probably need to remove
>> the old p4-hlir you have installed and install the latest version.
>>
>>
>>>
>>> Thanks you so much.
>>>
>>> Best regards,
>>> Abbie
>>>
>>> On Tue, Apr 18, 2017 at 2:38 AM, Antonin Bas <
>>> antonin at barefootnetworks.com> wrote:
>>>
>>>> There is somewhat of a hack in p4-hlir to support P4_14 header stacks.
>>>> This had been fixed for the P4 v1.1 support, but this version of the
>>>> language has been abandoned :(
>>>> Anyway the hack was such that only the push & pop primitives were able
>>>> to support header stacks in their parameter list.
>>>> I pushed a fix to p4-hlir (https://github.com/p4lang/p4-hlir/) and
>>>> p4c-bm (https://github.com/p4lang/p4c-bm). I suggest that you update
>>>> both and try again. Your approach looks correct to me, so it should work
>>>> once you update.
>>>>
>>>> On Sat, Apr 15, 2017 at 5:35 AM, yunchen chang <y2924uki at gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Antonin :
>>>>>
>>>>> Thanks for your reply. It's really clear to me.
>>>>>
>>>>> But I face another question.
>>>>>
>>>>> I want to extract the host_name of dns packet, my idea is :
>>>>>
>>>>> 1. Store them into a header stack. (in parser)
>>>>>
>>>>> header_type one_byte_payload_t {
>>>>>     fields {
>>>>>       data : 8;
>>>>>     }
>>>>> }
>>>>>
>>>>> header one_byte_payload_t one_byte_payload[32];
>>>>>
>>>>> parser parse_dns_payload {
>>>>>     extract(one_byte_payload[next]);
>>>>>     return select(latest.data){
>>>>>        0x00: ingress;
>>>>>        default: parse_dns_payload;
>>>>>     }
>>>>> }
>>>>>
>>>>> 2. Pass the header stack to an function and do pattern matching. (in
>>>>> action)
>>>>>
>>>>> pattern_match(one_byte_payload);
>>>>>
>>>>> class pattern_match : public ActionPrimitive<HeaderStack &> {
>>>>>   void operator ()(HeaderStack &pattern) {
>>>>>     BMLOG_DEBUG("size :{}\n", pattern.get_count() );
>>>>>   }
>>>>> }
>>>>>
>>>>> REGISTER_PRIMITIVE(pattern_match);
>>>>>
>>>>> But after I call my own primitive function :
>>>>>
>>>>> the log file ( p4s.s1.log ) has error messages:
>>>>> lt-simple_switch: ../../include/bm/bm_sim/actions.h:337: T
>>>>> bm::ActionParam::to(bm::ActionEngineState*) const [with T =
>>>>> bm::HeaderStack&]: Assertion `tag == ActionParam::HEADER_STACK' failed.
>>>>>
>>>>> (I have added my primitive description to primitives.json too.)
>>>>> I guess something happened on type "HeaderStack".
>>>>> Even though I copy the "pop" function and rename to another function,
>>>>> the error still happened.
>>>>>
>>>>> Thanks you so much.
>>>>>
>>>>> Best regards,
>>>>> Abbie
>>>>>
>>>>> On Wed, Apr 12, 2017 at 12:42 AM, Antonin Bas <
>>>>> antonin at barefootnetworks.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please see inline
>>>>>>
>>>>>> On Tue, Apr 11, 2017 at 12:41 AM, yunchen chang <y2924uki at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Antonin:
>>>>>>>
>>>>>>> Thanks for your reply.
>>>>>>>
>>>>>>> I have another questions about primitives function.
>>>>>>>
>>>>>>> 1) Can I call any C++ functions? (like string compare etc)
>>>>>>>
>>>>>>
>>>>>> Yes, why not? You can write any C++ code.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> 2) This https://github.com/p4lang/p4c-bm/blob/master/p4c_bm/pri
>>>>>>> mitives.json is uesd by p4 compiler to know the functions it have,
>>>>>>> am I right? But why there are only few functions in this file? Primitive
>>>>>>> functions of simple_switch (https://github.com/p4lang/beh
>>>>>>> avioral-model/blob/master/targets/simple_switch/primitives.cpp)
>>>>>>> don't need to be known by compiler?
>>>>>>>
>>>>>>
>>>>>> The standard primitives which are described in the P4_14 spec are in
>>>>>> a different location: https://github.com/p
>>>>>> 4lang/p4-hlir/blob/master/p4_hlir/frontend/primitives.json
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> 3) Because I only see https://github.com/p4lang/
>>>>>>> p4c-bm/blob/master/p4c_bm/primitives.json this code. I want to know
>>>>>>> what the format of properties: types and data_width means.
>>>>>>> For example : modify_field_rng_uniform :
>>>>>>> "type" : ["field", "int", "table_entry_data"]
>>>>>>> and "data_width" : "dst"
>>>>>>> Are there any documents explain the information?
>>>>>>>
>>>>>>
>>>>>> No document, just available examples.
>>>>>> In "type" you can list the acceptable types for a given parameter.
>>>>>> Accepted values are "field", "int" (for an literal constant),
>>>>>> "table_entry_data" (for integral values that come from action data),
>>>>>> "header_instance", "register", "counter", "meter",...
>>>>>> "data_width" is only required for parameters for which
>>>>>> "table_entry_data" is a possible type. This is needed to allocate the
>>>>>> appropriate memory for tables. It can either be an integer value, the name
>>>>>> of another parameter (which cannot itself be of type "table_entry_data") or
>>>>>> a field reference.
>>>>>> In the case of "modify_field_rng_uniform" for example, parameters
>>>>>> "begin" and "end" can either be literal constants, field references or
>>>>>> table action data ("table_entry_data"). In the case where "begin" (or
>>>>>> "end") comes from table action data, the number of bits we allocate for it
>>>>>> in a table entry is determined by the bitwidth of the "dst" parameter,
>>>>>> which is a field reference.
>>>>>> In the following code:
>>>>>> table t_set_port {
>>>>>>     reads { ... }
>>>>>>     actions { set_port; }
>>>>>> }
>>>>>> action set_port(b, e) {
>>>>>>   modify_field_rng_uniform(standard_metadata.egress_spec /* 9 bit by
>>>>>> definition */, b, e);
>>>>>> }
>>>>>> Both "b" and "e" come from action data. They are assumed to be 9-bit,
>>>>>> based on the width for standard_metadata.egress_spec. This is different
>>>>>> from P4_16, where types for action parameters are explicit.
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Thanks you so much.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Abbie
>>>>>>>
>>>>>>> On Fri, Apr 7, 2017 at 2:52 AM, Antonin Bas <
>>>>>>> antonin at barefootnetworks.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Yes you can add your code to primitives.cpp. There are many
>>>>>>>> examples you can look at so this shouldn't be too hard. Don't forget to
>>>>>>>> call the REGISTER_PRIMITIVE macro. Note that after you modify the
>>>>>>>> simple_switch code, it will not be the "standard" simple_switch any more.
>>>>>>>> It will be your own version of it, which supports one extra primitive.
>>>>>>>>
>>>>>>>> In order to have the p4c-bm compiler support your new primitive,
>>>>>>>> you will need to add your primitive description to
>>>>>>>> https://github.com/p4lang/p4c-bm/blob/master/p4c_bm/primitives.json.
>>>>>>>> Once again, there are many examples there that you can look at, but if you
>>>>>>>> need help you can send an email to the list.
>>>>>>>>
>>>>>>>> Finally, I just want to stress out what Andy already said. Just
>>>>>>>> because it runs on bmv2 simple_switch doesn't mean it will run on other
>>>>>>>> P4-programmable targets.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Antonin
>>>>>>>>
>>>>>>>> On Thu, Apr 6, 2017 at 1:01 AM, yunchen chang <y2924uki at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello Andy,
>>>>>>>>>
>>>>>>>>> Thanks for your response.
>>>>>>>>>
>>>>>>>>> 1) I am using mininet(simple_switch.cpp) to be my environment,
>>>>>>>>>     so I want to know where should I implement my C/C++ code? (
>>>>>>>>> primitives.cpp?)
>>>>>>>>>
>>>>>>>>> 2) Is there anything I should pay special attention to? ( like:
>>>>>>>>> restrictions of compiler )
>>>>>>>>>
>>>>>>>>> Thank you.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Abbie
>>>>>>>>>
>>>>>>>>> On Mon, Apr 3, 2017 at 8:35 AM, Andy Fingerhut <
>>>>>>>>> andy.fingerhut at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> The P4 language is focused on parsing and manipulating packet
>>>>>>>>>> headers, not payloads.
>>>>>>>>>>
>>>>>>>>>> One could write a custom extension that could search for patterns
>>>>>>>>>> in a payload, but it would be non-portable, and would have to be
>>>>>>>>>> implemented in a target-specific language, e.g. C/C++ for a software model,
>>>>>>>>>> Verilog for an ASIC or FPGA, etc.  It would be similar to having a library
>>>>>>>>>> written in assembler and calling it from C, with the assembler implemented
>>>>>>>>>> anew for each target processor.
>>>>>>>>>>
>>>>>>>>>> Andy
>>>>>>>>>>
>>>>>>>>>> On Mon, Mar 27, 2017 at 9:13 AM, yunchen chang <
>>>>>>>>>> y2924uki at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello everyone,
>>>>>>>>>>>
>>>>>>>>>>> I am doing an experiment for DPI in P4.
>>>>>>>>>>> (detect packet label in data plane, not in controller)
>>>>>>>>>>>
>>>>>>>>>>> I need some complex funtcions like lookup the host name of
>>>>>>>>>>> payload.
>>>>>>>>>>> Example:
>>>>>>>>>>> host_name: tw.yahoo.com
>>>>>>>>>>> mtach_key: .yahoo.
>>>>>>>>>>>
>>>>>>>>>>> I plan to write function in "primitives.cpp" at first.
>>>>>>>>>>>
>>>>>>>>>>> But after I read this posted : [http://lists.p4.org/pipermail
>>>>>>>>>>> /p4-dev_lists.p4.org/2016-August/000449.html], it pointed that
>>>>>>>>>>> P4 can't implement "pattern matching".
>>>>>>>>>>>
>>>>>>>>>>> I want to know is there any solution in current P4?
>>>>>>>>>>> (or any suggestion)
>>>>>>>>>>>
>>>>>>>>>>> Thank you.
>>>>>>>>>>>
>>>>>>>>>>> Best regards,
>>>>>>>>>>> Abbie
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> P4-dev mailing list
>>>>>>>>>>> P4-dev at lists.p4.org
>>>>>>>>>>> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> P4-dev mailing list
>>>>>>>>> P4-dev at lists.p4.org
>>>>>>>>> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Antonin
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Antonin
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Antonin
>>>>
>>>
>>>
>>
>>
>> --
>> Antonin
>>
>
>


-- 
Antonin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p4.org/pipermail/p4-dev_lists.p4.org/attachments/20170419/9696384b/attachment-0002.html>


More information about the P4-dev mailing list