[P4-dev] complex function like pattern matching

yunchen chang y2924uki at gmail.com
Thu Apr 20 00:00:45 EDT 2017


Hello Antonin:

Everything works fine now after I use ByteContainer object.
Thank you for explaining clearly and giving me advice:) .

Best regards,
Abbie



On Thu, Apr 20, 2017 at 12:39 AM, Antonin Bas <antonin at barefootnetworks.com>
wrote:

> Adding back the list, as others may run into the same issue.
>
> This error comes from a stupid optimization idea I had at the very
> beginning of the bmv2 implementation: arithmetic operations would only be
> enabled when a header field actually needs to be able to do arithmetic. I
> expect it does save a few cycles, but the impact on overall is likely to be
> negligible, and it has caused quite a few issues over time; which is why I
> have been considering getting rid of it altogether.
> The way it works is the bmv2 JSON parser detects which fields are never
> used for arithmetic (in the P4 code) and turns off arithmetic on them. Here
> you only access the fields from inside your extern and so the arithmetic is
> disabled. Calling set_arith here will avoid the switch crashing but it will
> not produce the correct result (get_uint will return an invalid value). I
> would recommend that you call get_bytes() instead and work with the
> ByteContainer object. That should not be a problem for you as your fields
> seem to have a single byte (otherwise that is a very dubious assignment
> that you are doing at the end):
>
> auto &bc = hdr_instance.get_bytes();
> assert(bc.size() == 1);
> text[i] = bc.at(0);
>
> On Wed, Apr 19, 2017 at 3:52 AM, yunchen chang <y2924uki at gmail.com> wrote:
>
>> Hello Antonin:
>>
>> It works fine now.
>> I am sorry for my mistake that I didn't remove the old version of p4-hlir
>> yesterday.
>>
>> I still have a question.
>>
>> void operator ()(Field & result, HeaderStack &data) {
>>
>> char text[100] ;
>>
>> memset(text, '\0', 100);
>>
>> for (size_t i = 0; i < data.get_count(); i++) {
>>
>> auto &hdr_instance = data.at(i);
>>
>> assert(hdr_instance.is_valid());
>>
>> hdr_instance.get_field(0).set_arith(1); // here
>>
>> text[i] = hdr_instance.get_field(0).get_uint() ;
>>
>> }
>>
>>   }
>>
>> If I comment the code : "hdr_instance.get_field(0).set_arith(1);"
>> There will be error message : lt-simple_switch:
>> ../../include/bm/bm_sim/data.h:188: unsigned int bm::Data::get_uint()
>> const: Assertion `arith' failed.
>>
>> Will it have effect if I uncomment the code? And what is the reason that
>> cause the error?
>>
>> Thanks you so much.
>>
>> Best regards,
>> Abbie
>>
>>
>> On Tue, Apr 18, 2017 at 11:34 PM, Antonin Bas <
>> antonin at barefootnetworks.com> wrote:
>>
>>> Please see inline
>>>
>>> On Tue, Apr 18, 2017 at 6:44 AM, yunchen chang <y2924uki at gmail.com>
>>> wrote:
>>>
>>>> Hello Antonin:
>>>>
>>>> Thanks for the fix. But it still fail after I update it.
>>>>
>>>> But I make some change in p4c-bm(https://github.com/p
>>>> 4lang/p4c-bm/blob/master/p4c_bm/gen_json.py#L1137).
>>>> I change it to " if is_stack_ref(call_idx, arg_idx, primitive_name): ",
>>>> then it works.
>>>>
>>>
>>> This is not normal. I tested it this morning one more time. Maybe it
>>> would help if you sent your P4 program and primitives.json file.
>>>
>>>
>>>>
>>>> ( Does it mean that I am using the version that has been abandoned? If
>>>> yes, what is the affect if I keep using this version?)
>>>>
>>>
>>> That would explain it, but based on your P4 snippets in this thread, you
>>> are not using p4v1.1. Also it would require you to pass --p4v1.1 as a
>>> command-line flag to p4c-bmv2 and I think you'd know if you were doing
>>> that...
>>>
>>>
>>>>
>>>> After it work:
>>>> I try to access the header_instance in header_stack.
>>>>
>>>> class pattern_match : public ActionPrimitive<HeaderStack &> {
>>>>     void operator ()(HeaderStack &data) {
>>>>         BMLOG_DEBUG("Header num : {}\n", data.get_count()  );
>>>>         BMLOG_DEBUG("Header instance valid : {}\n",
>>>> (data.get_next()).is_valid()  );
>>>>     }
>>>> }
>>>>
>>>> the result :
>>>> > Header num : 17
>>>> > Header instance valid : false
>>>>
>>>> I want to know why I can't access the header_instance.
>>>>
>>>
>>> the "next" element in a header stack is never valid by definition...
>>> here get_next() returns the 18th element in the stack, which is not valid
>>> you can use get_last() to access the last valid element in the stack (in
>>> this case the 17th element). You can also loop through the valid elements:
>>> for (size_t i = 0; i < data.get_count(); i++) {
>>>   auto &hdr_instance = data.at(i);
>>>   assert(hdr_instance.is_valid())'
>>> }
>>>
>>>
>>>>
>>>>
>>>> I used to combine the header_instance to metadata in parser.
>>>> But it can't access instance in parser too(it can only extract).
>>>> And I know that it can't have loop in tables or actions.
>>>> Are there other suggestions to combine the header_instances before I
>>>> pass it to function(then I can avoid using header_stack)?
>>>>
>>>
>>> I don't understand the question.
>>>
>>>
>>>>
>>>>
>>>> more little question:
>>>> I just installed p4c-bm and bmv2 before, and everything works fine.
>>>> Do I must install the p4-hlir?
>>>>
>>>
>>> p4-hlir is installed automatically when you install p4c-bm. However,
>>> since I pushed an update to p4-hlir yesterday, you probably need to remove
>>> the old p4-hlir you have installed and install the latest version.
>>>
>>>
>>>>
>>>> Thanks you so much.
>>>>
>>>> Best regards,
>>>> Abbie
>>>>
>>>> On Tue, Apr 18, 2017 at 2:38 AM, Antonin Bas <
>>>> antonin at barefootnetworks.com> wrote:
>>>>
>>>>> There is somewhat of a hack in p4-hlir to support P4_14 header stacks.
>>>>> This had been fixed for the P4 v1.1 support, but this version of the
>>>>> language has been abandoned :(
>>>>> Anyway the hack was such that only the push & pop primitives were able
>>>>> to support header stacks in their parameter list.
>>>>> I pushed a fix to p4-hlir (https://github.com/p4lang/p4-hlir/) and
>>>>> p4c-bm (https://github.com/p4lang/p4c-bm). I suggest that you update
>>>>> both and try again. Your approach looks correct to me, so it should work
>>>>> once you update.
>>>>>
>>>>> On Sat, Apr 15, 2017 at 5:35 AM, yunchen chang <y2924uki at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Antonin :
>>>>>>
>>>>>> Thanks for your reply. It's really clear to me.
>>>>>>
>>>>>> But I face another question.
>>>>>>
>>>>>> I want to extract the host_name of dns packet, my idea is :
>>>>>>
>>>>>> 1. Store them into a header stack. (in parser)
>>>>>>
>>>>>> header_type one_byte_payload_t {
>>>>>>     fields {
>>>>>>       data : 8;
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> header one_byte_payload_t one_byte_payload[32];
>>>>>>
>>>>>> parser parse_dns_payload {
>>>>>>     extract(one_byte_payload[next]);
>>>>>>     return select(latest.data){
>>>>>>        0x00: ingress;
>>>>>>        default: parse_dns_payload;
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> 2. Pass the header stack to an function and do pattern matching. (in
>>>>>> action)
>>>>>>
>>>>>> pattern_match(one_byte_payload);
>>>>>>
>>>>>> class pattern_match : public ActionPrimitive<HeaderStack &> {
>>>>>>   void operator ()(HeaderStack &pattern) {
>>>>>>     BMLOG_DEBUG("size :{}\n", pattern.get_count() );
>>>>>>   }
>>>>>> }
>>>>>>
>>>>>> REGISTER_PRIMITIVE(pattern_match);
>>>>>>
>>>>>> But after I call my own primitive function :
>>>>>>
>>>>>> the log file ( p4s.s1.log ) has error messages:
>>>>>> lt-simple_switch: ../../include/bm/bm_sim/actions.h:337: T
>>>>>> bm::ActionParam::to(bm::ActionEngineState*) const [with T =
>>>>>> bm::HeaderStack&]: Assertion `tag == ActionParam::HEADER_STACK' failed.
>>>>>>
>>>>>> (I have added my primitive description to primitives.json too.)
>>>>>> I guess something happened on type "HeaderStack".
>>>>>> Even though I copy the "pop" function and rename to another function,
>>>>>> the error still happened.
>>>>>>
>>>>>> Thanks you so much.
>>>>>>
>>>>>> Best regards,
>>>>>> Abbie
>>>>>>
>>>>>> On Wed, Apr 12, 2017 at 12:42 AM, Antonin Bas <
>>>>>> antonin at barefootnetworks.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Please see inline
>>>>>>>
>>>>>>> On Tue, Apr 11, 2017 at 12:41 AM, yunchen chang <y2924uki at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Antonin:
>>>>>>>>
>>>>>>>> Thanks for your reply.
>>>>>>>>
>>>>>>>> I have another questions about primitives function.
>>>>>>>>
>>>>>>>> 1) Can I call any C++ functions? (like string compare etc)
>>>>>>>>
>>>>>>>
>>>>>>> Yes, why not? You can write any C++ code.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> 2) This https://github.com/p4lang/p4c-bm/blob/master/p4c_bm/pri
>>>>>>>> mitives.json is uesd by p4 compiler to know the functions it have,
>>>>>>>> am I right? But why there are only few functions in this file? Primitive
>>>>>>>> functions of simple_switch (https://github.com/p4lang/beh
>>>>>>>> avioral-model/blob/master/targets/simple_switch/primitives.cpp)
>>>>>>>> don't need to be known by compiler?
>>>>>>>>
>>>>>>>
>>>>>>> The standard primitives which are described in the P4_14 spec are in
>>>>>>> a different location: https://github.com/p
>>>>>>> 4lang/p4-hlir/blob/master/p4_hlir/frontend/primitives.json
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> 3) Because I only see https://github.com/p4lang/
>>>>>>>> p4c-bm/blob/master/p4c_bm/primitives.json this code. I want to
>>>>>>>> know what the format of properties: types and data_width means.
>>>>>>>> For example : modify_field_rng_uniform :
>>>>>>>> "type" : ["field", "int", "table_entry_data"]
>>>>>>>> and "data_width" : "dst"
>>>>>>>> Are there any documents explain the information?
>>>>>>>>
>>>>>>>
>>>>>>> No document, just available examples.
>>>>>>> In "type" you can list the acceptable types for a given parameter.
>>>>>>> Accepted values are "field", "int" (for an literal constant),
>>>>>>> "table_entry_data" (for integral values that come from action data),
>>>>>>> "header_instance", "register", "counter", "meter",...
>>>>>>> "data_width" is only required for parameters for which
>>>>>>> "table_entry_data" is a possible type. This is needed to allocate the
>>>>>>> appropriate memory for tables. It can either be an integer value, the name
>>>>>>> of another parameter (which cannot itself be of type "table_entry_data") or
>>>>>>> a field reference.
>>>>>>> In the case of "modify_field_rng_uniform" for example, parameters
>>>>>>> "begin" and "end" can either be literal constants, field references or
>>>>>>> table action data ("table_entry_data"). In the case where "begin" (or
>>>>>>> "end") comes from table action data, the number of bits we allocate for it
>>>>>>> in a table entry is determined by the bitwidth of the "dst" parameter,
>>>>>>> which is a field reference.
>>>>>>> In the following code:
>>>>>>> table t_set_port {
>>>>>>>     reads { ... }
>>>>>>>     actions { set_port; }
>>>>>>> }
>>>>>>> action set_port(b, e) {
>>>>>>>   modify_field_rng_uniform(standard_metadata.egress_spec /* 9 bit
>>>>>>> by definition */, b, e);
>>>>>>> }
>>>>>>> Both "b" and "e" come from action data. They are assumed to be
>>>>>>> 9-bit, based on the width for standard_metadata.egress_spec. This is
>>>>>>> different from P4_16, where types for action parameters are explicit.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks you so much.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Abbie
>>>>>>>>
>>>>>>>> On Fri, Apr 7, 2017 at 2:52 AM, Antonin Bas <
>>>>>>>> antonin at barefootnetworks.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Yes you can add your code to primitives.cpp. There are many
>>>>>>>>> examples you can look at so this shouldn't be too hard. Don't forget to
>>>>>>>>> call the REGISTER_PRIMITIVE macro. Note that after you modify the
>>>>>>>>> simple_switch code, it will not be the "standard" simple_switch any more.
>>>>>>>>> It will be your own version of it, which supports one extra primitive.
>>>>>>>>>
>>>>>>>>> In order to have the p4c-bm compiler support your new primitive,
>>>>>>>>> you will need to add your primitive description to
>>>>>>>>> https://github.com/p4lang/p4c-bm/blob/master/p4c_bm/primi
>>>>>>>>> tives.json. Once again, there are many examples there that you
>>>>>>>>> can look at, but if you need help you can send an email to the list.
>>>>>>>>>
>>>>>>>>> Finally, I just want to stress out what Andy already said. Just
>>>>>>>>> because it runs on bmv2 simple_switch doesn't mean it will run on other
>>>>>>>>> P4-programmable targets.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Antonin
>>>>>>>>>
>>>>>>>>> On Thu, Apr 6, 2017 at 1:01 AM, yunchen chang <y2924uki at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Andy,
>>>>>>>>>>
>>>>>>>>>> Thanks for your response.
>>>>>>>>>>
>>>>>>>>>> 1) I am using mininet(simple_switch.cpp) to be my environment,
>>>>>>>>>>     so I want to know where should I implement my C/C++ code? (
>>>>>>>>>> primitives.cpp?)
>>>>>>>>>>
>>>>>>>>>> 2) Is there anything I should pay special attention to? ( like:
>>>>>>>>>> restrictions of compiler )
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Abbie
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 3, 2017 at 8:35 AM, Andy Fingerhut <
>>>>>>>>>> andy.fingerhut at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> The P4 language is focused on parsing and manipulating packet
>>>>>>>>>>> headers, not payloads.
>>>>>>>>>>>
>>>>>>>>>>> One could write a custom extension that could search for
>>>>>>>>>>> patterns in a payload, but it would be non-portable, and would have to be
>>>>>>>>>>> implemented in a target-specific language, e.g. C/C++ for a software model,
>>>>>>>>>>> Verilog for an ASIC or FPGA, etc.  It would be similar to having a library
>>>>>>>>>>> written in assembler and calling it from C, with the assembler implemented
>>>>>>>>>>> anew for each target processor.
>>>>>>>>>>>
>>>>>>>>>>> Andy
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Mar 27, 2017 at 9:13 AM, yunchen chang <
>>>>>>>>>>> y2924uki at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> I am doing an experiment for DPI in P4.
>>>>>>>>>>>> (detect packet label in data plane, not in controller)
>>>>>>>>>>>>
>>>>>>>>>>>> I need some complex funtcions like lookup the host name of
>>>>>>>>>>>> payload.
>>>>>>>>>>>> Example:
>>>>>>>>>>>> host_name: tw.yahoo.com
>>>>>>>>>>>> mtach_key: .yahoo.
>>>>>>>>>>>>
>>>>>>>>>>>> I plan to write function in "primitives.cpp" at first.
>>>>>>>>>>>>
>>>>>>>>>>>> But after I read this posted : [http://lists.p4.org/pipermail
>>>>>>>>>>>> /p4-dev_lists.p4.org/2016-August/000449.html], it pointed that
>>>>>>>>>>>> P4 can't implement "pattern matching".
>>>>>>>>>>>>
>>>>>>>>>>>> I want to know is there any solution in current P4?
>>>>>>>>>>>> (or any suggestion)
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Abbie
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> P4-dev mailing list
>>>>>>>>>>>> P4-dev at lists.p4.org
>>>>>>>>>>>> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> P4-dev mailing list
>>>>>>>>>> P4-dev at lists.p4.org
>>>>>>>>>> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Antonin
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Antonin
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Antonin
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Antonin
>>>
>>
>>
>
>
> --
> Antonin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p4.org/pipermail/p4-dev_lists.p4.org/attachments/20170420/9f82d927/attachment-0002.html>


More information about the P4-dev mailing list