[P4-dev] API change in v1model

Nate Foster jnfoster at cs.cornell.edu
Sun Oct 1 23:59:46 EDT 2017


Indeed, we will add this to the agenda of one of the upcoming meetings of
the P4 design group.

Without my co-chair hat on: my personal preference at this point in time
would be for a solution that consists of some community norms and practices
rather than a full-blown language-based solution here. For example, a
well-documented release process for p4c and associated architectures, with
versions, combined with @deprecated annotations as Mihai suggests might do
the trick. I do realize that architecture changes are particularly
disruptive -- in a sense, a new architecture creates a new dialect, since
it changes the top-level structure of programs and the set of built-in
externs -- but there are a lot of other balls in the air so I'm not sure
that designing a P4-specific build system it's where I'd focus energy next.

Having said all that, obviously if someone feels particularly passionate
about the issue, I'm sure the design group would consider a well-done
proposal.

-N

On Thu, Sep 28, 2017 at 1:34 PM, Mihai Budiu <mbudiu at vmware.com> wrote:

> This should be on the agenda of one of the future P4 language design
> meetings.
>
> Or perhaps it could be discussed as part of the PSA meetings; this is more
> relevant to the architecture working group.
>
> The question is: how to manage changes to the architectures and libraries.
>
>
>
> Mihai
>
>
>
> *From:* hemant at mnkcg.com [mailto:hemant at mnkcg.com]
> *Sent:* Thursday, September 28, 2017 10:27 AM
> *To:* 'King, Steven R' <steven.r.king at intel.com>; Mihai Budiu <
> mbudiu at vmware.com>; p4-dev at lists.p4.org
> *Cc:* 'Chris Dodd' <chris at barefootnetworks.com>
> *Subject:* RE: [P4-dev] API change in v1model
>
>
>
> Guys,
>
>
>
> Cisco VoiP Webex is free.  Maybe, folks can setup a Webex meeting and
> discus.
>
>
>
> Hemant
>
>
>
> *From:* P4-dev [mailto:p4-dev-bounces at lists.p4.org
> <p4-dev-bounces at lists.p4.org>] *On Behalf Of *King, Steven R
> *Sent:* Thursday, September 28, 2017 12:14 PM
> *To:* Mihai Budiu <mbudiu at vmware.com>; p4-dev at lists.p4.org
> *Cc:* Chris Dodd <chris at barefootnetworks.com>
> *Subject:* Re: [P4-dev] API change in v1model
>
>
>
> All decent build system (e.g. cmake) understand and address the need for
> custom flows for custom tools.
>
> How to move this discussion forward?  You’re giving the perception that P4
> breaking changes shall be handled by emails.
>
>
>
>
>
> *From:* Mihai Budiu [mailto:mbudiu at vmware.com <mbudiu at vmware.com>]
> *Sent:* Thursday, September 28, 2017 8:30 AM
> *To:* King, Steven R <steven.r.king at intel.com>; p4-dev at lists.p4.org
> *Cc:* Chris Dodd <chris at barefootnetworks.com>; Calin Cascaval <
> cascaval at barefootnetworks.com>
> *Subject:* RE: API change in v1model
>
>
>
> Currently P4 does not have a build system. It will probably be difficult
> to standardize on one, because we expect that each vendor will have a
> different compiler back-end, and requires a different tool-chain, depending
> on their target.
>
>
>
> Mihai
>
>
>
> *From:* King, Steven R [mailto:steven.r.king at intel.com
> <steven.r.king at intel.com>]
> *Sent:* Wednesday, September 27, 2017 10:27 AM
> *To:* Mihai Budiu <mbudiu at vmware.com>; p4-dev at lists.p4.org
> *Cc:* Chris Dodd <chris at barefootnetworks.com>; Calin Cascaval <
> cascaval at barefootnetworks.com>
> *Subject:* RE: API change in v1model
>
>
>
> OK, tricky problem.  Restating the goal, users want succinct build time
> error messages for source/library incompatibilities.  Specifically,
> incompatibilities should NOT appear as source code errors.
>
>
>
> How do you feel about solving this problem with a build system?  For
> example, the Haskell and javascript worlds use build systems that enforce
> versioned dependencies.  In this approach, users specify requirements in a
> “package” file, separate from their P4 source files.  The build system
> needs a way to inspect library versions, but we can imagine implementing
> this in backwards compatible way.
>
>
>
> A first class build system brings other benefits as well.   Maybe there’s
> a way to just use cmake here?
>
>
>
> Regards,
>
> -steve
>
>
>
>
>
> *From:* Mihai Budiu [mailto:mbudiu at vmware.com <mbudiu at vmware.com>]
> *Sent:* Wednesday, September 27, 2017 9:36 AM
> *To:* King, Steven R <steven.r.king at intel.com>; p4-dev at lists.p4.org
> *Cc:* Chris Dodd <chris at barefootnetworks.com>; Calin Cascaval <
> cascaval at barefootnetworks.com>
> *Subject:* RE: API change in v1model
>
>
>
> The situation is complicated.
>
>
>
> The architectural model is called v1model because it is a model of P4
> v1.0, aka P4-14. It is supposed to reflect two things:
>
> -          The architectural model of the switch from the P4-14 spec
>
> -          The implementation of this switch in the behavioral model, as
> the simple_switch model.
>
>
>
> So we do not have much freedom in what v1model looks like (and calling it
> v2model would be inappropriate; perhaps v1model-1.1). I can argue that I am
> just fixing a bug in v1model, because it did not really match these two
> pieces of specification that we have.
>
>
>
> On the other hand, you are right, people have started programming against
> v1model, and it has taken a life of its own, since there was no alternative
> available (the PSA standard architecture is not yet ready). This is
> problematic for two types of people:
>
> -          People who have written code using v1model for bmv2; for these
> I can argue that the (manual) migration cost is relatively small; I have
> manually migrated all programs in our testsuite in less than one hour
>
> -          People who have written back-ends that depend on v1model. This
> is much more difficult to handle, since the migration cost would be much
> larger. For these guys the @deprecated annotation would be helpful, but may
> not be sufficient.
>
>
>
> For a “language solution” we have several problems:
>
> -          I don’t know what the right solution is
>
> -          Even if we knew what the solution is, if it involves changing
> the language it would require us to wait for P4-18 (or whatever the next
> release), so it would not really solve our problem. Changing the language
> spec should be much harder than changing any library.
>
>
>
> Our language has some extensibility mechanisms designed exactly to absorb
> some change; ideally we should be able to leverage these, but I don’t know
> how to handle this case.
>
>
>
> We could also do some work in the compiler to continue to support both
> versions, but I think that this is a bad long term decision, because our
> current solution does not really work, (having about 7 open issues, some of
> which have been closed when the initial patch to fix this was merged, but
> not reopened when the patch was reverted).
>
>
>
> Mihai
>
>
>
> *From:* King, Steven R [mailto:steven.r.king at intel.com
> <steven.r.king at intel.com>]
> *Sent:* Wednesday, September 27, 2017 8:55 AM
> *To:* Mihai Budiu <mbudiu at vmware.com>; p4-dev at lists.p4.org
> *Subject:* RE: API change in v1model
>
>
>
> I don’t like the @deprecated idea because the library is not deprecated
> and presumably works as-is for customers such as myself.
>
>
>
> Your proposal breaks the 1.0.0 library, not for frivolous reasons, but
> still breaks it.  Barring a way to check compatibility, then you should not
> break the v1model, but call your new library a different name, .e.g.
> “v2model”.
>
>
>
> My real suggestion is implement a proper language solution for
> compatibility checks then deploy this library change to the community as
> the first acid test.
>
>
>
>
>
> *From:* Mihai Budiu [mailto:mbudiu at vmware.com <mbudiu at vmware.com>]
> *Sent:* Wednesday, September 27, 2017 8:41 AM
> *To:* Mihai Budiu <mbudiu at vmware.com>; King, Steven R <
> steven.r.king at intel.com>; p4-dev at lists.p4.org
> *Subject:* RE: API change in v1model
>
>
>
> And we appreciate suggestions on how to improve on this state of affairs.
>
>
>
> Mihai
>
>
>
> *From:* P4-dev [mailto:p4-dev-bounces at lists.p4.org
> <p4-dev-bounces at lists.p4.org>] *On Behalf Of *Mihai Budiu
> *Sent:* Wednesday, September 27, 2017 8:36 AM
> *To:* King, Steven R <steven.r.king at intel.com>; p4-dev at lists.p4.org
> *Subject:* Re: [P4-dev] API change in v1model
>
>
>
> We’ll add a @deprecated annotation for now.
>
> Another solution is to have a version number in the library as a constant
> which the compiler can check.
>
>
>
> But this is a difficult problem. Ideally a model should never change once
> it is released; we could have families of models, e.g. v1model-1.0.p4,
> v1model-1.1.p4. But this will require extensive testing before a model is
> officially released.
>
>
>
> Mihai
>
>
>
> *From:* King, Steven R [mailto:steven.r.king at intel.com
> <steven.r.king at intel.com>]
> *Sent:* Wednesday, September 27, 2017 8:32 AM
> *To:* Mihai Budiu <mbudiu at vmware.com>; p4-dev at lists.p4.org
> *Subject:* RE: API change in v1model
>
>
>
> P4 language and libraries (models?) need a succinct way for source code to
> express required compatibility.  This feature is typical of many
> languages.  Does such a feature exist in P4?  How does a P4 user guard
> against breaking changes?
>
>
>
> If there is no such compatibility guard feature, then I’d humbly suggest
> breaking changes be deferred until after a compatibility check mechanism is
> put in place.  P4 will fall to pieces if this problem goes unchecked.
>
>
>
> Regards,
>
> -steve
>
>
>
>
>
>
>
>
>
> *From:* P4-dev [mailto:p4-dev-bounces at lists.p4.org
> <p4-dev-bounces at lists.p4.org>] *On Behalf Of *Mihai Budiu
> *Sent:* Tuesday, September 26, 2017 11:26 AM
> *To:* Mihai Budiu <mbudiu at vmware.com>; p4-dev at lists.p4.org
> *Subject:* Re: [P4-dev] API change in v1model
>
>
>
> Here is the pull request which introduces these changes:
> https://github.com/p4lang/p4c/pull/939
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_p4lang_p4c_pull_939&d=DwMFAg&c=uilaK90D4TOVoH58JNXRgQ&r=tGW6TKXajnoXSyy1S1P4DHGPe8sj54GGvw-b21n7aWg&m=ivJpqTPl1dWO93e2B4PF2Jzo_V0jvxUeTrNtxugVbso&s=xfy7AB29mxGrnhVAU3P9WJuWUzec9yyYJTu7QEvECfI&e=>
>
>
>
> There are lots of examples of P4-16 programs in the testdata folder that
> were converted to use the new API; this includes all programs whose name
> ends in –bmv2.p4.
>
>
>
> The plan is to merge this patch towards the end of this week if it is
> approved.
>
>
>
> Please let us know if this will be causing you any problems or you need
> more time.
>
>
>
> Thank you,
>
> Mihai Budiu
>
>
>
> *From:* P4-dev [mailto:p4-dev-bounces at lists.p4.org
> <p4-dev-bounces at lists.p4.org>] *On Behalf Of *Mihai Budiu
> *Sent:* Monday, September 25, 2017 10:16 AM
> *To:* p4-dev at lists.p4.org
> *Subject:* [P4-dev] API change in v1model
>
>
>
> Hello everyone,
>
>
>
> This email is about a breaking change we are planning in the P4-16
> compiler when compiling P4-16 written for the v1model.p4 architecture. The
> compiler support for checksums has many open issues (i.e., bugs). One
> reason is that the way checksums were described in v1model.p4 does not
> really match the way checksums are implemented in the behavioral model
> simple_switch simulator. This change is trying to align the implementation
> of the compiler to be closer with the implementation of the target and help
> us close all these issues.
>
>
>
> This change will have no impact on users that write P4-14 programs; the
> compiler will handle these programs transparently. This change will only
> impact users who have written P4-16 programs against the v1model.p4
> architecture. Users will have to rewrite their programs to compile with the
> new API. Unfortunately the effort to automate this rewriting is
> prohibitive, so we ask users to make it by hand; we apologize for this
> inconvenience. We do not anticipate to make other breaking changes of this
> magnitude to v1model.p4 in the future.
>
>
>
> We are planning to make the following changes:
>
>    1. Change the API for the VerifyChecksum control block
>    2. Add an bit to standard_metadata_t , which will be used to indicate
>    checksum errors.
>    3. Change the API for performing checksum computations
>    4. Make explicit the rules about all legal ways to compute and update
>    checksums in P4-16 using v1model.p4
>
>
>
> This message explains these changes and how the P4-16 programs using
> v1model.p4 have to be rewritten manually to work with the new APIs.
> 1.    The VerifyChecksum control will have the headers as an inout
> parameter (instead of just in):
>
>
>
> control VerifyChecksum<H, M>(in H hdr, inout M meta)
>
>
>
> changes to
>
>
>
> control VerifyChecksum<H, M>(inout H hdr, inout M meta)
>
>
>
> Even though the control block is not expected to change the headers, this
> change is needed because the behavioral model API requires a left-value to
> be passed for the expected value of the checksum (i.e. it does not support
> a constant value for the expected value of the checksum).
> 2.    The standard_metadata_t structure will have an additional field:
>
>
>
> struct standard_metadata_t {
>
>>
>    bit<1> checksum_error;
>
> }
>
>
>
> The intended meaning for bit is to be set in the VerifyChecksum block
> whenever a checksum verification fails. Setting this bit will be done by
> the behavioral model runtime, and thus this will require a patch to the
> behavioral model, which will happen subsequently after the API is
> introduced. So this functionality may not be available right away, but we
> first need the bit to be exposed to implement the functionality.
> 3.    The way checksums are computed is the most significant change:
>
>
>
> The current API is:
>
>
>
> extern Checksum16 {
>
>     checksum16();
>
>     bit<16> get<D>(in D data);
>
> }
>
>
>
> The proposed new API is much closer to the way checksums work in P4-14 and
> the behavioral model today:
>
>
>
> /**
>
> Verifies the checksum of the supplied data.
>
> If this method detects that a checksum of the data is not correct it
>
> sets the standard_metadata checksum_error bit.
>
> @param T          Must be a tuple type where all the fields are bit-fields
> or varbits.
>
>                   The total dynamic length of the fields is a multiple of
> the output size.
>
> @param O          Output type; must be bit<X> type.
>
> @param condition  If 'false' the verification always succeeds.
>
> @param data       Data whose checksum is verified.
>
> @param checksum   Expected checksum of the data; note that is must be a
> left-value.
>
> @param algo       Algorithm to use for checksum (not all algorithms may be
> supported).
>
>                   Must be a compile-time constant.
>
> */
>
> extern void verify_checksum<T, O>(in bool condition, in T data, inout O
> checksum, HashAlgorithm algo);
>
> /**
>
> Computes the checksum of the supplied data.
>
> @param T          Must be a tuple type where all the fields are bit-fields
> or varbits.
>
>                   The total dynamic length of the fields is a multiple of
> the output size.
>
> @param O          Output type; must be bit<X> type.
>
> @param condition  If 'false' the checksum is not changed
>
> @param data       Data whose checksum is computed.
>
> @param checksum   Checksum of the data.
>
> @param algo       Algorithm to use for checksum (not all algorithms may be
> supported).
>
>                   Must be a compile-time constant.
>
> */
>
> extern void update_checksum<T, O>(in bool condition, in T data, inout O
> checksum, HashAlgorithm algo);
>
>
> 4.    The specification for the allowed code in the control blocks that
> manipulate checksums changes as follows:
>
>
>
> The only legal statements in the implementation of the VerifyChecksum
> control are: block statements, calls to the verify_checksum method, and
> return statements.
>
> The only legal statements in the implementation of the UpdateChecksum
> control are: block statements, calls to update_checksum method, and return
> statements.
>
>
>
> In particular, no conditionals, actions or table invocations are permitted.
> Impact on your programs:
>
>
>
> -          If you write P4-14 programs there is no impact; the converter
> to P4-16 will automatically use the new APIs.
>
> -          If you write P4-16 programs for v1model.p4, you will need to
> adapt your program.
>
>
>
> The following example shows how the flowlet_switching-bmv2.p4 program from
> the P4-16 testsuite must be rewritten .
>
>
>
> Original code:
>
>
>
> control verifyChecksum(in headers hdr, inout metadata meta) {
>
>     Checksum16() ipv4_checksum;
>
>     apply {
>
>         if (hdr.ipv4.hdrChecksum == ipv4_checksum.get({ hdr.ipv4.version,
> hdr.ipv4.ihl, hdr.ipv4.diffserv, hdr.ipv4.totalLen,
> hdr.ipv4.identification, hdr.ipv4.flags, hdr.ipv4.fragOffset, hdr.ipv4.ttl,
> hdr.ipv4.protocol, hdr.ipv4.srcAddr, hdr.ipv4.dstAddr }))
>
>             mark_to_drop();
>
>     }
>
> }
>
>
>
> control computeChecksum(inout headers hdr, inout metadata meta) {
>
>     Checksum16() ipv4_checksum;
>
>     apply {
>
>         hdr.ipv4.hdrChecksum = ipv4_checksum.get({ hdr.ipv4.version,
> hdr.ipv4.ihl, hdr.ipv4.diffserv, hdr.ipv4.totalLen,
> hdr.ipv4.identification, hdr.ipv4.flags, hdr.ipv4.fragOffset, hdr.ipv4.ttl,
> hdr.ipv4.protocol, hdr.ipv4.srcAddr, hdr.ipv4.dstAddr });
>
>    }
>
> }
>
>
>
> Code using new API:
>
>
>
> control verifyChecksum(inout headers hdr, inout metadata meta) {
>
>     apply {
>
>         verify_checksum(true, { hdr.ipv4.version, hdr.ipv4.ihl, hdr.ipv4.diffserv, hdr.ipv4.totalLen, hdr.ipv4.identification, hdr.ipv4.flags, hdr.ipv4.fragOffset, hdr.ipv4.ttl, hdr.ipv4.protocol, hdr.ipv4.srcAddr, hdr.ipv4.dstAddr }, hdr.ipv4.hdrChecksum, HashAlgorithm.csum16);
>
>     }
>
> }
>
>
>
> control computeChecksum(inout headers hdr, inout metadata meta) {
>
>     apply {
>
>         update_checksum(true, { hdr.ipv4.version, hdr.ipv4.ihl, hdr.ipv4.diffserv, hdr.ipv4.totalLen, hdr.ipv4.identification, hdr.ipv4.flags, hdr.ipv4.fragOffset, hdr.ipv4.ttl, hdr.ipv4.protocol, hdr.ipv4.srcAddr, hdr.ipv4.dstAddr }, hdr.ipv4.hdrChecksum, HashAlgorithm.csum16);
>
>     }
>
> }
>
>
>
>
>
> I will be very happy to assist personally anyone that has trouble
> migrating to the new API.
>
>
>
> We apologize again for this disruptive change, but we believe that this is
> the best way to resolve the many issues we have with checksum computations
> in v1model. The problems fundamentally stem from a disconnect between the
> way the behavioral model implements checksums and the way v1model.p4
> attempted to model them.
>
>
>
> Note that we expect that the Portable Switch Architecture PSA “standard”
> model, which is expected to replace v1model in the future, is *not *expected
> to adopt a similar API with the new proposal, but will continue to use an
> extern-based API.
>
>
>
> Thank you,
>
> Mihai Budiu
>
> _______________________________________________
> P4-dev mailing list
> P4-dev at lists.p4.org
> http://lists.p4.org/mailman/listinfo/p4-dev_lists.p4.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p4.org/pipermail/p4-dev_lists.p4.org/attachments/20171001/ba957a4d/attachment-0002.html>


More information about the P4-dev mailing list