The new requirement from Cisco and CableLabs is the ability to coalesce
multiple reports into one packet. In addition we need to add Domain
Specific ID, Domain Specific Md Bits, and an extension mechanism for
arbitrary domain specific information.
In order to meet these requirements, the packet format needs to be
restructured, separating fields common to all reports in the packet from
fields that are specific to each report in the packet.
In order to allow for multiple reports in a packet, each with its own
packet fragment or flow identifying information, the report header needs to
change from a small header that is followed by a packet fragment, to a
report header that encapsulates a packet fragment. With this in mind, I
suggest that we don't need a next protocol. All the contents are within the
reports. There is nothing that needs to come after the reports.
There are two high level options for the packet structure:
Compact Telemetry Report v2.0 proposal
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| RepMdBits |Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
0: None
1: Domain Specific Extensions
2: Ethernet Packet Fragment
3: IPv4 Packet Fragment
4: IPv6 Packet Fragment
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |D|Q|F|I| Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| RepMdBits | Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Prot | Length | Reserved | Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Rsvd | Length | Domain Specific Md Status | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
Any preferences? questions? comments? suggestions?
Mickey
Hi Mickey,
Thanks for suggesting the interesting options.
I see that the major difference between the two lies in the co-existence of Packet Fragment and Domain Specific Extensions. The compact option treats them mutually exclusive, the TLV allows co-existence. I'd let Ramesh and Randy comment on this based on their use case scenarios.
hw_id and seq # fields are sharing 24 bits.
Can you remind us the case when we need 8b hw_id (vs 6b in the v1.0 spec)?
Thanks,
JK
From: P4-apps p4-apps-bounces@lists.p4.org on behalf of Mickey Spiegel mspiegel@barefootnetworks.com
Sent: Wednesday, March 11, 2020 2:12 PM
To: p4-apps@lists.p4.org p4-apps@lists.p4.org
Subject: [P4-Apps] Telemetry Report 2.0 options
The new requirement from Cisco and CableLabs is the ability to coalesce multiple reports into one packet. In addition we need to add Domain Specific ID, Domain Specific Md Bits, and an extension mechanism for arbitrary domain specific information.
In order to meet these requirements, the packet format needs to be restructured, separating fields common to all reports in the packet from fields that are specific to each report in the packet.
In order to allow for multiple reports in a packet, each with its own packet fragment or flow identifying information, the report header needs to change from a small header that is followed by a packet fragment, to a report header that encapsulates a packet fragment. With this in mind, I suggest that we don't need a next protocol. All the contents are within the reports. There is nothing that needs to come after the reports.
There are two high level options for the packet structure:
Compact Telemetry Report v2.0 proposal
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| RepMdBits |Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
0: None
1: Domain Specific Extensions
2: Ethernet Packet Fragment
3: IPv4 Packet Fragment
4: IPv6 Packet Fragment
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |D|Q|F|I| Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| RepMdBits | Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Prot | Length | Reserved | Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Rsvd | Length | Domain Specific Md Status | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
Any preferences? questions? comments? suggestions?
Mickey
On Wed, Mar 11, 2020 at 5:09 PM Lee, Jeongkeun jk.lee@intel.com wrote:
Hi Mickey,
Thanks for suggesting the interesting options.
I see that the major difference between the two lies in the co-existence
of Packet Fragment and Domain Specific Extensions. The compact option
treats them mutually exclusive, the TLV allows co-existence. I'd let Ramesh
and Randy comment on this based on their use case scenarios.
hw_id and seq # fields are sharing 24 bits.
Can you remind us the case when we need 8b hw_id (vs 6b in the v1.0 spec)?
6 bits accommodates 16 line cards or stackable units * 4 hw_id values for
each. That is probably enough. I only rounded up from 6 to 8 bits because
I did not think 20 bits vs 22 bits of sequence number is significant. We
can go either way.
Mickey
Thanks,
JK
From: P4-apps p4-apps-bounces@lists.p4.org on behalf of Mickey
Spiegel mspiegel@barefootnetworks.com
Sent: Wednesday, March 11, 2020 2:12 PM
To: p4-apps@lists.p4.org p4-apps@lists.p4.org
Subject: [P4-Apps] Telemetry Report 2.0 options
The new requirement from Cisco and CableLabs is the ability to coalesce
multiple reports into one packet. In addition we need to add Domain
Specific ID, Domain Specific Md Bits, and an extension mechanism for
arbitrary domain specific information.
In order to meet these requirements, the packet format needs to be
restructured, separating fields common to all reports in the packet from
fields that are specific to each report in the packet.
In order to allow for multiple reports in a packet, each with its own
packet fragment or flow identifying information, the report header needs to
change from a small header that is followed by a packet fragment, to a
report header that encapsulates a packet fragment. With this in mind, I
suggest that we don't need a next protocol. All the contents are within the
reports. There is nothing that needs to come after the reports.
There are two high level options for the packet structure:
1. The more compact format uses an "InnerProtocol" field to identify
whether a packet fragment or domain specific extensions are included in the
report. In order to get to those fields, the receiver of the report has to
parse through variable optional metadata based on RepMdBits and Domain
Specific Md Bits. If it does not understand how to parse those bits, then
it cannot figure out where the packet fragment or domain specific
extensions begin. This format adds only 4 bytes, compared to the Telemetry
Report v1.0 format.
2. Nested TLV structure. An implementation that does not support
coalescing needs to generate two levels of TLVs, each with only one TLV,
with two different lengths values. This format adds 8 bytes compared to the
Telemetry Report v1.0 format.
Compact Telemetry Report v2.0 proposal
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| RepMdBits |Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
1. There is no outer "length" field, since it is largely redundant
with the UDP and IP length fields.
2. The Sequence Number has been reduced from 32 bits to 20 bits in
order to save 4 bytes. We could go with any of 16, 20, or 22 bit Sequence
Number.
3. Ingress Timestamp gets a RepMdBit value and moves into Variable
Optional Metadata.
4. Length of each report is limited to 1024 bytes.
5. RepMdBits and Domain Specific Md Bits share 28 bits. Propose 14
bits each.
6. Domain Specific Md Status has been dropped. Use Domain Specific
Extensions if necessary.
7. Inner Protocol:
0: None
1: Domain Specific Extensions
2: Ethernet Packet Fragment
3: IPv4 Packet Fragment
4: IPv6 Packet Fragment
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |D|Q|F|I| Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| RepMdBits | Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Prot | Length | Reserved | Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Rsvd | Length | Domain Specific Md Status | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
1. There is no outer "length" field, since it is largely redundant
with the UDP and IP length fields.
2. The Sequence Number has been reduced from 32 bits to 20 bits in
order to save 4 bytes. We could go with any of 16, 20, or 22 bit Sequence
Number.
3. Ingress Timestamp gets a RepMdBit value and moves into Variable
Optional Metadata.
4. Length of each report is limited to 1024 bytes. If we want to
expand that to 16k, then DQFI would have to move down, squeezing RepMdBits
and Domain Specific Md Bits to 28 bits, and either burn a separate type for
each packet fragment protocol, or move Protocol to the 16 Reserved bits on
the right.
5. Domain Specific Md Status has been moved to Domain Specific
Extensions.
6. Type (inner)
1. Packet Fragment
2. Domain Specific Extensions
Any preferences? questions? comments? suggestions?
Mickey
Thanks Mickey for coming up with the format options.
From my perspective, Nested TLV is flexible and address all know use cases. The only comment I have is that Instruction Bitmap in the INT Header is 16 bits and Domain Specific Instruction is 16 bits. RepMD bits and Domain Specific MD bits add up to a max of 28 bits (without any reserved bits) in Compact format and 32-bits in Nested TLV format. Does it make sense to move the Domain Specific MD bits to Domain Specific extensions, similar to Domain Specific Md Status.
Thoughts?
-Ramesh
From: P4-apps p4-apps-bounces@lists.p4.org on behalf of Mickey Spiegel mspiegel@barefootnetworks.com
Date: Wednesday, March 11, 2020 at 5:50 PM
To: "Lee, Jeongkeun" jk.lee@intel.com
Cc: "p4-apps@lists.p4.org" p4-apps@lists.p4.org, "Alvarez, Daniel A" daniel.a.alvarez@intel.com
Subject: Re: [P4-Apps] Telemetry Report 2.0 options
On Wed, Mar 11, 2020 at 5:09 PM Lee, Jeongkeun <jk.lee@intel.commailto:jk.lee@intel.com> wrote:
Hi Mickey,
Thanks for suggesting the interesting options.
I see that the major difference between the two lies in the co-existence of Packet Fragment and Domain Specific Extensions. The compact option treats them mutually exclusive, the TLV allows co-existence. I'd let Ramesh and Randy comment on this based on their use case scenarios.
hw_id and seq # fields are sharing 24 bits.
Can you remind us the case when we need 8b hw_id (vs 6b in the v1.0 spec)?
6 bits accommodates 16 line cards or stackable units * 4 hw_id values for each. That is probably enough. I only rounded up from 6 to 8 bits because I did not think 20 bits vs 22 bits of sequence number is significant. We can go either way.
Mickey
Thanks,
JK
From: P4-apps <p4-apps-bounces@lists.p4.orgmailto:p4-apps-bounces@lists.p4.org> on behalf of Mickey Spiegel <mspiegel@barefootnetworks.commailto:mspiegel@barefootnetworks.com>
Sent: Wednesday, March 11, 2020 2:12 PM
To: p4-apps@lists.p4.orgmailto:p4-apps@lists.p4.org <p4-apps@lists.p4.orgmailto:p4-apps@lists.p4.org>
Subject: [P4-Apps] Telemetry Report 2.0 options
The new requirement from Cisco and CableLabs is the ability to coalesce multiple reports into one packet. In addition we need to add Domain Specific ID, Domain Specific Md Bits, and an extension mechanism for arbitrary domain specific information.
In order to meet these requirements, the packet format needs to be restructured, separating fields common to all reports in the packet from fields that are specific to each report in the packet.
In order to allow for multiple reports in a packet, each with its own packet fragment or flow identifying information, the report header needs to change from a small header that is followed by a packet fragment, to a report header that encapsulates a packet fragment. With this in mind, I suggest that we don't need a next protocol. All the contents are within the reports. There is nothing that needs to come after the reports.
There are two high level options for the packet structure:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| RepMdBits |Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |D|Q|F|I| Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| RepMdBits | Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Prot | Length | Reserved | Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Rsvd | Length | Domain Specific Md Status | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
* Packet Fragment
* Domain Specific Extensions
Any preferences? questions? comments? suggestions?
Mickey
Hi Mickey,
Thanks for pulling together these two proposals.
It looks like both options will address our immediate need to collate multiple packet fragments into a single report. The 1024 size limitation would not be an issue for the use cases which we are exploring.
While we are not currently sending any domain specific TR data, just domain specific data in the INT report. I do see option where we would send some additional domain specific data from the switch/sink generating the TR. For instance, we may leverage some counters or include the current BGP flowspec rules. If this domain specific data doesn’t need to be repeated on each report within the packet, so both option will work.
I like the versatility of the Nested TLV Telemetry Report, since it allows for both domain specific metadata and the packet fragment to be in one report. I don’t have a compelling use for both domain specific and packet fragment info per report at this time.
Thanks,
Randy Levensalor
From: P4-apps p4-apps-bounces@lists.p4.org on behalf of Mickey Spiegel mspiegel@barefootnetworks.com
Date: Wednesday, March 11, 2020 at 6:50 PM
To: "Lee, Jeongkeun" jk.lee@intel.com
Cc: "p4-apps@lists.p4.org" p4-apps@lists.p4.org, "Alvarez, Daniel A" daniel.a.alvarez@intel.com
Subject: Re: [P4-Apps] Telemetry Report 2.0 options
CableLabs WARNING: The sender of this email could not be validated and may not match the person in the "From" field.
On Wed, Mar 11, 2020 at 5:09 PM Lee, Jeongkeun <jk.lee@intel.commailto:jk.lee@intel.com> wrote:
Hi Mickey,
Thanks for suggesting the interesting options.
I see that the major difference between the two lies in the co-existence of Packet Fragment and Domain Specific Extensions. The compact option treats them mutually exclusive, the TLV allows co-existence. I'd let Ramesh and Randy comment on this based on their use case scenarios.
hw_id and seq # fields are sharing 24 bits.
Can you remind us the case when we need 8b hw_id (vs 6b in the v1.0 spec)?
6 bits accommodates 16 line cards or stackable units * 4 hw_id values for each. That is probably enough. I only rounded up from 6 to 8 bits because I did not think 20 bits vs 22 bits of sequence number is significant. We can go either way.
Mickey
Thanks,
JK
From: P4-apps <p4-apps-bounces@lists.p4.orgmailto:p4-apps-bounces@lists.p4.org> on behalf of Mickey Spiegel <mspiegel@barefootnetworks.commailto:mspiegel@barefootnetworks.com>
Sent: Wednesday, March 11, 2020 2:12 PM
To: p4-apps@lists.p4.orgmailto:p4-apps@lists.p4.org <p4-apps@lists.p4.orgmailto:p4-apps@lists.p4.org>
Subject: [P4-Apps] Telemetry Report 2.0 options
The new requirement from Cisco and CableLabs is the ability to coalesce multiple reports into one packet. In addition we need to add Domain Specific ID, Domain Specific Md Bits, and an extension mechanism for arbitrary domain specific information.
In order to meet these requirements, the packet format needs to be restructured, separating fields common to all reports in the packet from fields that are specific to each report in the packet.
In order to allow for multiple reports in a packet, each with its own packet fragment or flow identifying information, the report header needs to change from a small header that is followed by a packet fragment, to a report header that encapsulates a packet fragment. With this in mind, I suggest that we don't need a next protocol. All the contents are within the reports. There is nothing that needs to come after the reports.
There are two high level options for the packet structure:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| RepMdBits |Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |D|Q|F|I| Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| RepMdBits | Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Prot | Length | Reserved | Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Rsvd | Length | Domain Specific Md Status | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
* Packet Fragment
* Domain Specific Extensions
Any preferences? questions? comments? suggestions?
Mickey
The outcome of today's P4 apps meeting.
There are two high level options for the packet structure, both will be
specified in Telemetry Report v2.0:
Compact Telemetry Report v2.0 proposal
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Node id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type 1|InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| Rsvd | RepMdBits | DS Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
0: None
1: Domain Specific Extensions
2: Ethernet Packet Fragment
3: IPv4 Packet Fragment
4: IPv6 Packet Fragment
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Node id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type 2|D|Q|F|I| Report Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- ||
| Type | Rsvd | Length | RepMdBits | /\ ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ || ||
| Variable Optional Metadata | / ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- ||
| Type | Prot | Length | Reserved | /
Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ || ||
| Packet Fragment | / ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- ||
| Type | Rsvd | Length | Domain Specific Md Bits | /\ ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ || ||
| Domain Specific Extensions | / /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- --
Notes:
On Wed, Mar 11, 2020 at 2:12 PM Mickey Spiegel <
mspiegel@barefootnetworks.com> wrote:
The new requirement from Cisco and CableLabs is the ability to coalesce
multiple reports into one packet. In addition we need to add Domain
Specific ID, Domain Specific Md Bits, and an extension mechanism for
arbitrary domain specific information.
In order to meet these requirements, the packet format needs to be
restructured, separating fields common to all reports in the packet from
fields that are specific to each report in the packet.
In order to allow for multiple reports in a packet, each with its own
packet fragment or flow identifying information, the report header needs to
change from a small header that is followed by a packet fragment, to a
report header that encapsulates a packet fragment. With this in mind, I
suggest that we don't need a next protocol. All the contents are within the
reports. There is nothing that needs to come after the reports.
There are two high level options for the packet structure:
1. The more compact format uses an "InnerProtocol" field to identify
whether a packet fragment or domain specific extensions are included in the
report. In order to get to those fields, the receiver of the report has to
parse through variable optional metadata based on RepMdBits and Domain
Specific Md Bits. If it does not understand how to parse those bits, then
it cannot figure out where the packet fragment or domain specific
extensions begin. This format adds only 4 bytes, compared to the Telemetry
Report v1.0 format.
2. Nested TLV structure. An implementation that does not support
coalescing needs to generate two levels of TLVs, each with only one TLV,
with two different lengths values. This format adds 8 bytes compared to the
Telemetry Report v1.0 format.
Compact Telemetry Report v2.0 proposal
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| RepMdBits |Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
1. There is no outer "length" field, since it is largely redundant
with the UDP and IP length fields.
2. The Sequence Number has been reduced from 32 bits to 20 bits in
order to save 4 bytes. We could go with any of 16, 20, or 22 bit Sequence
Number.
3. Ingress Timestamp gets a RepMdBit value and moves into Variable
Optional Metadata.
4. Length of each report is limited to 1024 bytes.
5. RepMdBits and Domain Specific Md Bits share 28 bits. Propose 14
bits each.
6. Domain Specific Md Status has been dropped. Use Domain Specific
Extensions if necessary.
7. Inner Protocol:
0: None
1: Domain Specific Extensions
2: Ethernet Packet Fragment
3: IPv4 Packet Fragment
4: IPv6 Packet Fragment
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |D|Q|F|I| Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| RepMdBits | Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Prot | Length | Reserved | Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Rsvd | Length | Domain Specific Md Status | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
1. There is no outer "length" field, since it is largely redundant
with the UDP and IP length fields.
2. The Sequence Number has been reduced from 32 bits to 20 bits in
order to save 4 bytes. We could go with any of 16, 20, or 22 bit Sequence
Number.
3. Ingress Timestamp gets a RepMdBit value and moves into Variable
Optional Metadata.
4. Length of each report is limited to 1024 bytes. If we want to
expand that to 16k, then DQFI would have to move down, squeezing RepMdBits
and Domain Specific Md Bits to 28 bits, and either burn a separate type for
each packet fragment protocol, or move Protocol to the 16 Reserved bits on
the right.
5. Domain Specific Md Status has been moved to Domain Specific
Extensions.
6. Type (inner)
1. Packet Fragment
2. Domain Specific Extensions
Any preferences? questions? comments? suggestions?
Mickey
Hi Mickey, Et al.,
I haven’t seen any updates adding this to a PR or follow-up meetings.
Is there a plan to close on this any time soon?
Can I help? Should I add this to my PR for multiple reports per packet or will this be added to another PR?
Many Thanks and looking forward to the release of the 2.0 Telemetry Report.
Randy Levensalor
From: P4-apps p4-apps-bounces@lists.p4.org on behalf of Mickey Spiegel mspiegel@barefootnetworks.com
Date: Friday, March 13, 2020 at 12:01 PM
To: "p4-apps@lists.p4.org" p4-apps@lists.p4.org
Subject: Re: [P4-Apps] Telemetry Report 2.0 options
CableLabs WARNING: The sender of this email could not be validated and may not match the person in the "From" field.
The outcome of today's P4 apps meeting.
There are two high level options for the packet structure, both will be specified in Telemetry Report v2.0:
Compact Telemetry Report v2.0 proposal
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Node id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type 1|InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| Rsvd | RepMdBits | DS Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
0: None
1: Domain Specific Extensions
2: Ethernet Packet Fragment
3: IPv4 Packet Fragment
4: IPv6 Packet Fragment
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Node id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type 2|D|Q|F|I| Report Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- ||
| Type | Rsvd | Length | RepMdBits | /\ ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ || ||
| Variable Optional Metadata | / ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- ||
| Type | Prot | Length | Reserved | /\ Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ || ||
| Packet Fragment | / ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- ||
| Type | Rsvd | Length | Domain Specific Md Bits | /\ ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ || ||
| Domain Specific Extensions | / /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ —- --
Notes:
There is no outer "length" field, since it is largely redundant with the UDP and IP length fields.
The Sequence Number has been reduced from 32 bits to 22 bits in order to save 4 bytes.
Ingress Timestamp gets a RepMdBit value and moves into Variable Optional Metadata.
Length of each report is limited to 1024 bytes. If we want to expand that to 16k, then DQFI would have to move down, squeezing RepMdBits and Domain Specific Md Bits to 28 bits, and either burn a separate type for each packet fragment protocol, or move Protocol to the 16 Reserved bits on the right.
Domain Specific Md Status has been moved to Domain Specific Extensions.
Type (inner)
* Report Metadata
* Packet Fragment
* Domain Specific Extensions
On Wed, Mar 11, 2020 at 2:12 PM Mickey Spiegel <mspiegel@barefootnetworks.commailto:mspiegel@barefootnetworks.com> wrote:
The new requirement from Cisco and CableLabs is the ability to coalesce multiple reports into one packet. In addition we need to add Domain Specific ID, Domain Specific Md Bits, and an extension mechanism for arbitrary domain specific information.
In order to meet these requirements, the packet format needs to be restructured, separating fields common to all reports in the packet from fields that are specific to each report in the packet.
In order to allow for multiple reports in a packet, each with its own packet fragment or flow identifying information, the report header needs to change from a small header that is followed by a packet fragment, to a report header that encapsulates a packet fragment. With this in mind, I suggest that we don't need a next protocol. All the contents are within the reports. There is nothing that needs to come after the reports.
There are two high level options for the packet structure:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |InProt | Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
|D|Q|F|I| RepMdBits |Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Report
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment or Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
Nested TLV Telemetry Report v2.0 proposal:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver | hw_id | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Switch id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
| Type |D|Q|F|I| Length | Domain Specific ID | /\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| RepMdBits | Reserved | Domain Specific Md Bits | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Variable Optional Metadata | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Prot | Length | Reserved | Report
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Packet Fragment | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Type | Rsvd | Length | Domain Specific Md Status | ||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ||
| Domain Specific Extensions | /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ --
Notes:
* Packet Fragment
* Domain Specific Extensions
Any preferences? questions? comments? suggestions?
Mickey