Skip to content

[sidecar/pipeline] P4 cleanup and a return to 18 stages#208

Open
zeeshanlakhani wants to merge 3 commits intomainfrom
zl/p4-stages-18-cleanup
Open

[sidecar/pipeline] P4 cleanup and a return to 18 stages#208
zeeshanlakhani wants to merge 3 commits intomainfrom
zl/p4-stages-18-cleanup

Conversation

@zeeshanlakhani
Copy link
Contributor

@zeeshanlakhani zeeshanlakhani commented Feb 3, 2026

P4/sidecar pipeline changes:

  • Unify route_result_t struct for Router4/Router6 to prevent PHV liverange divergence
    • this came up in checking logs for stage + bit overlap
  • Unify nexthop: replace separate nexthop_ipv4/nexthop_ipv6 with single nexthop (ipv6_addr_t) plus nexthop_is_v6 flag
  • Add TTL compound key (idx, route_ttl_is_1) to route tables for TTL offload
    • This includes Rust updates and a larger routing table for the compound key
  • Add @pa_no_init pragmas for metadata fields to guard against compiler init bugs, even though we've updated/removed the compiler pass in p4c
    • Add @pa_container_type("normal") pragmas for all ingress booleans and wider metadata fields to prevent mocha container corruption (whole-container writes clobbering neighboring fields)
    • Extend @pa_container_type to egress bridge header fields and drop_reason
  • Zero sc_pad at every sc_code write site (was uninitialized before)
  • Bridge header now includes is_mcast_routed for CPU copy detection

EgressFilter:

  • Move EgressFilter from ingress to egress pipeline, as NatEgressFilter ensuring one less ingress stage
  • Add nat_egress_hit to bridge header to carry NAT state across TM boundary
  • CP: update table path and match key (egress_port instead of ucast_egress_port)

Egress MacRewrite:

  • Create separate unicast_mac_rewrite and mcast_mac_rewrite instances of MacRewrite
  • Unicast instance: rewrites src_mac only (dst_mac already correct from ARP/NDP)
  • Multicast instance: derives dst_mac from group address per RFC 1112/2464

Multicast Egress changes:

  • Check egress_rid != 0 first to identify PRE-replicated packets
  • Check is_mcast_routed for CPU copies (egress_rid == 0 but routed to multicast)
  • Drop CPU copies with DROP_MULTICAST_CPU_COPY reason

Counters:

  • Moved Unicast, MulticastLL, EgressDropPort, EgressDropReason counters from multicast to base
  • Added a Forwarded counter (for every packet copy that egresses the pipeline)
  • Removed Egress counter (from ingress pipe, replaced by per-port counters)
  • Removed MulticastDrop from MULTICAST_COUNTERS, as it's covered in general drop w/ reason
  • Handle link-local multicast counting in both MULTICAST and non-MULTICAST paths

softnpu:

  • Route table updates for IPv4/IPv6 TTL handling compatibility
  • Skip route_ttl_is_1 entries and ttl_exceeded actions (sidecar-lite TODO)

Build:

  • Configure TOFINO_STAGES: 15 for base, 18 for multicast

@zeeshanlakhani zeeshanlakhani force-pushed the zl/p4-stages-18-cleanup branch from fea3562 to ed6d7d9 Compare February 3, 2026 21:36
P4/sidecar pipeline changes:
- Unify route_result_t struct for Router4/Router6 to prevent PHV liverange divergence
  - this came up in checking logs for stage + bit overlap
- Unify nexthop: replace separate nexthop_ipv4/nexthop_ipv6 with single nexthop
  (ipv6_addr_t) plus nexthop_is_v6 flag
- Add TTL compound key (idx, route_ttl_is_1) to route tables for TTL offload
  - This includes Rust updates and a larger routing table for the compound key
- Add @pa_no_init pragmas for metadata fields to guard against compiler init bugs,
  even though we've updated/removed the compiler pass in p4c
- Bridge header now includes is_mcast_routed for CPU copy detection

Egress MacRewrite:
- Create separate unicast_mac_rewrite and mcast_mac_rewrite instances of MacRewrite
- Unicast instance: rewrites src_mac only (dst_mac already correct from ARP/NDP)
- Multicast instance: derives dst_mac from group address per RFC 1112/2464

Multicast Egress changes:
- Check egress_rid \!= 0 first to identify PRE-replicated packets
- Check is_mcast_routed for CPU copies (egress_rid == 0 but routed to multicast)
- Drop CPU copies with DROP_MULTICAST_CPU_COPY reason

Counters:
- Moved Unicast, MulticastLL, EgressDropPort, EgressDropReason counters from multicast to base
- Added a Forwarded counter (for every packet copy that egresses the pipeline)
- Removed Egress counter (from ingress pipe, replaced by per-port counters)
- Removed MulticastDrop from MULTICAST_COUNTERS, as it's covered in general drop w/ reason
- Handle link-local multicast counting in both MULTICAST and non-MULTICAST paths

softnpu:
- Route table updates for IPv4/IPv6 TTL handling compatibility
- Skip route_ttl_is_1 entries and ttl_exceeded actions (sidecar-lite TODO)

Build:
- Configure TOFINO_STAGES: 15 for base, 18 for multicast
This commit forces metadata fields out of mocha containers via @pa_container_type pragmas.
Mocha containers lack ALUs and only support whole-container set operations. When a field
is the only one written by an action in a mocha container, the whole-container write
clobbers other fields sharing that container. The ipv4_checksum_err field was packed into
mocha MH0 alongside pkt_type, which could cause issues.

This adds similar protections for NAT/encap fields, egress bridge header fields, and
egress action fields in both multicast and non-multicast builds.

Other includes:

- Zero-inits sidecar header pad and unused fields (sc_pad, sc_egress,
  sc_payload) across all sidecar header construction sites to prevent stale
  PHV data from leaking into the header.

- Removes unreachable ternary entry (0, 0, _, _, _) from the multicast
  port_bitmap_check const entries table. The two preceding wildcard entries
  already cover that case.

- Simplifies multicast egress counting to use the mcast_tag local captured
  before egress decap rather than re-checking header validity.
@zeeshanlakhani zeeshanlakhani self-assigned this Feb 9, 2026
@zeeshanlakhani zeeshanlakhani force-pushed the zl/p4-stages-18-cleanup branch from 0d3cbfc to 4367608 Compare February 9, 2026 21:08
@zeeshanlakhani zeeshanlakhani marked this pull request as ready for review February 10, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant