Skip to content

[pfcwd] Add unit tests for hardware PFC watchdog - #4418

Open
arawat-nexthop wants to merge 9 commits into
sonic-net:masterfrom
nexthop-ai:hw-pfcwd-unit-tests
Open

arawat-nexthop wants to merge 9 commits into
sonic-net:masterfrom
nexthop-ai:hw-pfcwd-unit-tests

Conversation

@arawat-nexthop

@arawat-nexthop arawat-nexthop commented Mar 31, 2026 •

Copy link
Copy Markdown
Contributor

What I did

Add comprehensive unit test coverage for hardware-based PFC (Priority Flow Control) watchdog functionality. This PR includes two sets of tests:

1. PfcWdHwOrch orchestration tests (30 test cases):

  • Configuration validation: Invalid ports, time ranges, actions, and parameters
  • Action management: Global action setting, consistency enforcement, reset behavior
  • Port lifecycle: Reconfiguration, STATE_DB lifecycle, warm reboot recovery
  • Hardware configuration: Detection/restoration intervals, DLDR enable/disable
  • Deadlock handling: Callback registration, detection/restoration flow, statistics, edge cases
  • SAI failure handling: Rollback on switch action, interval, and queue DLDR failures
  • SKU detection: Hardware PFC watchdog SKU validation

2. PfcWdHwOrch counter management tests:

  • Storm detection and counter initialization
  • Periodic counter updates and delta calculation
  • Storm restoration and counter cleanup
  • Counter underflow protection
  • Hardware counter reading from COUNTERS_DB
  • Queue statistics persistence

Why I did it

Prior to this change, there was no unit test coverage for the hardware-based PFC watchdog component. These tests enable verification of correct behavior, catch regressions, and validate error handling.

How to verify it

cd sonic-swss/tests/mock_tests
make tests
./tests --gtest_filter="PfcWdHwOrchTest*"

Which release branch to backport (optional)

  • 202505
  • 202511

Dependencies

This PR is based on PR #4412 - [Hw PFCWD feature] Implementation for H/w based PFCWD

Once PR #4412 is merged, this PR will show only the 2 unit test commits.

Related PRs:

  • sonic-swss-common: Hardware state table schema definition
  • sonic-utilities: Hardware PFCWD CLI support
  • sonic-mgmt: Hardware-aware test suite

Signed-off-by: Abhishek <abhishek@nexthop.ai>
… functions.

Signed-off-by: Abhishek <abhishek@nexthop.ai>
Signed-off-by: Abhishek <abhishek@nexthop.ai>
Signed-off-by: Abhishek <abhishek@nexthop.ai>
Signed-off-by: Abhishek <abhishek@nexthop.ai>
Signed-off-by: Abhishek <abhishek@nexthop.ai>
…onality.

Signed-off-by: Abhishek <abhishek@nexthop.ai>
@arawat-nexthop
arawat-nexthop requested a review from prsunny as a code owner March 31, 2026 13:37
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

What I did:

Add comprehensive unit test coverage for the hardware-based PFC (Priority Flow Control) watchdog orchestration agent (PfcWdHwOrch). Prior to this change, there was no unit test coverage for this critical component.

Changes include:

1. **Test infrastructure**:
   - Created PfcWdHwOrchTest fixture with SAI API mocks for switch, port, and queue operations
   - Set platform environment variable required by PfcWdBaseOrch constructor
   - Implemented helper functions: bringUpPorts(), enablePfcOnPort(), deleteHwWdEntry()

2. **30 unit tests covering**:
   - Configuration validation (7 tests): Invalid ports, time ranges, actions, and parameters
   - Action management (4 tests): Global action setting, consistency enforcement, reset behavior
   - Port lifecycle (3 tests): Reconfiguration, STATE_DB lifecycle, warm reboot recovery
   - Hardware configuration (3 tests): Detection/restoration intervals, DLDR enable/disable
   - Deadlock handling (6 tests): Callback registration, detection/restoration flow, statistics, edge cases
   - SAI failure handling (4 tests): Rollback on switch action, interval, and queue DLDR failures
   - SKU detection (1 test): Hardware PFC watchdog SKU validation

Why I did it:

Enable verification of correct behavior, catch regressions, and validate error handling for hardware-based PFC watchdog.

How to verify it:

```bash
cd sonic-swss/tests/mock_tests
make tests
./tests --gtest_filter="PfcWdHwOrchTest*"
```

Which release branch to backport (optional):

- [ ] 202505
- [ ] 202511

Signed-off-by: Abhishek Rawat <arawat@nexthop.ai>
What I did:

Add comprehensive unit tests for the hardware-based PFC watchdog (PFCWD) counter management logic in PfcWdHwOrch.

Test coverage includes:
- Storm detection and counter initialization (initQueueCounters())
- Periodic counter updates and delta calculation (updateQueueCounters() periodic path)
- Storm restoration and counter cleanup (updateQueueCounters() restoration path)
- Counter underflow protection
- Hardware counter reading from COUNTERS_DB (readHwCounters())
- Queue statistics persistence (getQueueStats()/updateQueueStats())

Implementation details:
- Created pfcwdhw_counters_ut.cpp with comprehensive test cases
- Updated Makefile.am to include the new test in the build
- Follows existing test patterns from flexcounter_ut.cpp
- Uses MockOrchTest base fixture for DB infrastructure

Why I did it:

Ensure correct behavior of hardware counter tracking, delta calculation, and statistics management for hardware-based PFC watchdog.

How to verify it:

```bash
cd sonic-swss/tests/mock_tests
make tests
./tests --gtest_filter="PfcWdHwOrchTest*"
```

Which release branch to backport (optional):

- [ ] 202505
- [ ] 202511

Signed-off-by: Abhishek Rawat <arawat@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

arawat-nexthop added a commit to nexthop-ai/sonic-swss that referenced this pull request Sep 1, 2026
Cover PfcWdHwOrch's configuration and notification handling: entry validation,
the first-port action and the action reset on the last delete, reconfiguring a
port, the STATE_DB lifecycle and the recovery mechanism it publishes, warm
reboot recovery, the detection and restoration intervals, DLDR enable on the
lossless queues, the queue map, protection of a port while it is stormed, and
the SAI failure paths for switch action, DLD and DLR intervals and queue DLDR.

Add DlrDldrCapabilityCombinations for the four {DLR init, DLDR} capability
combinations: the hardware watchdog is selected on DLDR alone, and a platform
reporting both must still choose hardware rather than start a software
handler. Requested in review.

The tests come from the branch of PR sonic-net#4418. Only the additions are taken, on
top of the current file, so that the tests added to portsorch_ut.cpp since
that branch was cut are kept. The counter tests from that branch belong with
the counter change and are not included here.

Signed-off-by: arawat-nexthop <arawat@nexthop.ai>
arawat-nexthop added a commit to nexthop-ai/sonic-swss that referenced this pull request Sep 1, 2026
Adds the counter side of PfcWdHwOrch on top of the orchestration in sonic-net#4412:

- readHwCounters() reads a queue's hardware counters
- initQueueCounters() takes the baseline when the hardware reports a storm
- updateQueueCounters() updates on recovery and on the periodic poll
- getQueueStats() / updateQueueStats() for the COUNTERS_DB round trip
- initializeQueueStats() registers a port's lossless queues
- the dedicated PFC_WD FlexCounter group and its per-queue registration
- the poll timer and doTask()
- SAI_QUEUE_STAT_DROPPED_PACKETS and SAI_INGRESS_PRIORITY_GROUP_STAT_PACKETS

With the counters present, isPortInStormedState() reads a queue's operational
state from COUNTERS_DB again rather than from the set the orchestration change
tracks in memory.

Tests: pfcwdhw_counters_ut.cpp covers the baseline, recovery and periodic
paths and the COUNTERS_DB round trip; portsorch_ut.cpp adds the callback and
deadlock lifecycle cases, accumulation across events, an unknown queue, and
statistics preserved across a reconfigure.

The tests come from the branch of PR sonic-net#4418.

Signed-off-by: arawat-nexthop <arawat@nexthop.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants