Skip to content

fix(netprobe): count a ping attempt only when the probe sends it - #811

Merged
leoparente merged 1 commit into
developfrom
fix/netprobe-attempts
Nov 16, 2026
Merged

leoparente merged 1 commit into
developfrom
fix/netprobe-attempts

Conversation

@leoparente

Copy link
Copy Markdown
Contributor

A netprobe ping policy that probes one target every 5 s reported 24 attempts per 60 s window instead of 12. The handler counted echo requests it received as attempts.

Cause

The process-wide IPv4 ping receiver (src/inputs/netprobe/PingProbe.cpp) is a raw ICMP socket with no type filter. On Linux it gets every ICMP packet the host receives, and it passes each one to every ping probe. NetProbeStreamHandler::process_netprobe_icmp decided by message type alone: any ICMP_ECHO_REQUEST started a transaction and counted an attempt, whether the probe had sent it or received it.

  • A local target counts twice. A ping to 127.0.0.1, or to any address the host owns, comes back through lo as its own request before the reply. Each probe counted two attempts and one success. The looped-back request also restarted the transaction, so the round trip was timed from it rather than from the send.
  • Other hosts' pings count too. Every echo request another host sends to the agent host added an attempt to every ping target of every netprobe policy, which reads as packet loss on the dashboards.
  • Other targets get credit. Every probe on the pktvisor, in any policy, also handled the looped-back request and its reply under its own target name. That re-tagged the pending transaction, so other targets were credited with attempts and successes they never made, or took the local target's success and round trip.

The ICMPv6 receiver already passes only echo replies (ICMP6_FILTER and build_icmpv6_reply_carrier), so IPv6 targets were not affected. macOS doesn't deliver echo requests to ICMP sockets, so it doesn't show the problem either.

Measured on Linux before the fix

pktvisord from netboxlabs/orb-agent:develop (pktvisor 4.5.0-develop-c3c5379), one container per case, exporting OTLP to a collector. The tap and policy were the ones orb-agent creates for a Fleet netprobe policy: test_type: ping, interval_msec: 5000, one target.

Case Target attempts per window successes per window
A 127.0.0.1 24 12
B another container 12 12
C as B, while a neighbour runs ping -i 2 at this host 42 12

Each window was pushed once over OTLP, as a gauge with the same values as the REST API. Nothing downstream doubles it.

The fix

probe_signal_recv now skips received ICMPv4 echo requests. It skips ICMPv6 echo requests too, as a defensive check. Attempts are counted only on the send path, as the TCP path already does; replies end transactions as before; a local target's round trip is timed from the send again.

Tests (src/handlers/netprobe/test_net_probe.cpp)

They drive the input proxy's send and receive signals directly, so no sockets or root are needed.

Test Develop's handler This branch
A local ping is one attempt (send, looped-back request, reply) 2 attempts, round trip 11000 µs 1 attempt, 1 success, 12000 µs
A foreign request with the probe's own id and sequence, plus an ICMPv6 request 3 attempts, 7000 µs 1 attempt, 12000 µs
Targets local and remote in one policy, packets fanned out to both local has 2 attempts and 0 successes; remote gets metrics local has 1, 1 and 12000 µs; remote has none

Verification

  • The netprobe handler suite: 34 cases, 32 passed, 2 skipped (they need raw sockets or the network).
  • ctest on macOS: 24/25. unit-tests-input-netprobe fails the same way on develop: two TCP-timeout tests, which never reach the handler, expect connects to stall, and this host completes them. Linux CI is the reference for that target.
  • Not run end to end on Linux with the fix. The handler tests replay the packet sequences measured above.

Not changed

  • No ICMP type filter on the IPv4 socket. An ICMP_FILTER would save copying each foreign echo request to every probe, but correctness doesn't need it.
  • packets_timeout counts late replies only. A ping that never gets a reply shows only as attempts minus successes, which is how the dashboards already compute availability.

🤖 Generated with Claude Code

The shared ping receiver passes every ICMPv4 packet the host receives to
each ping probe, and the handler counted any echo request among them as
an attempt. A ping to a local address comes back as its own request
before the reply, so each probe counted two attempts and one success,
and the looped-back request restarted the round-trip timer. Echo
requests from a host pinging this one were counted as attempts on every
ping target.

Every ping probe on the pktvisor, in any policy, also handled the
looped-back request and its reply under its own target name, which
re-tagged the pending transaction: other targets were credited with
attempts and successes they never made, or took the local target's
success and round trip.

The IPv4 receive path now skips echo requests. The ICMPv6 receiver
already passes only echo replies, so IPv6 targets were not affected; the
handler skips ICMPv6 echo requests too, as a defensive check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leoparente

Copy link
Copy Markdown
Contributor Author

@codex review

@leoparente

Copy link
Copy Markdown
Contributor Author

/ai-review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T18:11:40.975266Z 4109b9a Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@nbl-ai-review

nbl-ai-review Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

AI Code Review — risk tier: Lite · dimensions run: correctness, security

Advisory only. A human owns the merge decision.

Summary — Net positive. The fix correctly restricts attempt-counting to the send path for ICMPv4/ICMPv6 echo requests, eliminating the double-counting of looped-back and foreign echo requests described in the PR. Both sub-reviewers traced the change through process_netprobe_icmp/icmpv6 and the transaction manager and found no issues; the new tests exercise the looped-back, foreign-request, and multi-target fan-out cases the description claims to fix.

🤖 AI Code Review · run · prompts: code-review/v1 · models: claude-sonnet-5/medium · context: description · 3 comment(s) · 0 prior AI review(s) · usage: $0.9 · 1693k in (96% cached) · 6.9k out

@nbl-ai-review

nbl-ai-review Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

AI Code Review — risk tier: Lite · dimensions run: correctness, security

Advisory only. A human owns the merge decision.

Summary — Net-positive, well-scoped fix: ping attempts are now only counted when the probe actually sends an ICMP echo request, correctly excluding received echo requests from the attempt count. No correctness or security issues found.

🤖 AI Code Review · run · prompts: code-review/v1 · models: claude-sonnet-5/medium · context: description · 3 comment(s) · 0 prior AI review(s) · usage: $0.56 · 1105k in (96% cached) · 3.2k out

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 4109b9a113

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@leoparente
leoparente marked this pull request as ready for review November 16, 2026 18:43
@leoparente leoparente self-assigned this Nov 16, 2026
@github-actions

Copy link
Copy Markdown

LCOV of commit 4109b9a during Debug Builds #283

Summary coverage rate:
  source files: 148
  lines.......: 86.9% (19314 of 22231 lines)
  functions...: 80.3% (2038 of 2538 functions)
  branches....: no data found
Message summary:
  no messages were reported

Files changed coverage rate: n/a

Full coverage report

@leoparente
leoparente merged commit 25ffa3f into develop Nov 16, 2026
23 checks passed
@leoparente
leoparente deleted the fix/netprobe-attempts branch November 16, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants