You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A full reporting run against a native catalog without the optional feature returns an explicit coverage_gaps[].reason: fixture_unsatisfied with zero authored reporting steps. The SDK's raw overall_passed boolean can still be true because no executed assertion failed. Addie's existing history writer stores that boolean and counts without a separate incomplete-coverage state; the manual matrix's clean count similarly means no failed rows, not tested feature support.
#8336 fixes Addie's visible report to show Incomplete — coverage gap and Not applicable for this run, while preserving actual SDK results, existing history fields and floor accounting. Training CLI now prints the real coverage-gap detail. No missing native fixture is treated as proof of nonsupport or as an executed reporting pass.
Follow-up: give hosted run history/aggregate presentation a reviewed representation of incomplete fixture coverage, retaining original failure evidence and distinguishing a completed no-match from discovery/schema/protocol errors. Preserve immutable historical records; do not backfill invented support, globally skip mandatory failures or silently lower grading floors. A failed selected implementation must stay failed even when another scope is unexercised.
This host accounting/presentation follow-up is separate from catalog selection and the per-invocation reporting adapter; no evaluator/registry policy implementation or release is assigned by this issue.
Activity