Skip to content

feat(experimentation): targeted audience for experiment rollouts - #8472

Merged
Zaimwa9 merged 6 commits into
mainfrom
feat/experiment-targeted-audience-api
Sep 8, 2026
Merged

feat(experimentation): targeted audience for experiment rollouts#8472
Zaimwa9 merged 6 commits into
mainfrom
feat/experiment-targeted-audience-api

Conversation

@Zaimwa9

@Zaimwa9 Zaimwa9 commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Experiment rollouts can now target an audience: Segment > % rollout > % variant weights.

  • Rollout create/PATCH accepts an optional audience: {"match": "any"|"all", "segment_ids": [...]} (omitted = unchanged).
  • The selected segment's rules are compiled as a frozen copy into the existing experiment-{id}-rollout system segment, so later segment edits don't affect a running experiment. No engine, SDK, or environment-document schema changes.
  • Snapshot stored on a new Experiment.audience JSONField (migration 0013); the audience is immutable once the experiment starts.
  • Validation rejects ineligible segments (foreign project/system/feature-specific, wrong-env or deleting cohorts, PERCENTAGE_SPLIT rules, no rules, duplicates, over the cap).
  • Cohort deletion is blocked while an experiment snapshot targets it.
  • AuditLog + experimentation.rollout.applied structlog event on rollout changes.

MAX_AUDIENCE_SEGMENTS is 1 for now. The multi-segment machinery is implemented and tested — the any/all combinator, the per-segment rule wrappers, the compile and enrolment paths — but the cap holds it to a single segment until the Java SDK is fixed:

Warning

A multi-segment any audience compiles to an ANY rule with nested sub-rules. The released Java SDK (flagsmith-java-client ≤ 8.1.0) ANDs sub-rules regardless of the rule type in local evaluation, so such an audience would evaluate as all and under-fill the experiment — silently, with no SRM signal. Raising the cap needs the Java engine to respect the rule type for sub-rules first.

Raising the cap also grows the environment document, since each targeted segment's rules are copied into the rollout segment (cohort segments compile to a single system-trait condition). A source segment with one top-level rule is inlined rather than wrapped, keeping the common case within the document builder's prefetch depth and the org exporter's two-level rule selection.

The dashboard UI comes in a stacked follow-up PR.

How did you test this code?

Unit and integration tests, 100% diff coverage. Multi-segment compilation and enrolment stay covered via a fixture that lifts the cap, so the behaviour is tested ahead of the cap being raised. A query-count regression test pins the environment-document cost at a fixed number of prefetches for a nested audience.

Manually on a local stack: created a segment + seeded identities, started an experiment targeting it at 50% rollout with a 50/50 split — ~50% of in-segment identities enrolled, none out-of-segment. Edited the source segment mid-run: enrolment unchanged (frozen copy). PATCHing a different audience after start returns 400.

@vercel

vercel Bot commented Sep 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 8, 2026 1:07pm UTC
2 Skipped Deployments
Project Deployment Actions Updated
flagsmith-frontend-preview Ignored Ignored Preview Sep 8, 2026 1:07pm UTC
flagsmith-frontend-staging Ignored Ignored Preview Sep 8, 2026 1:07pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 37d7547b-7272-42fe-a0fe-70f1ec769f1d

📥 Commits

Reviewing files that changed from the base of the PR and between 18c11ba and ca47ac3.

📒 Files selected for processing (4)
  • api/audit/constants.py
  • api/cohorts/services.py
  • api/experimentation/services.py
  • docs/docs/deployment-self-hosting/observability/_events-catalogue.md

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

Experiment rollouts now accept audience segments with any or all matching. The service validates, snapshots, compiles, persists, and returns audience rules and metadata. Nested rule prefetching supports the compiled trees. Audit and telemetry events include audience data. Cohort deletion is blocked while a non-completed experiment targets the cohort. OpenAPI schemas and tests cover the new behaviour.

Priority: ➖ Normal — Schedule the targeted experiment-audience change because it spans rollout APIs, audience rule compilation, persistence, validation, and cohort deletion safeguards across experimentation.

Estimated code review effort: 4 (Complex) | ~60 minutes

Merge Risk: 🔵 Low · up to ca47a

Audience targeting adds compiled nested segment rules to experiment rollouts. Deeply nested rules may cause elevated query volume when rollout configuration is loaded, so this should be confirmed before high-volume use.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added api Issue related to the REST API docs Documentation updates labels Sep 7, 2026
@Zaimwa9
Zaimwa9 force-pushed the feat/experiment-targeted-audience-api branch from 4f33cff to 8faabc4 Compare September 7, 2026 08:49
@github-actions github-actions Bot added feature New feature or request and removed docs Documentation updates labels Sep 7, 2026
@github-actions github-actions Bot added the docs Documentation updates label Sep 7, 2026
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.82%. Comparing base (62b5309) to head (ca47ac3).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main    #8472    +/-   ##
========================================
  Coverage   98.81%   98.82%            
========================================
  Files        1621     1622     +1     
  Lines       66299    66708   +409     
========================================
+ Hits        65514    65923   +409     
  Misses        785      785            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Zaimwa9
Zaimwa9 force-pushed the feat/experiment-targeted-audience-api branch from 182c216 to aa61c41 Compare September 7, 2026 08:59
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Sep 7, 2026
@github-actions github-actions Bot added the docs Documentation updates label Sep 7, 2026
@Zaimwa9
Zaimwa9 force-pushed the feat/experiment-targeted-audience-api branch from 4a2ec7c to 77234df Compare September 7, 2026 09:15
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Sep 7, 2026
@github-actions github-actions Bot added the docs Documentation updates label Sep 7, 2026
@Zaimwa9
Zaimwa9 force-pushed the feat/experiment-targeted-audience-api branch from 9695d3b to 79f40a9 Compare September 7, 2026 09:59
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Sep 7, 2026
@github-actions github-actions Bot added the docs Documentation updates label Sep 7, 2026
@Zaimwa9
Zaimwa9 force-pushed the feat/experiment-targeted-audience-api branch from 5932fc6 to ed13898 Compare September 7, 2026 10:24
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Sep 7, 2026
@Zaimwa9
Zaimwa9 force-pushed the feat/experiment-targeted-audience-api branch from f39e6d5 to d0bd8bc Compare September 7, 2026 15:40
@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Sep 7, 2026
@github-actions github-actions Bot added the docs Documentation updates label Sep 7, 2026
@Zaimwa9
Zaimwa9 removed the request for review from emyller September 8, 2026 07:58
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Sep 8, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
api/projects/services.py (1)

29-35: 🚀 Performance & Scalability | 🟡 Minor | ⚡ Quick win

Prefetch deepest-level conditions in all rule-loading paths

_compile_audience() can produce rules at the fifth nested level, and map_segment_rule_to_engine() recursively reads rule.conditions.all(). Add rules__rules__rules__rules__rules__conditions to the prefetch lists in api/projects/services.py, EnvironmentManager.filter_for_document_builder(), and Environment.get_segments_from_cache(). Otherwise, environment-document generation can issue one lazy query per deepest rule.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: b3fc7fac-2dfc-4463-a434-f4fac3d7479d

📥 Commits

Reviewing files that changed from the base of the PR and between aff6c8b and 18c11ba.

📒 Files selected for processing (9)
  • api/cohorts/services.py
  • api/experimentation/dataclasses.py
  • api/experimentation/migrations/0013_add_experiment_audience.py
  • api/experimentation/models.py
  • api/experimentation/services.py
  • api/tests/unit/cohorts/test_services.py
  • api/tests/unit/experimentation/test_experiment_views.py
  • api/tests/unit/experimentation/test_services.py
  • docs/docs/deployment-self-hosting/observability/_events-catalogue.md
💤 Files with no reviewable changes (1)
  • api/experimentation/dataclasses.py

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.

@gagantrivedi gagantrivedi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor changes requested

Comment thread api/cohorts/services.py Outdated
Comment thread api/experimentation/services.py Outdated
@github-actions github-actions Bot removed feature New feature or request docs Documentation updates labels Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API docs Documentation updates feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants