Skip to content

Idex l2b agnostic - #3410

Open
aldo9253 wants to merge 11 commits into
IMAP-Science-Operations-Center:devfrom
aldo9253:idex_l2b_agnostic
Open

Idex l2b agnostic#3410
aldo9253 wants to merge 11 commits into
IMAP-Science-Operations-Center:devfrom
aldo9253:idex_l2b_agnostic

Conversation

@aldo9253

@aldo9253 aldo9253 commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Change Summary

Closes #3407

Overview

Adds mass- and charge-agnostic IDEX L2B/L2C products for improved counting statistics when event rates are low.

The new products:

  • Count each event once, including only dust_hit_flag == 1.
  • Preserve the existing daily-bin L2B/L2C structure.
  • Provide count and rate products by spin phase and sky position.
  • Exclude invalid geometry only from spatial maps.
  • Calculate rates using daily science-acquisition uptime in s^-1.
  • Preserve computed mass/charge products internally but publish them as fill/NaN pending validation.

File changes

  • imap_processing/idex/idex_l2b.py

    • Adds agnostic count/rate products and maps.
    • Filters events using dust_hit_flag.
    • Withholds mass/charge products at the end of processing.
    • Couples spin histograms to the configured bin definition.
  • imap_processing/cdf/config/imap_idex_l2b_variable_attrs.yaml

    • Adds metadata for agnostic count/rate variables.
  • imap_processing/cdf/config/imap_idex_l2c_variable_attrs.yaml

    • Adds metadata for agnostic spatial maps and image axes.
  • imap_processing/tests/idex/test_idex_l2b.py

    • Tests dust filtering and missing-flag behavior.
    • Verifies mass/charge products are masked while agnostic products remain populated.

Testing

  • Focused IDEX L2B tests: 14 passed.
  • Local L2B/L2C processing completed using the July 19, 2026 products.
  • Generated agnostic L2B and L2C totals agree: 6 events.
  • _by_mass/charge products are serialized as fill values.
  • Python compilation, YAML parsing, and diff checks passed.

@aldo9253
aldo9253 marked this pull request as ready for review August 31, 2026 22:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There is a documented-vs-implemented units mismatch in the PR description (rates are computed in s^-1) and a small maintainability issue in the new agnostic histogram binning.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds new mass-/charge-agnostic IDEX L2B/L2C products (counts/rates by spin phase and by sky position) using dust_hit_flag == 1, alongside metadata updates and a regression test.

Changes:

  • Add agnostic counts/rate (L2B) and counts_map/rate_map (L2C) outputs derived from dust-hit filtering.
  • Update IDEX L2B/L2C CDF variable-attribute YAML to include new variables and adjust rate metadata.
  • Add a regression test ensuring non-dust events are excluded from the new agnostic products.
File summaries
File Description
imap_processing/idex/idex_l2b.py Adds agnostic count/rate computations and wires them into L2B/L2C outputs.
imap_processing/cdf/config/imap_idex_l2b_variable_attrs.yaml Adds metadata for new L2B agnostic variables and updates rate metadata.
imap_processing/cdf/config/imap_idex_l2c_variable_attrs.yaml Adds metadata for new L2C agnostic map variables and updates rate metadata.
imap_processing/tests/idex/test_idex_l2b.py Adds a regression test for dust-hit filtering and updates expected L2B variables.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread imap_processing/idex/idex_l2b.py Outdated
Comment thread imap_processing/cdf/config/imap_idex_l2b_variable_attrs.yaml

@lacoak21 lacoak21 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Had a few comments. Im going to keep this blocked until i get them resolved.

Comment thread imap_processing/cdf/config/imap_idex_l2b_variable_attrs.yaml
Comment thread imap_processing/idex/idex_l2b.py
Comment thread imap_processing/idex/idex_l2b.py Outdated
Comment thread imap_processing/idex/idex_l2b.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Published rates can contain invalid negative values and incorrect quality flags for non-positive uptime.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

imap_processing/cdf/config/imap_idex_l2c_variable_attrs.yaml:148

  • This DICT_KEY omits the Support> hierarchy segment and is inconsistent with the matching L2B agnostic rate metadata at imap_idex_l2b_variable_attrs.yaml:162. Classify this map as the same dust count-rate quantity so L2B and L2C resolve to the same valid SPASE metadata path.
  DICT_KEY: SPASE>SupportQuantity:CountRate,Qualifier:Array
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment on lines +160 to +162
rate, rate_map = compute_rates_agnostic(
counts, counts_map, epoch_doy_unique, daily_on_percentage
)
Comment thread imap_processing/idex/idex_l2b.py Outdated
Comment on lines +646 to +647
rate = np.full(counts.shape, -1.0)
rate_map = np.full(counts_map.shape, -1.0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

IDEX L2B Create mass/charge agnostic count and rate bins

3 participants