Skip to content

Idex release two updates #3399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tmplummer merged 17 commits into from
Aug 28, 2026
Merged

Idex release two updates #3399

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
2f10bf8
Add IDEX event classification flags
aldo9253 Aug 18, 2026
3a2b3af
Added saturation flags for all 6 waveform channels. Added flags to in…
aldo9253 Aug 19, 2026
e761beb
Merge remote-tracking branch 'origin/dev' into idex_release_two_updates
aldo9253 Aug 19, 2026
204e3c9
fixed expected nan condition in l2a for ion grid values
aldo9253 Aug 19, 2026
76a4275
added event category masking to l2b/c, left NaN block in place.
aldo9253 Aug 19, 2026
fec1ed1
fixed description in idex_event_flags.py
aldo9253 Aug 20, 2026
118d907
addressed round of comments
aldo9253 Aug 24, 2026
69b9c08
addressed round of comments
aldo9253 Aug 24, 2026
1c80d14
added log and mandatory state for saturation flags
aldo9253 Aug 24, 2026
8566c76
re-run checks
aldo9253 Aug 24, 2026
0f16d72
fixed tests
aldo9253 Aug 24, 2026
74db977
corrected fill values
aldo9253 Aug 25, 2026
c433242
addressed data type issue
aldo9253 Aug 25, 2026
6674c4c
reverted l2b changes for cleaner merging
aldo9253 Aug 25, 2026
06cc24d
Merge upstream dev into idex_release_two_updates
aldo9253 Aug 26, 2026
eeff14b
Use upstream mA conversions for IDEX TOF channels
aldo9253 Aug 26, 2026
0165f7f
pre-commit fix
aldo9253 Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions imap_processing/cdf/config/imap_idex_l1a_variable_attrs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,20 @@ trigger_base: &trigger_base
VALIDMIN: 0
VAR_TYPE: support_data

event_flag_base: &event_flag_base

@lacoak21 lacoak21 Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see. There are a couple of issues here.

I would not use "trigger_base" as the base because those are for variables that are not quality flags. So I would remove that part.

Second, make sure the array is actually np.unit8 e.g. data.astype(np.uint8)
It thinks the data is np.uint64 as implied by the warning.

CATDESC: "Boolean event classification flag."
CDF_DATA_TYPE: CDF_UINT1
DEPEND_0: epoch
DISPLAY_TYPE: no_plot
FIELDNAM: ""
FILLVAL: 255
FORMAT: I1
LABLAXIS: ""
UNITS: " "
VALIDMAX: 1
VALIDMIN: 0
VAR_TYPE: support_data

# <=== LABL_PTR_i Attributes ===>
time_high_sample_rate_label:
CATDESC: High sample rate time steps for a dust event.
Expand All @@ -91,6 +105,66 @@ time_low_sample_rate_label:
VAR_TYPE: metadata

# <=== Instrument Setting Attributes ===>
science_event_flag:
<<: *event_flag_base
CATDESC: Science event classification flag.
FIELDNAM: Science Event Flag
LABLAXIS: Science Event

noise_capture_flag:
<<: *event_flag_base
CATDESC: Noise capture event classification flag.
FIELDNAM: Noise Capture Flag
LABLAXIS: Noise Capture

pulser_flag:
<<: *event_flag_base
CATDESC: Pulser event classification flag.
FIELDNAM: Pulser Flag
LABLAXIS: Pulser

dust_hit_flag:
<<: *event_flag_base
CATDESC: Dust Hit waveform classification flag.
FIELDNAM: Dust Hit Flag
LABLAXIS: Dust Hit

tof_high_saturation_flag:
<<: *event_flag_base
CATDESC: TOF High waveform saturation flag.
FIELDNAM: TOF High Saturation Flag
LABLAXIS: TOF HG Saturation

tof_mid_saturation_flag:
<<: *event_flag_base
CATDESC: TOF Mid waveform saturation flag.
FIELDNAM: TOF Mid Saturation Flag
LABLAXIS: TOF MG Saturation

tof_low_saturation_flag:
<<: *event_flag_base
CATDESC: TOF Low waveform saturation flag.
FIELDNAM: TOF Low Saturation Flag
LABLAXIS: TOF LG Saturation

target_high_saturation_flag:
<<: *event_flag_base
CATDESC: Target High waveform saturation flag.
FIELDNAM: Target High Saturation Flag
LABLAXIS: Target HG Saturation

target_low_saturation_flag:
<<: *event_flag_base
CATDESC: Target Low waveform saturation flag.
FIELDNAM: Target Low Saturation Flag
LABLAXIS: Target LG Saturation

ion_grid_saturation_flag:
<<: *event_flag_base
CATDESC: Ion Grid waveform saturation flag.
FIELDNAM: Ion Grid Saturation Flag
LABLAXIS: Ion Grid Saturation

low_sample_rate_attrs:
<<: *sample_rate_base
CATDESC: Low sample rate time steps for a dust event.
Expand Down
70 changes: 70 additions & 0 deletions imap_processing/cdf/config/imap_idex_l1b_variable_attrs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,16 @@ trigger_base: &trigger_base
DISPLAY_TYPE: no_plot
UNITS: " "

event_flag_base: &event_flag_base
<<: *trigger_base
CATDESC: "Boolean event classification flag."
CDF_DATA_TYPE: CDF_UINT1
FILLVAL: 255
FORMAT: I1
UNITS: " "
VALIDMAX: 1
VALIDMIN: 0

spice_base: &spice_base
<<: *l1b_data_base
DISPLAY_TYPE: time_series
Expand All @@ -57,6 +67,66 @@ spice_base: &spice_base
VAR_TYPE: data

# <=== Instrument Setting Attributes ===>
science_event_flag:

@lacoak21 lacoak21 Aug 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you validated that these new attrs are ISTP compliant?

@lacoak21 lacoak21 Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alex is going to use SKTeditor to check

@aldo9253 aldo9253 Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best I can tell, these values are compliant. The webpage checker doesn't seem to like a fill-value of 255 but it's what we use across other variables so I think we leave it as is.

@lacoak21 lacoak21 Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the error? Can you attach a screen shot? 255 may be a valid fill val for other variables but not these if its not configured properly.

@aldo9253 aldo9253 Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ISTP checker now returning compliance for the flags! Thanks for catching those issues.

@lacoak21 lacoak21 Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

excellent. Super appreciate you checking.

<<: *event_flag_base
CATDESC: Science event classification flag.
FIELDNAM: Science Event Flag
LABLAXIS: Science Event

noise_capture_flag:
<<: *event_flag_base
CATDESC: Noise capture event classification flag.
FIELDNAM: Noise Capture Flag
LABLAXIS: Noise Capture

pulser_flag:
<<: *event_flag_base
CATDESC: Pulser event classification flag.
FIELDNAM: Pulser Flag
LABLAXIS: Pulser

dust_hit_flag:
<<: *event_flag_base
CATDESC: Dust Hit waveform classification flag.
FIELDNAM: Dust Hit Flag
LABLAXIS: Dust Hit

tof_high_saturation_flag:
<<: *event_flag_base
CATDESC: TOF High waveform saturation flag.
FIELDNAM: TOF High Saturation Flag
LABLAXIS: TOF HG Saturation

tof_mid_saturation_flag:
<<: *event_flag_base
CATDESC: TOF Mid waveform saturation flag.
FIELDNAM: TOF Mid Saturation Flag
LABLAXIS: TOF MG Saturation

tof_low_saturation_flag:
<<: *event_flag_base
CATDESC: TOF Low waveform saturation flag.
FIELDNAM: TOF Low Saturation Flag
LABLAXIS: TOF LG Saturation

target_high_saturation_flag:
<<: *event_flag_base
CATDESC: Target High waveform saturation flag.
FIELDNAM: Target High Saturation Flag
LABLAXIS: Target HG Saturation

target_low_saturation_flag:
<<: *event_flag_base
CATDESC: Target Low waveform saturation flag.
FIELDNAM: Target Low Saturation Flag
LABLAXIS: Target LG Saturation

ion_grid_saturation_flag:
<<: *event_flag_base
CATDESC: Ion Grid waveform saturation flag.
FIELDNAM: Ion Grid Saturation Flag
LABLAXIS: Ion Grid Saturation

trigger_mode_lg:
<<: *string_base
CATDESC: Low Gain Trigger Mode.
Expand Down
64 changes: 64 additions & 0 deletions imap_processing/cdf/config/imap_idex_l2a_variable_attrs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
int_fillval: &int_fillval -9223372036854775808
double_fillval: &double_fillval -1.0E+31

event_flag_base: &event_flag_base
CATDESC: Boolean event classification flag.
CDF_DATA_TYPE: CDF_UINT1
DEPEND_0: epoch
DICT_KEY: SPASE>Support>SupportQuantity:QualityFlag
DISPLAY_TYPE: time_series
FILLVAL: 255
FORMAT: I1
LABLAXIS: " "
UNITS: " "
VALIDMAX: 1
VALIDMIN: 0
VAR_TYPE: data
int_maxval: &int_maxval 9223372036854775807

# <=== Base Attributes ===>
Expand Down Expand Up @@ -141,6 +155,56 @@ mass_index:

# <=== Data Attributes ===>

science_event_flag:
<<: *event_flag_base
CATDESC: Science event classification flag.
FIELDNAM: Science Event Flag

noise_capture_flag:
<<: *event_flag_base
CATDESC: Noise capture event classification flag.
FIELDNAM: Noise Capture Flag

pulser_flag:
<<: *event_flag_base
CATDESC: Pulser event classification flag.
FIELDNAM: Pulser Flag

dust_hit_flag:
<<: *event_flag_base
CATDESC: Dust Hit waveform classification flag.
FIELDNAM: Dust Hit Flag

tof_high_saturation_flag:
<<: *event_flag_base
CATDESC: TOF High waveform saturation flag.
FIELDNAM: TOF High Saturation Flag

tof_mid_saturation_flag:
<<: *event_flag_base
CATDESC: TOF Mid waveform saturation flag.
FIELDNAM: TOF Mid Saturation Flag

tof_low_saturation_flag:
<<: *event_flag_base
CATDESC: TOF Low waveform saturation flag.
FIELDNAM: TOF Low Saturation Flag

target_high_saturation_flag:
<<: *event_flag_base
CATDESC: Target High waveform saturation flag.
FIELDNAM: Target High Saturation Flag

target_low_saturation_flag:
<<: *event_flag_base
CATDESC: Target Low waveform saturation flag.
FIELDNAM: Target Low Saturation Flag

ion_grid_saturation_flag:
<<: *event_flag_base
CATDESC: Ion Grid waveform saturation flag.
FIELDNAM: Ion Grid Saturation Flag

tof_snr:
CATDESC: Signal-to-noise ratio of the TOF High waveform.
DEPEND_0: epoch
Expand Down
6 changes: 3 additions & 3 deletions imap_processing/cdf/config/imap_idex_l2b_variable_attrs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ mass_labels:
VAR_TYPE: metadata

charge_labels:
CATDESC: Labels for Impact Charge (fC)
CATDESC: Labels for Impact Charge (pC)
DEPEND_1: impact_charge
DICT_KEY: SPASE>Support>SupportQuantity:Other
FIELDNAM: Impact Charge (fC)
FIELDNAM: Impact Charge (pC)
FORMAT: A8
VAR_TYPE: metadata

Expand Down Expand Up @@ -66,7 +66,7 @@ impact_charge:
LABLAXIS: Impact Charge
LABL_PTR_1: charge_labels
SCALETYP: log
UNITS: fC
UNITS: pC
VALIDMAX: 1.00e04
VALIDMIN: 0.0
VAR_TYPE: support_data
Expand Down
7 changes: 7 additions & 0 deletions imap_processing/idex/idex_constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,13 @@ class ConversionFactors(float, Enum):
Ion_Grid = 7.46e-4


# Constants for the ion-grid V(R) relation, where R is the ion-grid to target
# charge ratio and velocity is returned in km/s.
ION_GRID_VELOCITY_SCALE = 55.0
ION_GRID_VELOCITY_EXPONENT = -3.2
ION_GRID_VELOCITY_OFFSET = 1.5


SPICE_ARRAYS = [
"ephemeris_position_x",
"ephemeris_position_y",
Expand Down
Loading
Loading