Skip to content

Add required flag to DIVE_PARAM - #1639

Merged
BryonLewis merged 5 commits into
mainfrom
dev/dive-param-required
Jul 21, 2026
Merged

Add required flag to DIVE_PARAM#1639
BryonLewis merged 5 commits into
mainfrom
dev/dive-param-required

Conversation

@mattdawkins

@mattdawkins mattdawkins commented Apr 29, 2026

Copy link
Copy Markdown
Member

Summary

Pipelines can now mark a `DIVE_PARAM` as required so the user can't run the pipeline until they supply a value, and `config ` lines (kwiver globals) can be DIVE_PARAM targets — same syntax, parser-side recognition only.

Spec

```
:my:key = 0.5 # DIVE_PARAM ["My label", float, required]

config global:fps = 5 # DIVE_PARAM ["FPS", int, required]
```

`required` is a flag keyword in the DIVE_PARAM comma list — stripped from `type_props`, sets `required: true` on the param.

Coverage

  • Top-level assignments (`:key = value`) — keyed under `process:block...:key` from surrounding `process` / `block` context.
  • Nested `block` declarations — context stack pushes/pops per `block` / `endblock`, so e.g. `process tracker → block matcher → block weights → :iou` keys as `tracker:matcher:weights:iou`.
  • `config absolute:key = ` — absolute kwiver key, no process/block prefix applied. Used for globals referenced from multiple sites.

Changes

  • `apispec.ts` / `server/dive_utils/types.py`: `required?: boolean` on `DiveParam`.
  • `pipeline_discovery.py` (server) + `platform/desktop/backend/native/common.ts` (desktop): recognise the `required` keyword in the type-props list, plus `config = ` lines as DIVE_PARAM targets.
  • `PipelineParamsDialog`: red `*` on the label, vuetify rule on the input that blocks the Run button when empty.

mattdawkins added a commit that referenced this pull request Apr 29, 2026
Superseded by the `required` flag on DIVE_PARAM (see PR #1639). VIAME
\`.pipe\` files using the old `# Requirements:` header should be
migrated to the new spec — until they are, those parameters won't be
prompted for.

- Drop \`PipelineRequirement\` from apispec.ts and dive_utils/types.py.
- Drop \`description\` and \`requirements\` fields from \`Pipe\` /
  \`PipelineDescription\`; description is already exposed via
  \`metadata.description\`.
- Drop \`extract_pipe_description\`, \`extract_pipe_requirements\`,
  \`extractPipeDescription\`, \`extractPipeRequirements\`. Training
  config descriptions now read from \`extractPipeMetadata().description\`.
- Drop the Requirements dialog from RunPipelineMenu.
@mattdawkins
mattdawkins force-pushed the dev/dive-param-required branch from 389b487 to a10f41e Compare April 29, 2026 12:14
@Louis-Pagnier-KW

Copy link
Copy Markdown
Collaborator

Hi,
Thanks for this nice addition!
I am questioning myself if it could be better to change the param behavior to be keyword instead of positional.

For example to pass from:
# DIVE_PARAM["Max points", range_int, 0, 100, 5]
to
# DIVE_PARAM["Max points", range_int]{min=0, max=100, step=5}

And we would be able to combine it with the required argument:
# DIVE_PARAM["Max points", range_int]{min=0, max=100, step=5, required=true}

It is a bit more verbose, but I think it could be more scalable, add avoid problems if later we have a string parameter for a type.
The main advantage would also be that we will not have to add the required field to types, but directly use the generic params dict.

What do you think of this?

@mattdawkins
mattdawkins requested a review from BryonLewis June 9, 2026 17:35
@mattdawkins
mattdawkins force-pushed the dev/dive-param-required branch from 06e04b6 to 0feff06 Compare June 22, 2026 17:43
Comment on lines 52 to 68
const rules: Record<string, ValidationRule> = {
integer: (value: string | number) => Number.isInteger(Number(value)) || 'Please enter a whole number',
positive: (value: string | number) => Number(value) >= 0 || 'Please enter a number ≥ 0',
strictlyPositive: (value: string | number) => Number(value) > 0 || 'Please enter a number > 0',
};

function getRules(type: PipelineParamType): ValidationRule[] {
const res = [];
function getRules(type: PipelineParamType, required = false): ValidationRule[] {
const res: ValidationRule[] = [];
if (required) {
res.push((value) => {
if (value === undefined || value === null || value === '') return 'Required';
return true;
});
}
if (type.includes('int')) {
res.push(rules.integer);
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can integrate the rule directly in the rules dict to have it consistent.

Suggested change
}
const rules: Record<string, ValidationRule> = {
integer: (value: string | number) => Number.isInteger(Number(value)) || 'Please enter a whole number',
positive: (value: string | number) => Number(value) >= 0 || 'Please enter a number ≥ 0',
strictlyPositive: (value: string | number) => Number(value) > 0 || 'Please enter a number > 0',
required: (value: string | number) => (value !== undefined && value !== null && value !== '') || 'This field cannot be empty',
};
function getRules(type: PipelineParamType, required = false): ValidationRule[] {
const res: ValidationRule[] = [];
if (required) {
res.push(rules.required);
}
if (type.includes('int')) {
res.push(rules.integer);
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise looks good to me

Pipelines can now mark a DIVE_PARAM as required by adding `required`
as one of the type props in the comment header:

    # DIVE_PARAM ["My label", float, required]
    :my:key = 0.5

The parser strips it from `type_props` and sets `required: true` on
the param. PipelineParamsDialog shows a red `*` on the label and
blocks the Run button via vuetify form validation if the field is
empty. Both the desktop common.ts parser and the server
pipeline_discovery.py parser handle the keyword.
@mattdawkins
mattdawkins force-pushed the dev/dive-param-required branch from d851158 to 5f3ebb5 Compare July 21, 2026 00:22
@mattdawkins

Copy link
Copy Markdown
Member Author

Hi, Thanks for this nice addition! I am questioning myself if it could be better to change the param behavior to be keyword instead of positional.

For example to pass from: # DIVE_PARAM["Max points", range_int, 0, 100, 5] to # DIVE_PARAM["Max points", range_int]{min=0, max=100, step=5}

And we would be able to combine it with the required argument: # DIVE_PARAM["Max points", range_int]{min=0, max=100, step=5, required=true}

It is a bit more verbose, but I think it could be more scalable, add avoid problems if later we have a string parameter for a type. The main advantage would also be that we will not have to add the required field to types, but directly use the generic params dict.

What do you think of this?

Probably a good idea if we plan on adding a lot more parameters, though my text editor is starting to get sad because the max line ending on these pipes and that would further extend it. I'm kind of hoping we can partially do away with this system at some point because of that (e.g. taking the kwiver type definitions and adding max ranges and things to those macros so they don't need to be re specified). So I'm fine if it goes in as it is now because I want to refactor this eventually anyway.

Another thing I wouldn't mind doing is just adding a quick comment up here next to the parameters like # DP1, #DP2 for short, then define the actual parameters for DIVE elsewhere in the file, like references in a latex paper

Plain `mdi-cog-outline` gear for pipelines with DIVE params (still its
own click target for the params dialog) and no icon for pipelines
without, as before the params-dialog rework.
sprokit's pipe parser rejects a one-line `config <key> = <value>`
statement, so kwiver globals must stay in config blocks. Track
`config <key>` block context in both discovery parsers so
`config global` + `:scale ...  # DIVE_PARAM [...]` yields key
`global:scale` instead of bare `scale`.
Wrapper pipes that only `include` another pipe now inherit its params
instead of having to redeclare them. A file's own declarations override
inherited ones for the same key, matching kwiver's config override
order. Unresolvable includes ($ENV{...} paths handled by kwiver's own
search path) contribute no params.

@BryonLewis BryonLewis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to merge for proper testing of the pipeline params withe newer sealion metadata pipelines

@BryonLewis
BryonLewis merged commit 6a2e922 into main Jul 21, 2026
3 checks passed
@BryonLewis
BryonLewis deleted the dev/dive-param-required branch July 21, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants