Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,9 @@ By making a contribution to this project, I certify that:
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```

## Unit tests

Run `npm test` for the unit suite or `npm run test:coverage` for the coverage run used in CI.

For Angular tests using `createAsyncObservable`, use `fakeAsync` and `flush` to finish finite asynchronous initialization and save operations before making assertions. Fixed-duration sleeps do not guarantee that nested timers have completed. The SaveDialog candidate-enrollment regression uses this approach to verify enrollment in both the existing and newly selected tracks before the dialog closes.
103 changes: 101 additions & 2 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,9 @@ creator is not used as a substitute for the snapshot's creator.
Older snapshots without recorded provenance show **Creation cause unavailable**.
Saving virtual-track configuration can create a configuration draft after the
schedule is saved; the schedule's later execution creates a separate scheduled
snapshot. Standard tracks retain only their most recent rolling draft, so
these labels describe the surviving snapshots rather than every past action.
snapshot. Standard tracks retain their most recent rolling draft plus preserved
release sources and drafts referenced by virtual provenance, so these labels
describe surviving snapshots rather than every past action.

For a virtual track's recurring schedule, use **Find a schedule** in the Config
tab: typing `hou` suggests **Hourly — at minute 0**, and typing `every 15`
Expand All @@ -61,10 +62,108 @@ Saving only a schedule leaves the virtual track's composition and draft
unchanged. Deduplication configuration supports the strategy selector;
preferred tier and preferred status are not supported by the API.

When creating a virtual track or editing its **Config**, each component's
**Resolve from** selector offers **Latest tagged** (the default) and
**Latest preview**. Latest tagged uses the newest published release. Latest
preview uses the newest standard snapshot whether draft or tagged: draft
previews include members plus staged changes under the source's release
conflict rules, and tagged sources use published members. Candidates are
excluded. A blocking source conflict must be resolved before materialization;
virtual filters do not bypass it.

Each added component also has an editable **Priority** field, both in the
creation dialog and under **Config → Edit Config**. Lower numbers have higher
priority. Values must be unique, non-negative whole numbers; gaps are allowed.
Duplicate, empty or invalid priorities show an inline error and prevent saving.
Edit these numbers to change precedence without removing/re-adding tracks.
Existing priorities are preserved when saving or removing components, and new
components receive an unused priority. Cancel restores the saved values.

Priority determines precedence when using **prioritize higher priority** and
provides tie-breaking where the other deduplication strategies use it. After
saving changed priorities, use **Create Draft** to apply them to a new composed
snapshot; existing snapshots retain their original content.

Use **Create Draft** after saving composition to materialize the downstream
preview. This does not tag, promote or otherwise change the standard source.
The virtual snapshot freezes exact revisions and can be reviewed or tagged
independently. Its provenance shows **Draft preview** for an untagged source
or the source's actual version for a tagged one, with the exact source timestamp.
**Preview members** counts the prospective member set before filters, not merely
the source draft's stored members.

Loaded **Latest draft** rules are retired members-only configuration. The editor
requires explicitly choosing **Latest preview** or **Latest tagged** before
saving or creating another snapshot; it never silently includes staged content.
Historical draft-only provenance retains its original meaning.

Virtual-track history offers **Drafts only**, **Releases only**, and **All
releases** (the default). All releases includes both drafts and tagged releases;
Releases only never inserts a current draft into the results. The paginator loads
25 entries per page, newest first.

The smaller summary below the selector shows **Tagged releases**, **Drafts**, and
**Total** for the selected view across all matching pages, not just the visible
page. For example, a track with two tags and three drafts reports `2 / 3 / 5` in
All releases, `2 / 0 / 2` in Releases only, and `0 / 3 / 3` in Drafts only.
Zero-result views still show all three counters.

History refreshes automatically every 30 seconds while the Releases tab is
visible, and immediately when entering that tab or returning browser focus or
visibility. This picks up cron-created snapshots and other users' changes
without a Refresh history button. Refresh pauses while editing, while a dialog
is open, or while an operation/request is in flight. It preserves the selected
filter, page and scroll position; if cleanup removes the last page, it moves to
the last remaining page. Local changes still refresh immediately.

Automatic refresh reads lightweight history and cleanup status rather than
reloading full snapshot contents or configuration. The **LATEST** pill and
latest-only actions use unfiltered server identities, so an older filtered page
is not mistaken for the current snapshot.

**Create Draft** offers administrators **Delete older drafts after creating this
draft**, off by default every time the dialog opens. A positive whole-number
limit applies to that request only; it is never saved or inherited from a
schedule.

To configure persistent retention, click **Edit Config** and select **Recurring**
schedule mode. **Recurring draft retention** is read-only outside edit mode;
administrators can change its limit while editing. **Cancel** restores the saved
policy. **Save Config** sends schedule-only changes without creating a content
draft or deleting anything immediately. Actual cron runs apply this policy;
manual creation and dated schedules do not. The former global policy has been
removed, so configure the recurring policy explicitly.

Both policies count all untagged snapshots across the track, regardless of how
they were created. Tagged releases and protected sources survive. Configuration,
metadata, composition and quarantine edits never trigger retention.

The virtual release preview offers administrators an unchecked **Delete earlier
drafts after tagging** option. It shows eligible/protected counts and the strict
interval since the preceding release (or track creation). This permanently
deletes history, not content; the selected release and newer snapshots survive.
If the reviewed deletion set changes, fetch a fresh preview.

If a release commits but cleanup fails, the page says so and exposes **Retry
cleanup**. This repairs the existing operation without tagging again. Pending
operations are rediscovered when reopening the track. Completed cleanup appears
as a floating notification with an **X** dismiss button; dismissal survives
ordinary history refreshes. Pending/failed operations retain their repair
controls. Large cleanups can need more than one retry. If a draft is removed
while you are viewing it, the automatic history refresh lets you select a surviving snapshot.

The release preview offers minor and major relative tags as well as an exact `MAJOR.MINOR` version. Relative tags are calculated from the tagged snapshot immediately before the selected draft. When releasing an older draft, the exact version must also remain below the next tagged snapshot; the dialog shows these exclusive bounds. Optional release notes are stored on that snapshot and become the `x-mitre-collection` description in exported STIX bundles.

The release-track page follows a draft-then-tag flow: the Board tab manages what the next draft contains (candidates, staged objects, and for virtual tracks the Create Draft action), and the Releases tab previews and tags a draft from its card. For virtual tracks, each snapshot card shows its own Composition Resolution provenance: the exact component track snapshot, tagged version, snapshot creation timestamp, resolution strategy and filters, and source/filter/contribution counts used for that materialization. Any snapshot can be exported from its card as a STIX 2.0 bundle, a STIX 2.1 bundle, or Workbench JSON. Historical snapshot exports can also copy a concise summary. Every snapshot seals its content when its members are written, so exports replay the exact members, relationships, and supporting objects in either STIX version; released snapshots also show their stable bundle identifier and SHA-256 hashes. Saving a relationship resets its source and target to work-in-progress in place without creating new revisions of those objects. A standard release preserves its exact pre-release draft, which remains hidden while the release exists. Administrators can convert the most recent tagged release back to a draft from the Releases tab by confirming its version. Standard tracks restore the preserved pre-release draft; virtual tracks retain the same snapshot and composition provenance while removing its tag and publication metadata. Conversion is blocked when any downstream virtual snapshot resolved that release. Tagged releases cannot be deleted directly. Editors can separately delete the current draft, provided it is not the track's only snapshot, a preserved source of a tagged release, or a resolved component of a downstream virtual snapshot. Administrators can also correct a tagged snapshot's version from its card when the replacement remains valid between adjacent releases. A track can carry an alias (a short lowercase slug set in the Config tab) that works in place of its ID in page URLs and API paths; the track list opens aliased tracks by their alias. Virtual-track schedules are configured in the Config tab as manual, recurring, or specific dates; recurring schedules use guided cadence/day/time controls that generate a five-field UTC cron expression, and specific dates use controlled future UTC date and time inputs. Scheduled drafts run only when the connected REST API has its global scheduler enabled. The dashboard's Data Quality page adds a domain consistency report: relationships whose objects share no domain (and objects with no domain) can never ship in the same bundle, so fix them at the source rather than expecting the bundle to pull in related objects. Only the most recent tagged release offers Convert to draft, and only the current draft offers Delete draft, a progress bar with a status message appears under the page header while a long operation runs, and deleting an entire track lives in the danger zone at the bottom of the Config tab.

On a virtual track's **Board**, **Composition Resolution** starts collapsed to
leave more room for Members and Quarantine. Click its heading, or focus it and
press Enter or Space, to expand or collapse the table. The resolution timestamp
remains visible when collapsed. Opening a different track resets it to collapsed.
The expanded table fills the panel, with track IDs below their names and
right-aligned object counts. On narrow screens, scroll the table horizontally
to see all columns.

Each cached snapshot card displays server-generated SHA-256 hashes for the exact UTF-8 JSON files produced by its STIX 2.0 and STIX 2.1 bundle downloads. The adjacent copy buttons copy a hash for external file-integrity verification. Snapshot notes are locked while the bundle is cached; delete the cache, edit the notes, and cache the bundle again to generate matching hashes.

#### Adding a Collection Index
Expand Down
37 changes: 34 additions & 3 deletions src/app/classes/release-tracks/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@
type ExportFormatType,
type ReleasePreviewFormatType,
} from './enums';
import type { SnapshotSchedule } from './release-track';
import type {
DraftRetention,
DraftSquashPreview,
SnapshotSchedule,
} from './release-track';
import type { SnapshotCreationCause } from './snapshot-creation-cause';
import type { SnapshotCreationActor } from './snapshot-creation-actor';

Expand All @@ -22,6 +26,11 @@
snapshot_schedule?: SnapshotSchedule;
}

export interface CreateVirtualSnapshotPayload {
description?: string;
draft_retention?: DraftRetention | null;
}

export interface StixBundlePayload {
type: 'bundle';
id?: string;
Expand All @@ -43,7 +52,11 @@
| { increment: 'major' | 'minor'; version?: never }
| { increment?: never; version: string }
| { increment?: undefined; version?: undefined }
) & { description?: string };
) & {
description?: string;
squash_drafts?: boolean;
squash_fingerprint?: string;
};

export interface RetagReleasePayload {
version: string;
Expand Down Expand Up @@ -77,6 +90,22 @@
offset?: number;
}

/** Counts cover the selected tagged filter before pagination. */
export interface SnapshotHistoryCounts {
tagged: number;
drafts: number;
total: number;
}

export interface SnapshotHistoryResponse {
data: ReleaseTrackSnapshotHistoryItem[];
pagination: { total: number; limit: number; offset: number };
counts: SnapshotHistoryCounts;
/** Unfiltered live identities, even when neither snapshot is on this page. */
latest_snapshot_modified: string | null;
latest_tagged_snapshot_modified: string | null;
}

export interface SnapshotContentStatistics {
primary_count: number;
secondary_count: number;
Expand Down Expand Up @@ -151,6 +180,7 @@

export interface VirtualReleasePreviewSummary extends ReleasePreviewSummaryBase {
type: ReleaseTrackType.Virtual;
draft_squash?: DraftSquashPreview;
previous_release: {
version: string;
modified: string;
Expand All @@ -172,7 +202,8 @@
}

export type ReleasePreviewSummary =
StandardReleasePreviewSummary | VirtualReleasePreviewSummary;
| StandardReleasePreviewSummary

Check failure on line 205 in src/app/classes/release-tracks/api.ts

View workflow job for this annotation

GitHub Actions / static-checks

Replace `|·StandardReleasePreviewSummary⏎·` with `StandardReleasePreviewSummary`
| VirtualReleasePreviewSummary;

export interface SnapshotBundleHashes {
manifest_id: string;
Expand Down
73 changes: 70 additions & 3 deletions src/app/classes/release-tracks/component-track.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@
// Release track (standard or virtual) referenced by a virtual release track
// -----------------------------------------------------------------------------

import { ResolutionStrategyType } from './enums';
import {
HistoricalResolutionStrategyType,
ResolutionStrategyType,
} from './enums';

export interface ComponentTrackFilters {
object_types?: string[];
Expand All @@ -16,19 +19,83 @@ export interface ComponentTrack {
resolution_strategy: ResolutionStrategyType;
priority: number;
version?: string | null;
snapshot?: Date;
snapshot?: Date | string;
filters?: ComponentTrackFilters;
}

/** Loaded rules may be retired; they must be replaced before a new write. */
export interface LoadedComponentTrack extends Omit<
ComponentTrack,
'resolution_strategy'
> {
resolution_strategy: HistoricalResolutionStrategyType;
}

export interface ComponentSnapshotResolution {
track_id: string;
track_name: string;
track_type: string;
resolved_snapshot_id: Date | string;
resolved_version?: string | null;
strategy_used: string;
strategy_used: HistoricalResolutionStrategyType;
filters_applied?: ComponentTrackFilters;
total_objects_in_source: number;
objects_after_filter: number;
objects_contributed: number;
}

interface PrioritizedComponent {
priority?: number | null;
}

function isComponentPriority(
value: number | null | undefined
): value is number {
return typeof value === 'number' && Number.isSafeInteger(value) && value >= 0;
}

export function hasValidComponentPriorities(
tracks: readonly PrioritizedComponent[]
): boolean {
const seen = new Set<number>();
for (const { priority } of tracks) {
if (!isComponentPriority(priority) || seen.has(priority)) return false;
seen.add(priority);
}
return true;
}

export function getComponentPriorityError(
priority: number | null | undefined,
tracks: readonly PrioritizedComponent[]
): string | null {
if (!isComponentPriority(priority))
return 'Enter a valid non-negative whole number.';
let found = false;
for (const track of tracks) {
if (track.priority !== priority) continue;
if (found) return 'Each component must have a unique priority.';
found = true;
}
return null;
}

export function nextComponentPriority(
tracks: readonly PrioritizedComponent[]
): number {
let next = 0;
for (const { priority } of tracks) {
if (!isComponentPriority(priority)) continue;
next = Math.max(next, priority + 1);
}
if (Number.isSafeInteger(next)) return next;
// A component may already use the largest safe integer; fill a free slot
// rather than generate an invalid priority or renumber existing components.
const used = new Set<number>();
for (const { priority } of tracks) {
if (isComponentPriority(priority)) used.add(priority);
}
next = 0;
while (used.has(next)) next++;
return next;
}
13 changes: 12 additions & 1 deletion src/app/classes/release-tracks/composition.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@
// -----------------------------------------------------------------------------

import { DeduplicationStrategyType } from './enums';
import { ComponentTrack, ComponentSnapshotResolution } from './component-track';
import {
ComponentTrack,
ComponentSnapshotResolution,
LoadedComponentTrack,
} from './component-track';

export interface Composition {
component_tracks?: ComponentTrack[];
Expand All @@ -15,6 +19,13 @@ export interface Composition {
};
}

export interface LoadedComposition extends Omit<
Composition,
'component_tracks'
> {
component_tracks?: LoadedComponentTrack[];
}

export interface CompositionResolution {
resolved_at?: Date | string;
component_snapshots?: ComponentSnapshotResolution[];
Expand Down
6 changes: 4 additions & 2 deletions src/app/classes/release-tracks/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ import type {
} from './enums';

export type InheritedIdentitySetting =
{ inherit: true } | { inherit: false; value: string };
| { inherit: true }
| { inherit: false; value: string };

export type InheritedMarkingRefsSetting =
{ inherit: true } | { inherit: false; value: string[] };
| { inherit: true }
| { inherit: false; value: string[] };

// Publication metadata for the emitted x-mitre-collection object. Each rule
// inherits the global system configuration unless overridden at the track
Expand Down
5 changes: 5 additions & 0 deletions src/app/classes/release-tracks/enums.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,17 @@ export const CANDIDACY_THRESHOLD_OPTIONS: WorkflowStatusType[] = Object.values(

export enum ResolutionStrategy {
LatestTagged = 'latest_tagged',
LatestPreview = 'latest_preview',
SpecificVersion = 'specific_version',
SpecificSnapshot = 'specific_snapshot',
}

export type ResolutionStrategyType = EnumValue<typeof ResolutionStrategy>;

/** Read-only compatibility for saved rules and historical members-only results. */
export type HistoricalResolutionStrategyType =
ResolutionStrategyType | 'latest_draft';

export const RESOLUTION_STRATEGY_OPTIONS: ResolutionStrategyType[] =
Object.values(ResolutionStrategy) as ResolutionStrategyType[];

Expand Down
2 changes: 1 addition & 1 deletion src/app/classes/release-tracks/history.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ export interface VersionHistoryEntry {
staged_count?: number;
candidate_count?: number;
};
component_versions?: any; // virtual tracks only
component_versions?: Record<string, string | null>; // virtual tracks only
}
Loading
Loading