feat(event-display,ui): Add a loader for ATLAS ESD/AOD files - #975
Merged
Conversation
ATLAS ESD/AOD samples are hundreds of MB and are never committed. Ignoring them also keeps 'git clean -fd' from deleting a local test file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jsroot cannot open any ROOT file compressed with LZMA, which is what ATLAS writes ESD and AOD with. modules/base/lzma.mjs skipped a hard-coded 29-byte prefix and ran the LZMA1 core over the remainder in a single pass. That is only valid when the xz block holds exactly one LZMA2 chunk, and a chunk caps at 2 MiB uncompressed / 64 KiB compressed, so anything larger failed with "corrupted input". An ATLAS ESD CollectionTree is ~2.2 MB in two chunks, and the second inherits the decoder state and dictionary from the first rather than resetting them. scripts/jsroot-patch/lzma.mjs is a drop-in replacement, kept whole so the same file can go upstream. The LZMA1 core is untouched; what changed is: - the xz stream and block headers are parsed instead of assumed, so blocks with stored sizes or a non-default check size work; - decompress() drives the LZMA2 chunk sequence, carrying decoder state, probability models and the dictionary window across chunks according to each chunk's reset flags, and loops over blocks for streams that have more than one; - the dictionary size comes from the block header rather than a fixed 8 MiB, clamped to the output size, and the window is a Uint8Array instead of a holey Array reallocated on every call. Installed over node_modules by the existing postinstall hook, which is renamed from patch-jsroot-three.js since it now applies two patches. A missing payload warns rather than failing the install, and the file carries a marker so an older copy of the patch is upgraded in place. Verified against a real ATLAS ESD: all 959 LZMA blocks (32.7 MB, 11 of them multi-chunk, up to 21 chunks) decode byte-identical to xz, plus 96 synthetic streams across presets 0/1/6/9e, every check type, and threaded multi-block. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reads an ATLAS ESD (.pool.root) with jsroot and converts it to Phoenix event data, alongside the existing PHYSLITE loader. The formats differ in more than size. PHYSLITE puts each variable in its own flat branch (AnalysisJetsAuxDyn.pt); an ESD keeps kinematics in the static auxiliary store, a single unsplit branch holding a streamed C++ object whose members are the aux vectors, and prefixes branch names with the persistent class and version: xAOD::TrackParticleAuxContainer_v5_InDetTrackParticlesAux. So branches are matched by pattern and looked up by aux class family, which keeps the loader working when a container is rewritten at a newer _vN. A container allow-list keeps the dozens of trigger and HLT containers an ESD also carries out of the collections menu; it is the documented extension point, since read cost is per container rather than per field. Covers tracks, vertices, calo clusters, MET, electrons, photons, muons, jets and taus. Unlike PHYSLITE the ESD supplies chiSquared/numberDoF and vertexType, emitted as chi2/dof/vertexType so the existing cut sliders in object-type-registry pick them up. Two details worth flagging: - MET is a container of terms (RefEle, RefJet, ... FinalTrk, FinalClus), so the total is selected by name rather than by index. - PhoenixLoader.getCompound derives charge as `pdgId > 0 ? 1 : -1`, reading the sign of pdgId as the sign of the charge, which is inverted from the PDG convention for leptons. Where the ESD gives a charge, pdgId is signed to match Phoenix's convention so extrapolated tracks bend correctly. Not handled: POOL type/persistent-separated collections (Trk::TrackCollection_tlp7_*), calorimeter cells, and ElementLink resolution, so compound objects carry no LinkedTracks/LinkedClusters and Phoenix extrapolates from their kinematics, exactly as for PHYSLITE. Verified against a real ESD: 762 InDetTrackParticles, 93 GSF, 33 vertices, 483 topo clusters, 26 electrons, 20 photons, 77 jets, 37 taus in 0.2 s, all building Three.js objects with no non-finite vertices and track extent inside the inner-detector envelope. The jsroot test mock gains TSelector and treeProcess so loaders that read trees can be tested at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the ESD format to the IO options dialog and to the ATLAS section, so a
.pool.root can be opened from the menu.
Both ROOT formats accept .root and the dialog renders "Load {{ fileType }}",
which would have given two identically labelled buttons, so PHYSLITE's label
becomes "PHYSLITE (.root)" and the new one "ATLAS ESD (.root)". fileType is
display-only; the real filename check is isFileOfExtension.
Not added to the masterclass section, which is deliberately simplified.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The loader silently dropped things, so a collection missing from the menu had
no explanation. It now tallies every skip with a reason and prints a summary
to the console once a load finishes:
ATLASESDLoader: read 17 collection(s) from 1 event(s); skipped 3 event(s)
Muons — empty in this event: 1 event(s)
CombinedMuonTrackParticles — empty in this event: 1 event(s)
ExtrapolatedMuonTrackParticles — empty in this event: 1 event(s)
Reasons are kept distinct so the summary is actionable, in particular
separating a container that is empty in this event from one that is not in
the file at all — the two most common causes of a collection not appearing.
Object-level skips are attributed to their container, and the track guard's
reasons are now reported separately rather than as one condition.
That guard also gains a finite check on d0/z0/phi. It already existed to keep
NaN out of the Runge-Kutta extrapolation, but only covered qOverP and theta,
so a non-finite perigee parameter would still have reached the renderer.
The resolveCollections warnings fold into the same summary instead of being
separate console.warn calls, so there is one place to look.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The headline summed every tally and called the total "skipped", so a file
where two collections happened to be empty in the single event read reported:
read 17 collection(s) from 1 event(s); skipped 2 event(s)
No events were skipped. The count was a number of events, but the thing
skipped was a collection, and rolling the two together produced a sentence
that was simply untrue.
Each unit is now counted on its own terms. Objects are summed, since dropping
12 tracks is worth stating as 12. Collections are counted once each, whether
unavailable outright or merely empty, and the event count becomes detail on
the line rather than a headline figure:
read 17 collection(s) from 1 event(s); 3 collection(s) empty
Muons — empty: 1/1 events
CombinedMuonTrackParticles — empty: 1/1 events
ExtrapolatedMuonTrackParticles — empty: 1/1 events
Reported by a user reading the output, who pointed out that one event was
read and none skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the POOL type/persistent-separated track collections alongside the xAOD
ones. These are a different shape entirely: parallel arrays joined by
TPObjRef ({m_typeID:{m_cnvID}, m_index}, where m_cnvID 0 means null), walked
m_trackCollections[0] -> m_tracks -> m_trackState
-> m_trackStates -> m_trackParameters -> m_parameters
with the perigee identified by m_surfaceType === 3. There is exactly one per
track; surface types 4 and 5 are measurement surfaces, and their transforms
are not persisted (m_surfaces and m_detElementSurfaces are empty), so no
measured polyline can be reconstructed and tracks are extrapolated from the
perigee just as the xAOD ones are.
Off by default. In practice each collection duplicates an xAOD TrackParticle
container — CombinedInDetTracks and InDetTrackParticles are the same 762
tracks, agreeing to |dtheta| 7e-7 and |dqOverP| 0 — and reading all 13 costs
~460 ms because an unsplit branch streams whole, including arrays never used
(m_hepSymMatrices alone is 32k entries). Opt in per collection:
new ATLASESDLoader({ extraContainers: ['CombinedInDetTracks'] })
Note this leaves them API-only: the IO options dialog builds the loader with
no options, so they are not reachable from the menu yet.
The per-track guards and their skip reasons move into a shared makeTrack()
used by both converters, so there is still one place where a track can be
rejected. One new reason, 'no perigee parameters', covers a track whose
states yield no perigee.
Verified on a real ESD: CombinedInDetTracks 762, GSFTracks 93,
ResolvedForwardTracks 26, MuonSpectrometerTracks 1, the other nine empty and
reported as such; 882 tracks render with 0 non-finite vertices.
CaloCells and PrepRawData were requested too but are not included: both store
only local coordinates plus detector identifiers, so placing them in 3D needs
the ATLAS detector description that the ESD does not carry. The class TSDoc
records the specifics.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Correcting an earlier claim: I said no measured polyline could be reconstructed because the surface transforms are not persisted. That is only true of parameters expressed on a detector-element surface. Two cases do carry global positions: - Curvilinear parameters (m_surfaceType 6) store *seven* values rather than five — global position, global momentum and charge — because the frame is defined by the track itself. ResolvedForwardTracks has 78 of them, and the momentum direction matches its own perigee (identical theta, exact q/|p|). - Plane parameters (m_surfaceType 4) sometimes carry their transform, 9 row-major rotation elements plus a translation, so the global point is R * (loc1, loc2, 0) + T. MuonSpectrometerTracks has 18. Points are collected in track-state order, which is trajectory order, and consecutive duplicates are dropped because Phoenix fits a CatmullRomCurve3 through them and repeated points give NaN tangents. Line surfaces are excluded on purpose: their first local parameter is a signed drift distance perpendicular to the wire, not a Cartesian offset. dparams is still emitted alongside pos, so cut values stay the perigee ones rather than being derived from the first two points. On the sample: MuonSpectrometerTracks 1/1 tracks now follow a 15-point, 8.3 m measured trajectory through the muon endcap (r 2948-3376 mm, |z| 13.3-21.6 m), and ResolvedForwardTracks 17/26. The measured extent is only as long as the fit, though, and that cuts both ways — several forward tracks are 3-point ~250 mm stubs near the beamline, which is truthful but shorter than the extrapolation it replaces. Hence the new measuredPositions option (default true) to turn the whole behaviour off, and a per-collection console line reporting how many tracks used it. Reported by a user pointing at AtaSurfaceCnv_p1 in Athena. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
They were registered but left out of the default allow-list, which in practice made them unreachable: the IO options dialog constructs the loader with no options, so nothing short of a code change could switch them on. Adding them to the default list is what makes them visible in the menu. The duplication that motivated leaving them out is real and unchanged — CombinedInDetTracks and InDetTrackParticles are the same 762 tracks, so the menu now shows both — and reading all 13 adds ~460 ms, taking the sample from 0.2 s to 0.7 s. What they buy is the collections with no xAOD counterpart and the measured trajectories, which only exist on this side. `containers` still excludes them for anyone who wants the xAOD view alone. Three tests encoded the old behaviour and have been rewritten rather than deleted: the POOL branch that stood in for "unreadable" is now a genuinely unreadable pair (CaloCompactCellContainer, PixelClusterContainer), and the opt-in test becomes its inverse plus one asserting the xAOD and Trk collections coexist rather than shadowing each other. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The allow-list was only discoverable by reading the source, so there was no
way to tell from a running app what else a file offered. The loader now lists
the containers it has a converter for but was not asked to read:
ATLASESDLoader: 131 readable container(s) not in the allow-list; add any of
these with the containers or extraContainers option
CaloClusters CaloCalFwdTopoTowers, ForwardElectronClusters, ...
Electrons ForwardElectrons, LRTElectrons
Tracks InDetForwardTrackParticles, InDetLargeD0TrackParticles, ...
(86 trigger container(s) not listed: HLT_*, L1_*)
Two filters keep it useful rather than overwhelming. Containers with no
converter are never mentioned — there are 200 of them in the sample and
nothing could be done about any of them — and trigger containers are counted
rather than named, since 86 HLT and L1 entries would bury the 45 offline ones
anyone is likely to want.
Kept separate from the skip tally so the "skipped" counts continue to mean
what they say.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The single MuonSpectrometerTracks track was loaded but never drawn. Its perigee carries qOverP = 1e-8, which is how a fit records that it did not measure the momentum — a straight-line track. Taken literally that is |p| of 100 TeV and pT of 4.7 TeV, well outside the default pT cut of 50 GeV, so the track was hidden the moment cuts were applied, with nothing to say why. pT is now emitted only when the momentum is physically possible; no LHC track approaches the ~7 TeV beam energy, so anything above 10 TeV is a sentinel. Tracks without one carry momentumMeasured: false instead. Cuts skip fields that are absent, so the pT cut simply does not apply to such a track rather than hiding it, and the collection reports the case on load. This was not specific to Trk::TrackCollection: MuonSpectrometerTrackParticles holds the identical value and has been invisible since the loader was first written. Both are fixed, since both go through makeTrack. Reported by a user who noticed the collection had one track that never appeared, and that clearing the cuts did not bring it back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI enforces 100% documentation coverage via compodoc and this one interface member had no comment, taking the package to 99%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EdwardMoyse
force-pushed
the
add-esd-loader
branch
from
August 16, 2026 18:31
8da406f to
96ea5e0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a new loader which reads ROOT ESD/AOD files.
There are some limitations: since we do not have access to the geometry, we cannot look up e.g. calo cell positions, so it relies on the same hack as with JiveXML, to place them on a cylinder. And for PRDs there's just nothing to do - we only have local positions and so these cannot be shown. For tracks, we rely on Curvilinear track parameters, since only these have global positions.
Entirely vibe coded with claude BTW.