Repository navigation
Preview video outputs on canvas cards - #78
Merged
Merged
Conversation
A step whose outputs were videos showed only its ports and status on the canvas; the video played only in the details panel. Its card now shows the video's first frame with a play button, and plays it in place. SVG has no video that every browser draws in place: Safari draws an HTML video inside foreignObject outside the SVG's transforms and above the rest of it. So the card's first frame is an ordinary SVG image, read in the browser from the file once the card comes into view (once per file, two at a time), and one HTML player laid over the card's frame plays it on request. - One rule picks a card's preview for local runs and published exports: the first image output, or without one the first MP4 or WebM output. A declared video output reserves the preview area, as an image does. - The play button is a labelled control: it neither selects the card nor opens its deck. The player follows its card as the camera and decks move, survives live redraws while the card shows the same file, and stops when it ends, on Esc, or when another card or a deck's veil covers it. - spec/layout.md, the viewing guide and the website's asset checklist (video first frames need CORS too) say so. Closes #77
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Resting on a video's frame for 300 ms plays it over the card, muted and looping, without controls or focus; leaving stops it, and a click on it plays it on with sound and controls. Hovering never replaces a video played with sound, and needs a pointer that hovers and motion not reduced. The frame's checkered backdrop now belongs to the video, so the whole frame takes the pointer. The player is transparent and shows the first frame as its poster, so the backdrop shows around the video as it does around the first frame, and the first frame hides while the video plays over it.
A hovered video now plays muted and looping with the native controls, as a played one does. A native video takes clicks for itself, so a click no longer turns on sound: pressing the hovered video, its controls included, keeps it playing after the pointer leaves, and its volume control unmutes it.
Native video controls are sized for a page, not a zoomable canvas: they crowd a card seen from afar and do not match the canvas. The player now has its own (video-controls.ts): play, a seek slider showing what has played and loaded, time, mute and full screen. They keep their size on screen at any zoom, drop the time and then seek and full screen as the player narrows, and fade while it plays once the pointer rests. With focus in the player, Space or K plays and pauses, the arrow keys seek 5 seconds, M mutes, F goes full screen and Esc closes it; its buttons and slider keep their own keys. A press on the picture of a hovered video keeps it playing with sound, as pressing its controls keeps it playing. Play and pause are filled glyphs of our own, on the card too: Lucide's outlined ones read poorly on a video. The details panel keeps the browser's controls.
Hovering a video again, or playing it, now continues from where it last stopped (pointer left, Esc, covered) rather than from the start; one that played to its end starts over. The player remembers each file's place for as long as the canvas lives, and skips the first-frame poster when it continues, so the start does not flash before the seek lands.
The frame was 328 by 140, about 2.34:1, a shape almost nothing a step makes: 16:9 video and pictures were cut down to a strip, and portrait and square ones came out tiny. It is now 16:9 (328 by 185 on a card with ports, 218 by 123 on one without), the commonest shape of what steps make; the file is still fitted whole, so other shapes show the backdrop around them. The height is derived from the card's width in one place, canvasNodeGeometry, instead of a 140 repeated in the card. Cards stay sized from declarations, never from the media they load.
Hovering a video again went blank, its controls showed 0:00, and only then did it jump to where it had stopped: every stop unloaded the video, so every play fetched, parsed and decoded it again and seeked from the nearest keyframe. A stop now pauses and hides the player but keeps its file for 30 seconds (or until another video plays), so playing it again shows the paused frame at once and goes on from it. The card's first frame and play control also stay until the video shows its own frame there, with the player clear and passing the pointer through until then; nothing blanks while a video loads, and a video that has to seek never shows the time it is leaving. The player no longer needs the first frame as its poster.
The video fades in over the card's first frame and play control, which hide once it has, and fades out on a stop as they show again at once, so switching between them no longer cuts. It is CSS transitions, with @starting-style for a kept video that shows its frame as it appears; the player now hides only after its 150 ms fade, and a play during the fade takes it back. Reduced motion keeps the cut.
A press on a video card while its player was still clear (the hover delay, or a first frame still loading) went to the SVG; the player then appeared under the pointer and took the release, which the SVG never saw. The drag stayed open, so the next move past 4 px captured the pointer and panned the canvas with no button held, the player following its card. Firefox shows its first frame later, so it hit this more often. Releases are now heard on the canvas container, which holds the player and the toolbar, and a move with no button held ends any drag, also covering a release outside the window.
From a review of the whole PR for reuse, simplification, efficiency and whether each fix sits at the right depth. Fixes: - Esc on a card's player closed the viewer's details or the website's selection instead, since their own Esc handlers run first. One exported check, ownsKeys (a text field or the player keeps its own keys), now serves all three handlers. - A covered card could start its video from the keyboard (a back card's play control is still a tab stop), playing it over the front card or a deck's veil. The canvas now owns one rule for what covers a card, which the player asks before it plays and again whenever a deck rests, opens or closes, in place of two inline checks. - A press on a video's frame could outlast the hover delay, so the player appeared under it and took the release. The hover timer now waits for the release. - Clearing a deck's hover now listens for the pointer leaving the canvas, so moving onto the player needs no exception and leaving it onto a panel still restores the resting card. - Reading a first frame kept its video and canvas alive until the canvas was disposed (an abort listener never removed), and each live redraw while a frame was still being read kept the previous render's cards alive. A frame now reaches the cards that show the file when it is ready. - The player followed its card every frame by re-querying the frame and rewriting its position; it now keeps the frame and writes only changes. Simplifications: one first-frame cache instead of two; no flags that repeat the player's class or source; media listeners set up once; timers cleared directly; the play and pause glyphs built with card.ts's element() helper beside the other SVG glyphs; the canvas's own hover and motion settings passed in; one file walker for recorded runs and exports, with cardPreview walking a step's outputs itself.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #77. A step whose outputs were videos showed only its ports and status on the canvas, and the video played only in the details panel. Its card now shows the video's first frame with a play button, and plays it in place.
Why not a
<video>inside the cardThe canvas is SVG, and SVG has no video element. The obvious route, an HTML
<video>insideforeignObject, is broken in Safari: it draws the video outside the SVG's transforms and above everything else in the SVG (WebKit 198660, 23113, 201747). Serving was never the gap: the viewer host already serves verified artifacts with byte ranges.What it does
cardPreviewingraph.ts): one rule for local runs and published exports. A card shows its first image output, or without one its first MP4 or WebM output; the count reads "N video outputs". A declared video output reserves the preview area before the file exists, as an image does.video.ts): read in the browser from the file, drawn to a canvas and shown as an ordinary SVG<image>. It loads only once the card comes into view, once per file, at most two at a time, and is reused across live redraws.video-controls.ts): the viewer's own, because native controls are sized for a page, not a zoomable canvas. They are play, a seek slider showing played and buffered, time, mute and full screen. They keep their size on screen at any zoom, drop the time and then seek and full screen as the player narrows, and fade while it plays once the pointer rests. With focus in the player: Space or K plays and pauses, the arrow keys seek 5 s, M mutes, F goes full screen and Esc closes. Play and pause are filled glyphs of our own, on the card too; Lucide's outlined ones read poorly on a video, an intended exception to the Lucide rule. The details panel keeps the browser's controls. It never replaces a video played with sound, and is off on touch screens and with reduced motion. The landing reel's canvas ignores the pointer, so nothing plays there.canvasNodeGeometryinstead of a repeated140. Cards are still sized from declarations only, so nothing shifts when a file arrives. Sizing each frame to its file's own proportions is left for later: the engine would first have to report each file's width and height to the viewer.ownsKeys(exported) lets the canvas's, the viewer's and the website's Esc handlers leave a focused player alone, so Esc closes the player before a panel or a selection.spec/layout.md(§4, §5.7), the viewing guide, and the website's asset checklist.Website impact
Seven published showcase recordings have video-only steps (
animate#1,take#1), so those cards will show a first frame and a play button. The first frame is read withcrossOrigin="anonymous", so the asset host must send CORS headers for video files too.www/README.mdalready requires CORS for the asset host. If a host doesn't send them, the card shows the play button over an empty frame and playback still works. Steps with both a poster image and a video, as in the local motion showcase, are unchanged.Verification
test:viewer: 145 pass, with new cases for the controls' time format, seeking and keys, and for video-only steps, image-over-video preference, MKV and unverified URLs skipped, video ports reserving space, video previews through the export boundary, and player placement.check:viewer,build:viewer,check:www,test:www(94 pass),build:www,tools/check_spec.py,cargo test -p grida-fx-viewerandtools/check_viewer_fixtures.py(13 run views) pass.animate#1from the real asset host): nothing before the delay, then muted and looping with controls, without focus; leaving stops it; a press on the picture keeps it playing with sound and selects nothing; controls drop as the player narrows (all at 361 px; no time at 210; play and mute at 140; none at 60); M mutes, Esc closes with focus back on the card's play button, Enter on that button plays with sound and focuses the player; leaving at 3.0 s and hovering back shows 3.0 s at once, with no second load; the player appears only once the video has its frame;