Repository navigation
Fast Movies - #524
Open
mudhoney wants to merge 23 commits into
Open
Fast Movies#524mudhoney wants to merge 23 commits into
mudhoney wants to merge 23 commits into
Conversation
FastMovie builds a movie in one class, in the order it happens, and the Resque job calls it instead of HelioviewerMovie::build(). The frames are byte for byte those of the previous code; the old classes stay for screenshots, tiles and the status API.
…) returned a new image that nothing kept, once per frame; the frame was already flat. Same frames, byte for byte.
…me and set_time_limit(600) per decode counted the PHP process own CPUtime and could only end the worker with a fatal error, leaving the movie at "processing"; the CLIworker has no limit otherwise. Same frames, byte for byte.
…ops Both loops walked the selected bodies of an observer and then asked in_array() whether the body wasin that same list. A request with labels and trajectories draws the same frames, byte for byte.
The frames_with_selections chunks go through a Guzzle pool, six in flight (events_api_events_per_frame_concurrency), spread over the concurrency and capped at 25 timestamps a chunk so no request comes near the events server's 60 s cutoff; replies are merged in chunk order, so the result is the one the sequential path gave. Measured on events.helioviewer.org: a 300-frame day from 92 s to 21 s, a month from 123 s to 28 s, same events.
A frames_with_selections chunk that fails is requested again, up to three attempts, before the movie gives up on its events (one failed chunk and every frame comes out without events; the failures seen are a proxy 504 under load, a dropped connection, a slow TLS handshake, so the frames requests also get a 10 s connect timeout). And a chunk never spans more than a day of movie time, on both paths: the events server assembles every event alive in a request's span, so a 300-frame month now goes as 27 chunks of 11 or 12 instead of 12 chunks of 25 spanning days each. Same events, same frames; the worker log names the movie that lost its events and the chunk that was retried.
…frame is rasterised The cut-outs of an event's tints and ghost lines under its own fills were luminance masks, which librsvg renders each on a full-canvas surface (about 80 % of the rasterise time of a busy frame); they are clip paths now, and only the fill polygons whose bounding box touches the tints or ghosts take part. The SVG then covers only the part of the frame the shapes are drawn on, through a viewBox on whole pixels, and that PNG is composited at its offset instead of a full-frame PNG. The frames of a request without regions are byte for byte the previous ones; with regions, single pixels along the dashed ghost lines inside a fill differ by up to 9 of 255 (the clips and the two strokes are antialiased separately), PSNR above 79 dB.
…ry frame A movie draws the same labels on every frame at different positions, and the two annotateImage calls per line (3 px outline, then the text) were about a fifth of a busy frame. A label is now rendered once, outline then text, into a transparent image just big enough for it (EventLabelRenderer), kept per process up to 500 sprites, and composited at its position on every frame; a fractional position is still drawn straight onto the frame. Over-compositing is associative, so the frames match the direct drawing to within rounding.
An AIA layer with a brightness correction ran a gamma curve and a brightness scale over the whole decoded image on every frame, and then the colour table. The decoded image is 8-bit, so the two steps and the table are now run once over a strip of the 256 gray levels, and that strip is the table applied to the frame: one lookup per pixel instead of three passes. The tables are kept per process, by file and correction, up to a hundred. Same pixels: a 256-wide table is looked up exactly at each level, and the unit test checks the folded table against the step-by-step result for every colour table and the corrections AIA 304 gets over the mission.
…rame The watermark's layer names were drawn on every frame with two annotateImage calls, a 2 px black outline and the white text, and the outlined text goes through ImageMagick's vector path rasteriser: about 35 ms for one name, 90 ms for three, on every frame of a movie whose names never change. The names are now rendered once per movie, outline then text, into a transparent image just big enough for them (WatermarkNames), kept per process up to 50, and composited bottom left on every frame; a fractional position is still drawn straight onto the frame. The times, which change every frame, are drawn as before, with the same draws the names use. Over-compositing is associative, so a frame matches the direct drawing to within rounding, which the unit tests check for one, two and three layers.
The player uses the mp4 and Flash is gone, so the webm and the flv were encoded for nobody, and the webm was the slowest step of a movie: libvpx on one core, minutes under load. encodeMovie keeps the two mp4 encodes, a movie gets one format row, and a request for webm (status, play, download) is answered with the mp4. Fixed on the way: the mp4 conversion lacked -y, so a rebuilt movie kept its old mp4. The old build path in Movie_HelioviewerMovie and Movie_FFMPEGEncoder, which the worker no longer runs, is left as it is.
…at veryfast The frames were read twice: once for the hq mp4, then the hq mp4 was decoded again for the normal mp4, a second generation. Now one ffmpeg pass decodes the frames once and feeds two encoders, the hq (ultrafast, crf 15) and the normal (crf 18), both from the frames. From the frames the normal mp4 at veryfast has the quality the conversion gave at fast (the same PSNR and SSIM against the frames, a slightly larger file) in half the time, so the example's x264_preset is veryfast; ffmpeg_max_threads stays 8, each of the two encoders gets it. The worker's PHP memory limit is raised to 512 MB in resque_env.php: a movie holds the events of all its frames (the events and, per frame and event, the position and visibility) in memory, about 106 MB for a 300-frame movie with events before the first frame, 132 MB at its peak, over PHP's default of 128 MB, at which a queued 300-frame movie died at frame 13.
…rives A movie holds the events of all its frames until the last frame is drawn, as the events API decoded them: every footprint point a PHP array with the keys x and y, about 400 bytes for two doubles. A month of AIA 304 with the four event sources is 15,579 events with 1.58 million footprint points, 839 MB, over the worker's 512 MB; the movie died while still fetching. EventRecord now holds the fields the renderer reads and the footprint packed: per ring one string of its doubles (pack 'd', bit for bit the values the API sent) and one string of the indexes of its far-side points. An observation is 17 bytes, the two doubles and the far-side flag. The events client hands each chunk's reply to the caller as it arrives (getEventsForFramesWithSelections' $onChunk) and EventContext packs it on the spot, so one decoded chunk is in memory at a time; a client that returns the whole merged reply instead is absorbed the same way. getEventsForDate unpacks one frame's events into the arrays the renderer always had: built from the same recorded reply with main's EventContext and with this one, every frame's event list is identical, for movies and screenshots alike. The month movie's context is 70 MB.
… it up "[Movie:v966n +45.892s] frame 30/60 (50%) in 1159ms": the time since the process started on the movie, with milliseconds, so a worker log reads as a timeline of the build (events fetched at +2.6 s, halfway at +45.9 s, complete at +88.5 s) without subtracting timestamps by hand.
… from it A movie's frames were built by the object that planned them, in one loop that chose each frame's image as it went. Now planMovie() also chooses the image of every frame per layer, once, and writes the plan next to the frames (plan.json: the timestamps, the images, the frame size, rate, length and file name), and buildFrames(first, last) builds any range of frames from that plan in an object that reads it back from disk and fetches the events of its own frames only. run() builds the whole movie as one such range, in a second object, so the frames come out of the same path a worker of its own will take. A failed frame leaves a gap in the numbering instead of shifting the frames after it; closeFrameGaps() renumbers at the end, so the files are the ones the old loop made. The previews are made by the range that builds the plan's middle frame, while it is in memory; when that frame failed, the end of the build makes them from the last frame's BMP, where the old loop had none. Same frames and previews on every request, the one with a frame made to fail included.
…t has pieces A movie long enough is cut into pieces of consecutive frames (at least movie_piece_min_frames each, at most movie_pieces_max of them, both in Config.ini: 8 pieces of 37 or 38 frames for 300 frames), each a Job_MoviePiece on the piece queue, which the workers read before the movie queue so a movie in progress is finished before a new one is started. The movie job plans, writes the plan and queues the pieces; a piece builds its range from the plan on a worker of its own and reports its frames and its end to the movie's record in Redis (Pieces); the piece that ends last queues the end of the movie, the movie job again with stage "finish": the gaps of failed frames closed, the previews if no piece made them, the encode, the statuses, the frames deleted. A piece that failed fails the movie. The status API reads the progress from the record, since the pieces write their frames in no fixed order, and fails a movie no piece has reported on for ten minutes. A short movie, or one the tests build in-process, runs its pieces one after another in the movie job and ends the same way, so the harness still builds in one process. On the 8-core dev box a 60-frame movie with events went from 93 s to 30 s and a 300-frame one from 476 s to 121 s, and the month movie with the four event sources from 326 s to 82 s.
…n the pieces A label's outline was a 3 px stroke, which ImageMagick 6 draws through its general path rasteriser, walking the whole canvas for every glyph: 30 to 58 ms a label. A movie whose labels repeat paid it once per label; a month of coronal holes, magnetic connectivity and flare predictions, whose labels carry dates and numbers, pays it on nearly every frame (50 to 80 new labels), and that was 1.3 to 4 s of a 2.3 s frame, the whole difference between the pieces (105 s) and the old branch's frame jobs (58 s) on the same movie. As on that branch: the glyphs are drawn once as a mask, grown by one and two pixels (a dilation over the small sprite), faded to the darkness the stroke left at those distances, laid over each other with the white text on top, 5 ms where the stroke took 58, and the same look to within antialiasing. The sprites are also shared through a folder next to the frames, as exact 16-bit MIFFs with a lock per label, so the pieces of a movie render each label once between them. The frame log line now carries the time of each step of the frame (layers, composite, events with list, regions and labels, overlays, write), which is how this was found.
…s the sprites too When the viewport follows the Sun's rotation, each frame's markers are shifted by the viewport's drift, a fractional number of pixels, and the marker and label positions came out fractional. ImageMagick truncates a fractional marker position and PHP 8.1 prints a deprecation for every marker, thousands of lines of noise in a movie's log; and a label at a fractional position could not be a sprite, so the renderer drew it straight onto the frame with the 3 px stroke, 415 ms a label on a 1708×848 frame where a sprite is 6 ms the first time and 0.2 ms after. A month movie with labels on every source and a following viewport, a few hundred labels a frame, spent 55 to 128 s a frame in its labels, 30 pieces on 64 cores at 16% after twenty minutes, as it would on main. The positions are now rounded to whole pixels after the drift is applied, so a following movie draws its labels as sprites with the halo, like every other movie since the previous step, and its markers land on the nearest pixel instead of the truncated one, at most one pixel away. Movies whose viewport does not follow are byte for byte the same. The shared-video markers are truncated explicitly, as ImageMagick did, without the deprecation. The same month movie from the queue, 30 pieces on 64 cores: 20.5 s.
A worker registers its name (host, pid, queues) in Redis when it starts and removes it when it is shut down cleanly. A container that is stopped or recreated kills its workers before they can, and their names stay: a worker prunes only the dead names of its own host, and a recreated container has a new host name, so every recreate left a container's worth of names behind (138 on the dev box after a resize and a reboot). The queue does not read them, but movie_queue.tcsh counts them as workers when it decides whether to start any, matching their pids against this container's processes, and a Resque dashboard shows them as workers. A small script run by movie_queue.tcsh before the count removes the names of every other host and the dead ones of this host, assuming one movies container per Redis, which is how the movie host runs. Two unit tests: another host's name and a dead pid of this host go, a running process whose command line says resque stays; nothing to prune is not an error.
Since b3eabea EventRegionSvg's document covers only the part of the frame the regions are drawn on. FastMovie rasterises it at that size and places it at that offset; the screenshot composite still rasterised it at the full canvas size at the top left corner, so every region in a screenshot with events was stretched and shifted. The composite now does what the movie does. Checked against main on three screenshot requests with the same events: the regions differ by at most 7 of 255 on a few hundred edge pixels (the clip paths of b3eabea); without events the bytes are the same.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds movie generation around one file, src/Movie/FastMovie.php, and builds a movie's frames in pieces on several Resque workers.
Movie code
Pieces
the encode and statuses.
Changes for Config.ini
Worker configuration