An agency's video editor gets forty minutes of takes back as a finished vertical cut, with every restart and dead pause already gone.
Forty minutes of takes, mapped by silence. It keeps the last one.
The situation
How this used to work
A short video for TikTok or Reels starts as forty minutes of somebody talking to a camera, and almost all of that is the same sentence said four times.
Restarts are everywhere. Somebody begins a line, stumbles on a word, and starts it again. The work of turning that into ninety usable seconds is finding, by ear, which take of each line was the clean one, and then closing every gap between them.
The trap is that reading the transcript does not show you the problem. A whole-file transcription quietly merges identical restarts into one tidy line, so the duplicate is invisible on the page and perfectly audible in the video. That is the single most common way a doubled line survives all the way to a published post.
What we built
The system
One command that owns the whole of short form, from raw footage to a finished vertical file. It replaced three separate flows that were never really three things.
Underneath it the rule is: waveform first, transcript second. The edit is derived from the actual audio, because the transcript lies in two specific ways. It merges the restarts, and it drops or garbles the trailing words of a line.
The machine is fixed and the plan is not. Everything mechanical, the cutting, the rendering, the captions, the encode, is settled and never argued about again. What gets authored per video is the scene plan, and that is the one thing a person and the system decide together.
How it works
The pipeline, step by step
The pauses, before the words
The audio is measured for loudness every ten milliseconds and every silence longer than a fifth of a second is written down. That pause map, not the transcript, is what the edit gets built from. Each speech burst is then transcribed on its own rather than in one pass, precisely so two attempts at the same sentence stay two entries instead of collapsing into one.
Keep the last complete take
Every kept range has to be a whole sentence with both of its ends inside a real pause in the waveform. Where a line was said more than once, the last complete attempt wins. Every silence longer than eight tenths of a second is closed. That is the entire afternoon of scrubbing back and forth, done in one derivation.
The gate, and who it is protecting
The cut is then checked by a separate pass that hunts for exactly what the edit is most likely to have got wrong: restarts left in, overlaps, stutters, and any surviving dead air. It reports clean or it reports the timestamps. The instruction written next to it is blunt about whose job this is, and it says the clipped tail gets caught here, never by Andreas.
The one thing decided by a person
A scene plan is drafted as a table, beat by beat: what is being said, what is on screen, and whether his face is full, tucked into the corner, or off entirely. The heuristic is that the face belongs where he is the value and the screen belongs where the screen is the value. He vetoes and adjusts, and then it is locked. This is the single approval checkpoint in the whole pipeline.
The one table a person has to agree with before anything renders.
Rendered once, encoded once
Scenes are rendered and laid over the base with each window overlapping the next slightly, so the underlying footage never flashes through between them. Captions are burned in phrase by phrase, short enough to read, inside a safe band that never crosses his face, with a correction dictionary for the words transcription reliably mishears. Then one encode pass, not a chain of them.
Captions live in the lower band. The face zone is never overlaid.
Colour and sound are not ours to touch
The grade is deliberately not automated, and neither is the audio mix. Andreas does both in CapCut, and the system is built to leave exposure, contrast, saturation and levels exactly where it found them. The pipeline adds layers, scenes and captions, and it stops precisely where the taste starts.
What changed
The result
Three hundred and fifty three vertical videos sit in the studio, counted by measuring the dimensions of every file rather than by trusting a filename, out of four hundred and forty video files in total. Most of those are working versions rather than finished pieces: one build alone accounts for fourteen of them.
The revision loop is the thing that actually changed. Chasing flagged duplicates and clipped tails one at a time across a v1, a v2 and a v3 never settled and frustrated everybody. The rule now is to stop patching and re-derive the whole edit from the pause map in a single pass, which is what turned one of those churning builds into a version he called finished.
The screen recordings are still recorded by hand, from a shot list, and never faked or auto-captured. Where an asset is missing the pipeline stops and asks rather than substituting something close. Neither of those is a limitation anyone has got around to removing.
The stack
Cut
- RMS silence mapping
- per-burst transcription
- whisper large-v3-turbo
- ffmpeg
Compose
- ffmpeg scene templates
- Remotion
- HyperFrames
- burned-in captions
Faces
- HeyGen lip-sync
- ElevenLabs for localisation only
Left alone
- colour grade
- audio mix
- screen recordings