How Hite works

What Hite is

Hite is a video editor with two doors into the same timeline. You can move a clip by hand, or you can describe the change in plain language and a model does it. Both routes emit the same typed commands, run through the same pure reducer, and land in the same edit decision list. There is no separate AI mode and no second data path.

The model never touches pixels and never generates footage. It calls analysis and advisory tools against your media, then emits a batch of edit commands. What comes back is an ordinary timeline you can keep editing by hand, and every AI turn sits on the same undo stack as your own edits.

The spine

Five modules in a line. Everything else in the codebase is a leaf hanging off one of them, and the layering is the main thing a contributor is asked not to break.

A command is a typed intent. The reducer is a pure function that applies a whole batch as one transaction. The EDL is the single source of truth for what the video is. The compiler turns that into a render IR, a frame-space description with content hashes. The Remotion composition paints the IR, in the browser for preview and in the worker for export.

The rule that keeps this honest: edlToRenderIR is imported by both the preview and the export, never reimplemented. Preview and export disagreeing is a class of bug this architecture removes rather than fixes.

The timeline: EDL.2

The edit decision list is a Zod schema tagged Edl.2, with every node carrying its own version tag (Clip.1, Track.1, and so on) so a single node can be migrated without a whole schema bump.

Effect windows are absolute timeline ticks, not clip-relative. Volume keyframes are the exception and are node-relative; the compiler is the single place that crossing happens.

The command union

Twenty-four variants in two groups. Fifteen the model may emit, nine the editor dispatches but the model never sees. A batch is at most forty commands with a required summary, which is one turn's worth of edits rather than a provider limit.

Emitted by the model

Editor only

SET_CLIP_VOLUME · REMOVE_EFFECT · REMOVE_TRANSITION · REMOVE_OVERLAY · REMOVE_CAPTION · REMOVE_AUDIO_BED · CLEAR_LOOKS · SET_CAPTION_STYLE · ADJUST_WORD_TIMING

Each applied command is wrapped in an envelope for the audit log: a ULID, a batch id, a monotonic sequence number unique per session, the source (ai, user or seed), and an optional rationale. That ULID is the only randomness in the system, and it is minted at the boundary so nothing inside the reducer is non-deterministic.

The reducer

reduceBatch(edl, commands, options) returns a new EDL plus forward and inverse patches. It performs no I/O, reads no registry and consults no clock. That is enforced by a lint rule, not by convention: Math.random, Date.now, crypto.randomUUID, a bare new Date() and setInterval are all banned inside the reducer, the compiler and the composition.

A batch is one transaction, in this order:

1. Every command is applied in array order to a single draft.
2. Each track is normalised: adjacent gaps merge, zero-length gaps are dropped, trailing gaps are removed.
3. Transitions are pruned if either clip vanished or the two are no longer strictly adjacent, and clamped (not dropped) if longer than the shorter neighbour.
4. The duration is recomputed from the longest track and the furthest overlay, caption or bed.
5. The revision is bumped, then the whole result is re-parsed against the schema as a tripwire.
6. Range sanity and clip-id uniqueness are asserted, then the content hash is recomputed.

Anything that throws escapes the draft, so the batch rolls back whole. Errors are typed and named: trim_collapses_clip, transition_not_adjacent, clip_exceeds_media, degenerate_window, and a dozen more. The editor shows the message verbatim rather than a generic failure.

Removals are deliberately asymmetric. Removing a clip or an effect by an unknown id throws, because the caller has lost track of the timeline. Removing a transition, overlay, caption or bed that is already gone is a no-op, because those are decorations and a repeated delete should be safe.

Identifiers are content-addressed rather than random. Effects, overlays, looks, transitions, captions, beds and markers get a hash of their own content; clips get a hash of their lineage. Reseeding an id therefore never busts the render cache, and identical subtrees deduplicate.

Time, frames and determinism

Time is an integer count of ticks at 30,000 per second. No float ever enters the EDL, the render IR or a hash. That rate is frame-exact at 24, 25, 30, 50 and 60 frames per second and also millisecond-exact, so ticks convert to frames by exact integer division at every supported rate.

There is exactly one ticks-to-frames conversion in the codebase, and it lives in the compiler. For the degenerate case of a frame rate that does not divide the tick rate it rounds half to even, so a frame boundary is identical on every machine — a hard requirement for content-addressed rendering.

Frame rate is not stored in the EDL. It is resolved from the first clip whose asset reports a usable rate, defaulting to 30. That resolution lives in one file specifically because a preview and an export disagreeing about frame rate is otherwise very easy to reintroduce.

History and undo

Two things are recorded per edit: the semantic command batch, for the audit log and for replay from a seed, and the inverse patches, which are what undo actually applies. Synthesising a precise inverse command for every operation would be strictly more fragile.

A drag coalesces: successive commands carrying the same key amend the top entry rather than stacking, so one undo reverts the whole gesture. An AI turn arrives as a snapshot — the server already reduced and persisted, so the client records the diff to that exact EDL rather than re-running the reducer and risking a different result.

Both routes push onto one stack held by one controller. That is the whole mechanism behind undoing an AI edit: there is nothing special about it to undo.

The render IR

edlToRenderIR(edl, env, resolver) is synchronous and pure given its three arguments. Signing URLs and reading the database happen at the boundary and arrive through the resolver, which is what lets the same function run in a browser and in a worker.

The IR materialises exactly one frame space: absolute timeline frames. A stack of track nodes, each holding clips, gaps, transitions, overlays, captions and audio, every one carrying both its tick and its frame form. Nodes are hashed bottom-up with SHA-256 over a stable stringification, with identity fields excluded by construction so that identity and cache key stay orthogonal.

Resolution is a function of aspect and quality rather than a preset table:

Exports are H.264 in an MP4 container, always at full quality. When the compiler cannot honour something — a look recipe step whose variable was never supplied — it drops that step and records a diagnostic rather than substituting a zero. Diagnostics deliberately do not affect the hash.

One renderer

A single Remotion composition paints the IR. The browser preview mounts it in a player; the worker selects the same composition by id and renders it to a file. Both compile the IR with the same function and the same resolver, differing only in an engine fingerprint string.

Each clip becomes a sequence containing a video or image, wrapped by whichever effect renderers are registered for it. Registration, not engine type, is the gate: an effect with no renderer is skipped rather than faked. Transitions paint in a second pass on top of the clips so the treatment covers both sides of the cut.

The planner

A turn is a tool loop with a grounding phase, a terminal emit, and — above the lowest effort setting — a critique round in which the model is shown what its own batch actually did and given the chance to revise it.

Grounding first

For the first few steps the emit tool is withheld and a tool call is required. That constraint is load-bearing rather than stylistic: the loop only continues after a step that made a call, so a grounding step answered with prose would end the turn with no batch at all. The effect is that a plan is built on what the tools actually returned, not on what the request implied.

The critique round

The emit tool does not simply accept the batch. It runs it through the real reducer in process — no database, no network — and hands back the resulting timeline, a description of each command as applied, the before and after durations and clip counts, and a set of server-measured checks. Those checks are blunt on purpose:

• the batch changed nothing at all, by content hash;
• the timeline now has no clips;
• a requested length versus the length actually produced, as a percentage over or under;
• a transition the reducer silently dropped or shortened because the clips were not adjacent;
• how many clips a grade actually reached.

The model either revises or confirms by re-emitting the same batch, and confirmation is tracked by content hash so a reworded repeat is not mistaken for agreement. A batch the reducer rejected always gets a repair round. This is the largest quality lever in the layer, because the feedback is measured rather than imagined.

Effort

Four settings, chosen per request, controlling reasoning budget, step count, revision rounds, grounding steps and a wall-clock ceiling. The highest rung is bounded by wall clock rather than by steps, so a slow-thinking model gets fewer critique rounds, not more. The ceiling is also derived from what the provider can actually do: a provider that cannot be forced to call a tool cannot be trusted with the grounding phase, so its effort is capped rather than silently ignored. Deployers can cap it further.

Everything is visible

The turn streams as events: the tool name, its arguments and its result; warnings; the resulting EDL; a summary; the saved edit. The editor renders the real tool names and counts read off the tools' own returned arrays. A plan built on an empty transcript looks different from one built on a real one, because tools that find nothing say so explicitly rather than returning a bare empty list that reads like a clean bill of health.

The tool library and the router

Twenty-one tools, each in one file, each declaring a capability tier and a line about when to reach for it. Model tool-selection accuracy degrades once too many tools are in play, so the router exposes only the tiers a request actually touches, keeping any single turn under that ceiling.

Tiers are matched from the request by keyword stems, written as stems rather than whole words so that "caption" also catches captions and captioning. A compound request keeps every tier it touches rather than the strongest one. A request that matches no tier at all falls back to the tiers that a vague ask ever turns out to mean: trim it, or restyle it.

The flat schema

The command union uses tuples, records and unions, all of which some providers' function-calling layers reject outright. So the model is not handed that union. It emits a deliberately flat, shallow schema — strings, numbers, booleans and enums, nothing nested — which is then mapped onto the real command union before it reaches the reducer.

This is the ceiling on what a model can express in one turn, and it is a real one: an expressiveness limit in the flat schema is felt as the model being unable to ask for something the editor can do by hand. Widening it is the single highest-leverage change for output quality, and it costs compatibility with the strictest providers.

Models and keys

Hite is bring-your-own-key. The key travels on the request, is used for that request, and is never pooled on a server you do not control. A request without a key is refused rather than quietly routed somewhere else. Self-hosted OpenAI-compatible endpoints are first-class, so a local model is a supported configuration rather than an afterthought.

The provider registry is data and imports no vendor SDK, which is what keeps the browser bundle free of them; one module does the dynamic import for whichever provider a request actually names.

Eight providers ship with the registry: Google, OpenAI, Anthropic, Groq, xAI, DeepSeek, OpenRouter, and a self-hosted OpenAI-compatible endpoint. Reasoning effort is normalised to five intents and translated per provider from that provider's own published option shape; an intent a provider cannot express is dropped rather than approximated with something nearby.

The key is treated as hostile-adjacent data throughout: header only, never a body or a query string, validated by length and character class rather than by a format guess, and redacted from every user-visible string in both raw and percent-encoded form. That redaction exists because a provider's own error body for an invalid key can contain the key, and the SDK's default error logger prints the whole body.

The renderable gate

The effect catalog advertises more than the renderer can paint. Rather than let a model emit a key that quietly does nothing, everything model-facing is filtered through one gate that asks whether this build can actually render that entry, and anything withheld comes back with the reason attached.

This is why the numbers on this property are lower than the catalog size, and why they differ from each other: what the renderer can paint and what the landing catalog lists are two different questions with two different answers. The catalog section shows 39, scoped to the categories it covers.

The most visible consequence: transitions that need two clips' pixels blended together are withheld. What ships is a set of boundary treatments — a dip through black, a flash, a burn, a chromatic cut, a whip — which are faithful to their names. A cross dissolve is not among them, because v1 cannot blend two clips, and calling a dip to black a cross dissolve would be a lie told by the software rather than by a person.

Colour LUTs render as filter approximations rather than true 3D-LUT sampling, and this page says so for the same reason.

Jobs, the worker and analysis

Analysis and export are queued in Postgres and drained by a second process. It is a real queue, not a convention: claiming a job is a single statement that locks a row and skips ones already taken, so two workers never collide.

What the worker guarantees

• Heartbeats and a reaper. A worker that dies has its jobs requeued after a stale window, and the reaper runs once at boot as well as on a timer.
• A per-job deadline. A wedged handler heartbeats exactly as diligently as a working one, so the reaper alone can never catch it. Each job carries its own timeout and is abandoned when it expires.
• Fencing. Every terminal write is conditional on still holding the claim, so a worker that lost its job cannot overwrite the outcome of the worker that took it.
• Idempotency. Analysis rows upsert on asset and kind; an export overwrites a stable path. A repeated job is therefore safe, which is what makes requeueing safe.
• Graceful shutdown. On a signal it stops claiming, lets in-flight work finish, and releases anything unfinished back to the queue rather than leaving it to time out.
• One render at a time, because Chromium painting a 1080p composition is the heaviest thing the process does. Scale by running more workers, not by raising the number.

What analysis produces

Four things are computed and stored: a probe of the media, a transcript, a tempo and beat grid, and scene boundaries. Each branch persists as soon as it succeeds, and failures are collected and reported together, so one bad audio stream does not cost you the scene detection. A probe failure is not a branch — it fails the whole job, because everything downstream depends on knowing what the media is.

Silence and filler words are not separate stages. They are derived from gaps and tokens in the transcript at the moment a tool asks for them. That has a consequence worth stating plainly: without a transcription key there is no transcript, and therefore no silence detection, so the request Hite is best known for cannot be served. The tools say that rather than returning an empty list.

Data model, storage and RLS

Projects own assets and edits. Assets own transcripts and analyses. Edits own exports. Jobs point at an asset or an export, never both, enforced by a check constraint. One live analysis job per asset is enforced by a partial unique index, which removes duplicate work as a class rather than by convention.

Row-level security is the boundary, and it is written as joins back to ownership rather than trusting an application layer. You can read a transcript only if its asset belongs to a project you own. Clients may read jobs and never write them: jobs are created by server routes after an ownership check, and the worker bypasses the policy entirely with a service role that never reaches a browser.

Rate-limit and budget tables have row-level security enabled with no policies at all, which denies every client role. That is deliberate defence in depth against a leaked public key.

Storage is three buckets: private media, private exports, and a public bucket for preview clips. Object policies key on the first path segment being your own user id, so a file at the bucket root is reachable by nobody, which is the intended default.

Sessions without accounts

There is no sign-in screen and nothing asks for an email. The first request to the editor mints an anonymous session, and row-level security scopes everything to it. The trade is worth stating plainly: that session lives in your browser's cookie and there is no account to recover it with, so a different browser starts you on an empty workbench. Export what you want to keep.

Running it

Two processes. The web app, and the worker that drains the queue. Without the second one, uploads and the editor work but every analysis and export sits queued.

git clone https://github.com/Imdevsup/hite
cd hite
pnpm install

# the app
pnpm dev

# a second terminal: the queue drainer
pnpm worker

A local Supabase stack supplies Postgres and storage; the migrations apply on start. Rendering needs a Chromium that Remotion can drive, and ffmpeg is vendored. The repository's readme carries the exact steps, including the two that reliably bite: the Supabase CLI's install is blocked by a build-script allowlist and needs one manual command, and this project's local ports are not the defaults.

Gates

pnpm typecheck, pnpm lint and pnpm exec vitest run. The integration suites need a local Supabase stack and skip without one; they cover the async backend against a real database, including a real render to a decodable file and a worker killed mid-render being reaped.

Extending it

A new tool

One file exporting a spec with a name, a tier, a line about when to use it, and the tool itself. Add it to the generated index. Nothing in the planner or the reducer changes. To make a phrase reach it, widen that tier's keywords rather than the fallback set, which is sized to sit just under the accuracy ceiling.

A new edit command

Four coordinated changes, and skipping one is the usual way this breaks: a variant in the command union, a case in the reducer, render support, and an entry in the flat mapper so a model can actually emit it.

A new effect

An entry in the catalog and a registered renderer. The build fails if a clip effect exists with no renderer behind it, which is the gate that stops the catalog drifting ahead of the picture again.

What is not there yet

This section exists because a technical document that only lists strengths is not a technical document.

• No segment cache. The IR is hashed for one and the key is defined, but nothing calls it. Every export renders from scratch.
• No true cross dissolve.Transitions are boundary treatments; blending two clips' pixels is not implemented, and the transitions that would need it are withheld from the model.
• No face tracking. Cut for being Python-only. Face-anchored placement degrades visibly and records a diagnostic instead of guessing.
• LUTs are approximations. Filter-based, not 3D-LUT sampling.
• One export format.1080p H.264 in an MP4, at the encoder's defaults. No bitrate or codec controls are exposed.
• Transition length does not shorten the timeline. Clips are laid strictly end to end and a transition treats the boundary; the offsets are computed for the future work and currently unused.
• The flat schema bounds the model. Anything it cannot express, the model cannot ask for, however capable the model is. In particular the vocabulary is additive: there is no command to weaken, retune or remove an effect that a previous turn added.
• A turn has no memory. A refinement is sent with the current timeline and the new request, not with the conversation. Multi-turn reasoning is structurally unavailable.
• The model cannot see the picture.There is deliberately no frame-sampling tool: an earlier one passed a URL as text and returned the model's guess as though it were an observation, which is worse than nothing. Real vision needs real frames, and that work has not been done.
• No model has been measured. The verification harness has never been run, so every provider badge reads untested and no ranking of models exists.

Every one of these is visible in the code with the reason attached. If you are evaluating Hite, these are the things to weigh; if you are contributing, they are the most valuable places to start.