A tools company shipped video-to-video, then text-to-video

The process wall

In February 2023, Runway published a research page for Gen-1, a model that takes an existing video's motion and applies a new look drawn from a reference image or a text prompt. The page describes five modes: stylization, a mask-guided storyboard render, mask-based replacement, turning an untextured render into a finished look, and a customization mode for higher fidelity. Before Gen-1, a maker who wanted a video restyled had to rotoscope, key frames by hand, or hire a compositor shot by shot. Gen-1 replaced that manual pass with a model conditioned on the source footage's own structure, so an edit followed the existing camera movement and timing automatically rather than being redrawn.

What the documents show

Weeks later Runway published a second page for Gen-2, describing a multimodal system that adds a mode with no source video at all: text to video. The page lists eight modes in total, keeping Gen-1's video-to-video options and adding text-only and text-plus-image generation. Both pages cite the same user study: results 'preferred over existing methods' by 73.53 percent against Stable Diffusion 1.5 and 88.24 percent against Text2Live, for image-to-image and video-to-video translation. That figure is Runway's own comparison, run and reported by the company that built the model, and neither page states a sample size, a rater pool, or a testing protocol.

Authorship and limitation

The mechanism worth naming is conditioning, not invention: Gen-1's stylization and render modes still require a maker's own source video to supply motion, framing and duration, and the model replaces surface material and style on top of it. Only Gen-2's text-to-video mode removes that footage requirement entirely. The 'customization' mode on each page is the clearest limit stated: reaching 'even higher fidelity results' required additional footage and processing per project, meaning consistent output at a usable standard was not free at the model's default setting. That is the closest either page comes to naming a cost for a few seconds of coherent motion in 2023: paid in extra source material and training time, not in a published currency figure.

What survives

A maker keeps authorship of the underlying shot list, camera moves and reference imagery in every video-to-video mode; the model's contribution stays confined to the style applied on top of that footage. Editorially, a producer evaluating either generation should treat the preference percentages as the vendor's own marketing evidence rather than a controlled result, since neither page names its raters, prompt set, or comparison method.

• Compare Gen-1's five listed modes against Gen-2's eight to see exactly which capability text-to-video added.
• Ask a vendor quoting a user-preference statistic for the study's sample size and any independent replication.
• Check whether a 'customization' or fine-tuning tier is required to reach the fidelity shown in a vendor's own examples.

Read together, the two pages mark a specific transition rather than a single leap: a company that shipped conditioned editing of existing footage first, and unconditioned generation from text second, inside one research programme.