Video API

Six operations cover the video surface. Every endpoint is async (returns a VideoJob), the provider-backed ones accept a provider field to pin an engine, and every endpoint emits a webhook on completion when webhook_url is set.

text

POST /v1/video/generate     text → video
POST /v1/video/avatar       image + audio (or text + voice) → talking-head video
POST /v1/video/lipsync      video + audio → lipsync'd video
POST /v1/video/translate    video → captions (SRT/VTT or burned in) and an optional dub in another language
POST /v1/video/dub          video → dubbed in another language, lipsync preserved
POST /v1/video/analyze      video → embeddings, transcription, semantic Q&A
GET  /v1/video/:jobId       poll job status

See Providers → Video for the full engine matrix.

Generate

Text-to-video. Render up to 30 seconds of generated video from a prompt.

POST /v1/video/generate

Request

json

{
  "prompt": "A barista pulling a perfect espresso shot, warm morning light, 35mm film",
  "negative_prompt": "blurry, distorted faces",
  "duration_seconds": 6,
  "width": 1280,
  "height": 720,
  "fps": 24,
  "provider": "runway",
  "webhook_url": "https://your-app.com/hooks/video"
}

Field	Type	Default	Description
`prompt`	string	required	1–2,000 chars.
`negative_prompt`	string	—	What to avoid.
`duration_seconds`	number	`5`	1–30.
`width`	int	`768`	256–1920.
`height`	int	`512`	256–1080.
`fps`	int	`24`	8–60.
`provider`	enum	auto	`sora`, `runway`, `ltx-video`, …
`webhook_url`	url	—	Notified on completion.

Response `202`

json

{
  "id": "vj_01HM…",
  "type": "generate",
  "status": "pending",
  "provider": "runway",
  "created_at": "2026-05-05T18:00:00Z",
  "owner_id": "user_…"
}

Poll GET /v1/video/:jobId or wait on the webhook.

Avatar

Talking-head video from a single still image plus audio. Pre-render the audio yourself, or chain through TTS in one call.

POST /v1/video/avatar

Request — pre-rendered audio

json

{
  "image_url": "https://uploads.example.com/headshot.jpg",
  "audio_url": "https://uploads.example.com/script.mp3",
  "provider": "heygen"
}

Request — TTS chaining

json

{
  "image_url": "https://uploads.example.com/headshot.jpg",
  "text": "Welcome to our launch. We're shipping in three weeks.",
  "voice_id": "rachel",
  "tts_provider": "elevenlabs",
  "provider": "hallo3"
}

Field	Type	Notes
`image_url`	url	Required. JPG/PNG. Face must be visible.
`audio_url`	url	Provide this or `text` + `voice_id`.
`text`	string	1–10,000 chars. Synthesizes audio first.
`voice_id`	string	Library or cloned voice. See /voices.
`tts_provider`	string	TTS engine. Default `default`.
`target_language`	string	If set, translates `text` before synthesis.
`provider`	enum	Avatar engine: `hallo3`, `heygen`, `liveportrait`, `echomimicv2`, `v-express`, `skyreels-a1`.

Self-hosted engines (hallo3, liveportrait, echomimicv2, v-express, skyreels-a1) run on ph0ny GPUs and are billed per second of generated video. heygen requires BYOK.

Lipsync

Re-align lip motion in an existing video to a new audio track. Useful for dubbing, ADR, fixing flubbed takes.

POST /v1/video/lipsync

json

{
  "video_url": "https://uploads.example.com/clip.mp4",
  "audio_url": "https://uploads.example.com/replacement.mp3",
  "provider": "sync-labs"
}

Provider	Notes
`sync-labs`	Best commercial fidelity. BYOK.
`heygen-lipsync`	If you already use HeyGen.
`latentsync`	Self-hosted, open-source.
`video-retalking`	Self-hosted, robust on noisy footage.
`musetalk`	Realtime; lower-latency at lower fidelity.

Translate

Turn a video into another language: captions, and optionally a dub of each speaker over the original.

POST /v1/video/translate

json

{
  "video_url": "https://uploads.example.com/interview.mp4",
  "target_language": "en",
  "captions": "burn",
  "captionStyle": "viral",
  "cover_source_captions": true,
  "audio": "original"
}

Or upload the file (up to 100 MB): multipart/form-data with the video as file and the same fields JSON-encoded in request. Videos up to 10 minutes. Jobs run one at a time on ph0ny's render host; when its queue is full the request is refused with 503 CAPACITY_UNAVAILABLE. DELETE /v1/video/:jobId cancels a job that has not finished, and a cancelled job is not billed.

ph0ny will:

Transcribe a default single-speaker job on ph0ny's CPU Whisper service with word timestamps, then cut caption cues at sentences and pauses. For requested multi-speaker diarization (speakers greater than 1 or voice: "speakers"), supply your own ElevenLabs key; Scribe supplies speaker labels and the cues also break at speaker changes.
Brief the translator on the whole transcript (setting, speaker roles, how each name is written), then translate cue by cue, condensed to fit each cue's time.
Return SRT and VTT (two lines of at most 42 characters; a rapid exchange becomes one dashed two-speaker cue).
With captions: "burn", draw them into the picture. captionStyle: "viral" shows 1–3 words at a time; "karaoke" highlights the active word. Translated words are timed proportionally inside their source cue; dubbed speech uses matching word timestamps from the isolated dub track. cover_source_captions paints a solid band over the bottom fifth first, hiding captions already burned into the source.
With audio: "dub", speak each speaker's lines over the ducked original audio (music and room tone come back between lines), sped up or shortened at a pause where a line would overrun its slot.
With lipsync: true (needs audio: "dub"), lip-sync the picture to the dub.

Field	Default
`target_language`	—	ISO 639-1 code or English name.
`source_language`	detected
`reviewed_source_sha256`	—	Exact input SHA-256 from the prior job's `result.translation.source_media_sha256`. Supply with `expected_start_seconds` and `expected_end_seconds` on each override to bind corrections to the same source bytes and cue time window.
`translation_overrides`	—	Reviewed target text for source cue indices from a prior job. Each item has `source_cue_index`, `expected_source_text`, and `text`. Without media binding, the indexed cue must match exactly or be the only close match after bounded transcription drift. With media binding, each item also needs both cue times from the prior job. The cue must overlap at least 75% of both windows, and each endpoint must stay within one second. Text must fit its speech window. A second job repeats transcription and billing.
`captions`	`burn`	`burn` or `none`. SRT and VTT come back either way.
`captionStyle`	`plain`	`plain` keeps the existing broadcast look; `viral` uses bold short chunks; `karaoke` highlights the active word. Applies to burned captions only.
`caption_position`	`bottom`	`bottom` or `top`.
`cover_source_captions`	`false`
`audio`	`original`	`original` or `dub`.
`voice`	`neutral`	`neutral`: a stock voice per speaker in their register (from the source's grammar, else their pitch). An OpenAI voice name (`onyx`, `coral`, …): one voice for everyone. `clone`: your consented clones from `speaker_voices`.
`speaker_voices`	—	`{ "speaker_0": "<voice id>" }`, per diarized speaker.
`speakers`	1	Expected number of speakers. More than 1 requires a caller-supplied ElevenLabs key for diarization.
`lipsync`, `lipsync_provider`	`false`

Cloned voices need consent. ph0ny clones only your own voice, with your consent recorded on the clone (POST /v1/voices/clone with consent: { subject: "self", statement }). A video's speakers are never cloned from the video. voice: "clone" without a consented clone in speaker_voices is refused with 400 CONSENT_REQUIRED.

The job's result.translation carries source_media_sha256, captions, cues with timing and speaker, source cues, each speaker's role, gender and dub voice, the translator's brief, and for dubs a dub_fit report. result.video_url is the rendered video when captions were burned or the audio dubbed. stage on the job reports the step in progress.

Billed per step: speech recognition per minute, translation per token, dub speech per character, and the render as one ph0ny-render video job. A styled burned dub uses one additional speech-recognition pass on its isolated dub track and counts that duration in usage.

Dub

The translate pipeline with a lip-synced dub and no burned captions. One call.

POST /v1/video/dub

json

{
  "video_url": "https://uploads.example.com/launch.mp4",
  "target_language": "es",
  "lipsync_provider": "sync-labs"
}

voice_id takes a voice as voice does on translate (default neutral): an OpenAI voice name or your consented clone's id; other vendor voice ids are refused with 404 VOICE_NOT_FOUND. tts_provider is ignored. If source_language is omitted it is detected. The source is fetched by ph0ny (up to 100 MB, 10 minutes), result.provider is ph0ny-render, and the job carries result.translation like a translate job. Needs a lip sync provider; without one the request is refused with 503 PROVIDER_UNAVAILABLE.

Analyze

Multimodal video understanding — semantic search, transcription, scene detection, Q&A.

POST /v1/video/analyze

json

{
  "video_url": "https://uploads.example.com/meeting.mp4",
  "provider": "twelve-labs",
  "tasks": ["transcribe", "scenes", "embed"]
}

Use cases:

Search across hours of footage — embed once, query in natural language. Pairs with Collections.
Auto-chapter long videos — scenes returns timestamps + descriptions.
Q&A over recorded calls — pass the analyze output as agent context.

Provider	Strength
`twelve-labs`	Best commercial — Marengo + Pegasus. BYOK.
`internvideo2.5`	Self-hosted; long-form.
`qwen2.5-vl` / `qwen3-vl`	Self-hosted; charts and screen capture.
`videochat-flash`	Realtime conversational video Q&A.

Cost estimate

GET /v1/video/estimate?capability=generate&provider=runway&duration=6&resolution=1080p

json

{
  "estimated_credits": 270,
  "estimated_cost_usd": 0.27,
  "usd_per_credit": 0.001,
  "pricing_tier": "list",
  "provider": "runway",
  "capability": "generate",
  "base_credits": 150,
  "duration": 6,
  "resolution": "1080p"
}

Credits are valued at the published rate card, so usd_per_credit is the same figure GET /v1/pricing/rate-card reports and the same one a charge uses. The quote is given at the list tier, which is the highest of the three, so it is never below what you are billed.

Job lifecycle

Every endpoint returns:

json

{
  "id": "vj_…",
  "type": "generate" | "avatar" | "lipsync" | "dub" | "analyze" | "translate",
  "status": "pending" | "processing" | "completed" | "failed",
  "provider": "runway",
  "progress": 0.42,
  "result": {
    "video_url": "https://r2.ph0ny.com/jobs/vj_…/output.mp4",
    "duration_seconds": 6.0,
    "width": 1280,
    "height": 720,
    "provider": "runway"
  },
  "error": { "code": "provider_timeout", "message": "…" },
  "created_at": "2026-05-05T18:00:00Z",
  "completed_at": "2026-05-05T18:00:42Z",
  "owner_id": "user_…"
}

duration_seconds, width and height are measured from the file the job produced, not from what was requested. A file whose length cannot be read carries neither — the fields are absent rather than zero, so an absent measurement is never mistaken for a zero-length video.

Poll: GET /v1/video/:jobId (1–5s interval recommended). Webhook: signed with HMAC-SHA256 over your webhook_url secret. See Sessions → Webhooks for verifier code.

SDK

import { ph0ny } from '@ph0ny/sdk'

const job = await ph0ny.video.avatar({
  image_url: 'https://uploads.example.com/headshot.jpg',
  text: 'Welcome to ph0ny.',
  voice_id: 'rachel',
  provider: 'hallo3',
})

const result = await ph0ny.video.waitForJob(job.id)
console.log(result.video_url)

// Translate an uploaded interview into English captions over the original audio
const translateJob = await ph0ny.video.translateFile(fs.readFileSync('interview.mp4'), { target_language: 'en' })
const translated = await ph0ny.video.waitForJob(translateJob.id, { timeout: 30 * 60_000 })
console.log(translated.result?.translation?.captions.srt)

From a shell, ph0ny video translate interview.mp4 --to en does the same and saves the video and an SRT next to it.

Video API ​

Generate ​

Request ​

Response 202 ​

Avatar ​

Request — pre-rendered audio ​

Request — TTS chaining ​

Lipsync ​

Translate ​

Dub ​

Analyze ​

Cost estimate ​

Job lifecycle ​

SDK ​

Video API

Generate

Request

Response `202`

Avatar

Request — pre-rendered audio

Request — TTS chaining

Lipsync

Translate

Dub

Analyze

Cost estimate

Job lifecycle

SDK