Smelter TypeScript SDK
Smelter is a video/audio composition framework using React components to define scenes. You write React JSX describing the layout; Smelter renders it as actual video frames.
Version mismatch
This documentation refers to ^0.3.0 version of the smelter packages. When encountering API errors such as missing fields, nonexistent properties, unexpected types, or deprecated options, read package.json and check if this project is using the correct version.
If there is a version mismatch, inform the user about the actions they can take to resolve it and finish the answer immediately. → For detailed instructions on what actions the user can take: references/update.md
Core Concept
// Define a scene as React JSX
function MyScene() {
return (
<View style={{ width: 1920, height: 1080 }}>
<InputStream inputId="input_1" />
</View>
);
}
// Wire it to actual media
await smelter.registerOutput("main", <MyScene />, { type: "rtmp_client", url: "..." });Runtime Packages — Choose One
| Package | Use when | Details |
|---|---|---|
@swmansion/smelter-node | Server-side Node.js app | Auto-spawns Smelter binary; live + offline modes |
@swmansion/smelter-web-client | Browser app, external server | Connects to deployed Smelter server; live + offline modes |
@swmansion/smelter-web-wasm | Browser app, no server | Runs Smelter as WASM in Web Worker; Chrome only |
→ For detailed setup and API for each runtime: references/runtimes/nodejs.md, references/runtimes/web-client.md, references/runtimes/web-wasm.md
Components
Import from @swmansion/smelter:
import { View, Text, InputStream, Tiles, Rescaler, Image, Mp4, Shader, Show, SlideShow, WebView } from "@swmansion/smelter";Layout Components
| Component | Summary | When to use |
|---|---|---|
| View | Core container, like <div> | Structure any layout; supports absolute and static positioning, overflow, background color |
| Tiles | A layout component that arranges all children side by side in equally sized, non-overlapping tiles, automatically calculating optimal rows/columns. | Multi-stream grids (e.g., video conferencing layout, side by side layouts). Use as default for multiple inputs if user does not specify a layout explicitly. |
| Rescaler | Scales single child to fit, preserving aspect ratio | Fit any stream/content into a fixed area |
→ references/components/View.md, references/components/Tiles.md, references/components/Rescaler.md
Media Components
| Component | Summary | When to use |
|---|---|---|
| InputStream | Displays a registered input stream | Show any registered input (camera, RTP, RTMP, WHIP, etc.) |
| Mp4 | Plays MP4 file directly (no registration needed) | Simple one-off MP4 playback without registration overhead |
| Image | Renders an image (URL or registered asset) | Static images, logos, overlays |
| Shader | Renders output of a WGSL GPU shader | Custom visual effects, chroma key, color grading |
| WebView | Renders a live website via Chromium | Embedding web-based graphics or interactive content |
→ references/components/InputStream.md, references/components/Mp4.md, references/components/Image.md, references/components/Shader.md, references/components/WebView.md
Utility Components
| Component | Summary | When to use |
|---|---|---|
| Text | Renders styled text | Whenever stream needs to display text, specifically for lower thirds, captions, labels, titles, etc. |
| Show | Conditionally shows children based on timestamp | Scheduling elements in offline processing |
| SlideShow | Sequences <Slide> children one after another | Intro/outro sequences, sequential content |
→ references/components/Text.md, references/components/Show.md, references/components/SlideShow.md
Props (Styling)
| Props Type | Used by | Summary |
|---|---|---|
| ViewStyleProps | <View> | Width/height, direction (row/column), absolute positioning, overflow, background, padding |
| TextStyleProps | <Text> | fontSize (required), font family/weight/style, color, alignment, wrapping |
| TilesStyleProps | <Tiles> | Width/height, tile aspect ratio, margin, padding, alignment |
| RescalerStyleProps | <Rescaler> | Rescaling mode (fit/fill), alignment, absolute positioning |
| Transition | <View>, <Tiles>, <Rescaler> | Animated scene updates with duration and easing |
| EasingFunction | Transition | "linear", "bounce", or custom cubic_bezier |
→ references/props/ViewStyleProps.md, references/props/TextStyleProps.md, references/props/TilesStyleProps.md, references/props/RescalerStyleProps.md, references/props/Transition.md, references/props/EasingFunction.md
Hooks
| Hook | Summary | When to use |
|---|---|---|
| useInputStreams() | Returns state of all registered inputs | Conditionally render based on stream status (ready/playing/finished) |
| useAudioInput(id, opts) | Controls audio for an input without rendering it visually | Background audio mixing without visual component |
| useAfterTimestamp(ms) | Returns true once a timestamp passes | Time-based scene changes in offline processing |
| useBlockingTask(fn) | Runs async fn, blocks offline rendering until resolved | Load remote data before offline rendering proceeds |
→ references/hooks/useInputStreams.md, references/hooks/useAudioInput.md, references/hooks/useAfterTimestamp.md, references/hooks/useBlockingTask.md
Inputs
Registered via smelter.registerInput(id, options). Displayed via <InputStream inputId="id" />.
| Input type | Runtime | Use when |
|---|---|---|
mp4 | All | Play a local/remote MP4 file |
rtp_stream | Node.js, Web Client | Receive RTP stream over UDP/TCP |
hls | Node.js | Consume HLS playlist |
whip_server | Node.js, Web Client | Accept WebRTC stream via WHIP protocol |
whep_client | Node.js, Web Client | Pull stream from WHEP server |
rtmp_server | Node.js, Web Client | Accept RTMP stream (OBS, FFmpeg) — experimental |
camera | WASM only | Browser camera via getUserMedia() |
screen_capture | WASM only | Browser screen capture via getDisplayMedia() |
stream | WASM only | Any MediaStream object |
whep_client (WASM) | WASM only | Pull from WHEP server in browser |
→ Full details: references/inputs.md
Outputs
Registered via smelter.registerOutput(id, <ReactRoot />, options).
| Output type | Runtime | Use when |
|---|---|---|
mp4 | Node.js, Web Client | Save to MP4 file |
rtp_stream | Node.js, Web Client | Stream over RTP |
hls | Node.js, Web Client | Stream over HLS. User will need to handle serving the files on their own. |
whip_client | Node.js, Web Client | Push via WebRTC WHIP |
whep_server | Node.js, Web Client | Serve via WebRTC WHEP to multiple viewers |
rtmp_client | Node.js, Web Client | Push to RTMP server (YouTube, Twitch) |
canvas | WASM only | Render to HTMLCanvasElement |
stream | WASM only | Return a MediaStream |
whip_client (WASM) | WASM only | Push via WebRTC WHIP from browser |
→ Full details: references/outputs.md
Resources
Pre-registered assets used by components.
| Resource | Registered via | Used by component |
|---|---|---|
| Image | smelter.registerImage(id, opts) | <Image imageId="id" /> |
| Shader | smelter.registerShader(id, opts) | <Shader shaderId="id" /> |
| WebRenderer | smelter.registerWebRenderer(id, opts) | <WebView instanceId="id" /> |
| Font | smelter.registerFont(source) | <Text> components |
→ Full details: references/resources.md
Patterns & Best Practices
Real-world patterns for building Smelter apps:
| Pattern | Summary |
|---|---|
| Shader Wrapping | Recursively nest <Shader> components to chain multiple effects |
| Input State Rendering | Use useInputStreams() for spinner/offline/playing conditional rendering |
| Animations via Timers | setInterval + React state for swap transitions, marquee, fades |
| Shader Color Params | Convert hex colors to per-channel f32 fields (_r, _g, _b) |
| Multi-Output Shared Store | Same store for WHEP live + MP4 recording = identical output |
| Scrolling Text | Animate <View> position inside overflow: 'hidden' container |
→ Full details and code examples: references/patterns.md