Models & software
A record of the software, AI models, and third-party components that make up NK-Studio and its runtime engine. This list is part of the deliverable and will be kept up to date as the render pipeline is finalised.
Studio application
| Component |
Description |
License |
| Unity (URP) |
Real-time 3D engine powering the studio interface and avatar preview |
Unity Software License |
| FMOD Studio |
Audio engine for voice playback, preview, and recording capture |
Firelight Technologies License |
| TextMesh Pro |
UI text rendering, included with Unity |
Unity Companion License |
Runtime engine
| Component |
Description |
License |
| Python 3.11 |
Runtime language for the local API and all background services |
PSF License |
| FastAPI |
HTTP API framework for the local runtime engine |
MIT |
| Uvicorn |
ASGI server running the runtime API |
BSD |
| SoundFile / libsndfile |
Audio file I/O for WAV handling |
BSD / LGPL-2.1 |
| FFmpeg |
Video encoding — raw frames + audio to MP4 |
LGPL-2.1+ |
Voice generation (TTS)
| Component |
Description |
License |
| Qwen3-TTS-12Hz-1.7B-Base |
Base TTS model (Alibaba / Qwen team); provides the zero-shot voice cloning foundation |
Apache 2.0 |
| Custom trained talker model |
Character-specific voice model trained on top of Qwen3-TTS. Combines zero-shot reference cloning with a trained adapter for a more stable and consistent voice identity than zero-shot alone |
Proprietary |
| PyTorch (CUDA) |
GPU inference framework |
BSD |
Lip sync
| Component |
Description |
License |
| Montreal Forced Aligner (MFA) |
Forced phoneme alignment — timestamps phonemes against a transcript |
MIT |
| G2P (grapheme-to-phoneme) |
Generates pronunciations for words outside the phoneme dictionary; bundled with MFA |
MIT |
AI render
| Component |
Description |
License |
| ComfyUI |
Node-based workflow engine executing the AI render pipeline locally |
GPL-3.0 |
| PyTorch (CUDA) |
Inference runtime for the AI render models |
BSD |
Render pipeline
The AI render runs two sequential workflows on the local ComfyUI server. The studio injects the recorded take as input and retrieves the output video on completion.
Workflow 1 — Stylization. Transforms the low-resolution 3D capture into the final visual look of the avatar.
| Stage |
Role |
| Input loader |
Loads the recorded take capture |
| Video generation model — to be finalised |
Core stylization pass; renders the avatar in its final visual style |
| LoRA / style adapter — to be finalised |
Character and look-specific fine-tuning applied on top of the base model |
| Output saver |
Writes the stylized video for use as input to Workflow 2 |
Workflow 2 — Lip sync refinement. Runs a dedicated lip-sync pass on the stylized output to produce accurate mouth movement in the final image.
| Stage |
Role |
| Input loader |
Loads the stylized video from Workflow 1 |
| Audio loader |
Loads the voice WAV captured with the take |
| Lip sync model — to be finalised |
Drives mouth movement in the stylized video against the voice audio |
| Post-processing — to be finalised |
Blending, compositing, or cleanup as needed |
| Output saver |
Writes the final rendered video |
Note
Specific render models, their versions, and licensing will be completed here as the render pipeline is finalised.
Audio enhancement
| Component |
Description |
License |
| Resemble Enhance |
Two-stage audio enhancement: denoising and bandwidth restoration (Resemble AI) |
Apache 2.0 |
| PyTorch (CUDA) |
GPU runtime for the enhancement model |
BSD |
Output and delivery
| Component |
Description |
License |
| Topaz Video AI |
Post-render upscaling from pipeline output to 4K delivery quality |
Topaz Labs Commercial License |