CrucibleCook Houdini HDAs at Karma Render Time as a Native Hydra Scene Index

Overview

Crucible is an open-source Hydra scene index plugin that cooks Houdini Digital Assets (HDAs) at Karma render time. Author a CrucibleProcedural prim in your USD stage that points at an HDA, and when Karma renders, Crucible cooks the referenced asset on Houdini's main thread, injects the resulting mesh into the Hydra scene, and hides the input geometry, all at the scene index level.

There is no pre-cooking step, no LOPs recook, and no HAPI session to manage. The HDA is evaluated lazily as Karma pulls on the scene graph, and saving the .hda file re-cooks the procedural live without restarting the render.

GitHub: github.com/plattipus/crucible

---

Why This Matters

If you don't write pipeline code, here is the short version.

A Houdini Digital Asset is how a studio packages a procedural effect, a destruction setup, a crowd system, a growth simulation, so it can be reused and art-directed rather than rebuilt per shot. Getting one into a final render normally means baking it down to geometry first. That bake is slow, it is one-way, and it has to be repeated every single time an artist changes their mind.

Crucible removes the bake. The effect stays live and procedural right up to the moment Karma renders it, so an artist adjusts the setup while looking at the finished frame instead of a proxy, and sees the result in seconds rather than after another round trip.

That is the whole reason we build tooling in-house: it shortens the distance between a creative decision and seeing its consequence, and it leaves the judgment exactly where it belongs. The tool does not decide what the shot should look like. It stops making the person who does decide wait. Crucible was written by the same practitioners who run our visual effects supervision and animation work, which is why it solves a problem that actually shows up on a Tuesday afternoon in production, rather than one that looks good in a paper.

---

What It Looks Like

A CrucibleProcedural prim cooking in Solaris. Editing the HDA and its exposed parameters re-cooks the mesh live in the Karma viewport, with no LOPs recook or render restart.

---

The Core Idea

A CrucibleProcedural is authored as a USD GenerativeProcedural prim. All of its configuration lives in the primvars: namespace, so the Hydra scene index layer reads it through the standard HdPrimvarsSchema, with no custom schema translator required:

def GenerativeProcedural "myProc" (
    prepend apiSchemas = ["HydraGenerativeProceduralAPI"]
) {
    token primvars:hdGp:proceduralType = "CrucibleProcedural"
    asset  primvars:Recipe             = @/absolute/path/to/myEffect.hda@
    string primvars:InputPrim          = "/root/sourceGrid"
}

The HDA contract is deliberately minimal: a SOP-context asset with a single geometry output that optionally accepts input geometry on port 0. Any parameter you want to drive from USD is overridden by name through a child params Scope, where each primvars:* attribute maps directly to an HDA parm token.

---

How It Works

Crucible inserts two scene indices between the USD stage and Karma. Neither modifies USD; they operate purely at the Hydra data model level. Data is pulled top-down by Karma; dirty notifications propagate bottom-up.

Scene Index Chain

• CrucibleOverlaySceneIndex: re-types the procedural prim so the resolving scene index recognises it, forces the InputPrim source geometry invisible so it does not appear in the beauty render, and synthesises dirty notifications to drive live updates.
• HdGpGenerativeProceduralResolvingSceneIndex: Hydra's stock resolving index, which instantiates CrucibleGenerativeProcedural and calls Update() to evaluate the asset.

The Cook

When Karma evaluates the procedural, Crucible reads the HDA path, the input prim, and every parameter override, then dispatches the cook through a Python C API bridge into a standalone crucible_houdini.cook module. Because Houdini's object model (hou.*) is not thread-safe, the cook is marshalled onto Houdini's main thread via dispatch_async, while the render thread waits on a semaphore (timeout configurable via CRUCIBLE_COOK_TIMEOUT).

The cook installs the HDA, wires the input mesh read directly from the live USD stage, applies parameter overrides, and triggers the SOP cook. The returned geometry is deserialised into Hydra data sources and emitted as child mesh prims (myProc/mesh_0, myProc/mesh_1, …) that Karma renders.

Keeping the cook logic in Python means the evaluation behaviour can be changed with no recompile of the C++ plugin.

Live Updates

Two triggers converge on a single in-place re-cook:

• Parameter edits: changing a primvar in the params Scope fires a USD dirty notification. The overlay index detects that the dirty prim is a child of a registered procedural and synthesises an additional dirty entry for the procedural itself.
• HDA file saves: a background thread polls the modification time of every watched HDA once per second and re-cooks when the file changes.

In both cases, the resolving index diffs the new cook result and sends targeted PrimsDirtied notifications, so Karma updates vertex buffers in place: no mesh teardown, no GPU buffer recreation, no render restart.

---

Beyond Render Time

The cook backend is a standalone Python module usable independently of the Karma pipeline:

• cook_hda: cook an HDA with parameter overrides and an optional input prim, returning a plain mesh dictionary. Useful for baking, batch processing, and Python Script LOPs.
• lop_expand: bake every CrucibleProcedural in a stage to a real UsdGeom.Mesh, for render farm submission or USD interchange without the plugin installed.
• Debug LOP: a built-in LOP HDA that cooks at author time rather than render time.
• CLI introspection: create_procedural_prim.py introspects an HDA and writes a complete .usda layer, exposing every parameter tagged for USD in the Houdini parameter editor.

---

Requirements

Dependency	Version
Houdini	21.0.559 or later, provides Karma, USD, Python 3.11
CMake	3.21 or later
Platform	macOS / Linux
rez	Optional; recommended for managed pipelines

---

License

MIT: open for production use, modification, and redistribution. Contributions welcome via the GitHub repository.

---

From the Lab

Crucible is one output of our open research practice: problems hit during live production get solved properly, documented, and published rather than patched and forgotten. Everything we learn building these tools feeds back into the visual effects and animation work that raised the problem in the first place.