
trellis2-lora
Preprocess meshes, train TRELLIS.2 geometry LoRA adapters on Hugging Face Jobs, and generate meshes with them
- GitHub stars
- 8
Install
/plugin install trellis2-lora@trellis2-finetuningRun it inside Claude Code after adding the marketplace with /plugin marketplace add suvadityamuk/trellis2-finetuning.
Contents
Discovered from sourceSkills 1
Fine-tune TRELLIS.2 geometry with LoRA
TRELLIS.2-4B turns one image into a textured mesh. This workflow trains LoRA adapters on
its two geometry flows (sparse structure ss, and shape) from a folder of 3D meshes.
Textures always come from the base model. Source and full docs:
https://github.com/suvadityamuk/trellis2-finetuning (trained shoe adapters:
https://huggingface.co/suvadityamuk/trellis-2-finetune-gso).
meshes (Hub dataset) → trellis2-prep → latents (Hub dataset) → trellis2-train --config ss → LoRA (Hub model)
→ trellis2-train --config shape ↗ + Trackio Space
| image | command |
|---|---|
cr.hf.co/suvadityamuk/trellis2-finetuning-prep:latest | trellis2-prep |
cr.hf.co/suvadityamuk/trellis2-finetuning-train:latest | trellis2-train |
cr.hf.co/suvadityamuk/trellis2-finetuning:latest | starter image: TRELLIS.2 at /workspace/TRELLIS.2, all CUDA extensions, PEFT, Trackio. Use it for inference |
Nothing here runs without a CUDA GPU (24 GB+). On a laptop, always launch on HF Jobs.
Rules for agents
- Confirm cost before every GPU job. Tell the user the flavor and
--timeoutyou plan to use and get a yes. Do not quote prices from memory; runhf jobs hardware. - Dry-run first on any new dataset, into separate repos (both commands resume from
whatever their output repo already holds, so a dry run must not share one with the real
run): prep with
--max-objects 8 --push-to <user>/my-meshes-trellis2-dryrunonl4x1, then train from that repo with--set max_steps=20 i_save=10 batch_size=1and--push-to <user>/my-trellis2-lora-dryrunonl4x1. Scale up only after both succeed. - Private by default. Pass
--privateon prep and train, and--trackio-private(Trackio Spaces are public otherwise). Drop them only if the user asks for public. - Always
--push-to. Job containers are deleted when they end; anything not pushed is lost. - Launch with
-dand give the user the job URL. Never block a turn onhf jobs logs -f; check withhf jobs inspect <id>or a boundedhf jobs logs <id> | tail -50.
Prerequisites (check before launching)
hf auth whoamiworks and the token has write access.- HF Jobs needs a Pro, Team or Enterprise account.
- Training only: the token's account must have accepted the gated
DINOv3 license.
Verify:
hf download facebook/dinov3-vitl16-pretrain-lvd1689m config.jsonmust succeed. If not, send the user to that page; the training job fails without it. - Inputs are 3D meshes, not images:
.glb .gltf .obj(+.mtl/textures) .fbx .ply .stl .usd, any folder layout, one object per file. Merge multi-part scenes into one file first. - For scale: the reference run used 179 shoes of one category.
Step 0: meshes on the Hub
hf repos create <user>/my-meshes --repo-type dataset --private
hf upload <user>/my-meshes ./my_meshes . --repo-type dataset
If the meshes already live in a model repo, add --meshes-repo-type model in step 1.
Step 1: preprocess (one GPU job)
hf jobs run -d --flavor a100-large --timeout 6h --secrets HF_TOKEN \
cr.hf.co/suvadityamuk/trellis2-finetuning-prep:latest \
trellis2-prep --meshes <user>/my-meshes --push-to <user>/my-meshes-trellis2 --private \
--holdout 0.05 --views 16 --workers 8
- Dry run:
--flavor l4x1 --timeout 1h,--max-objects 8, and a-dryrun--push-torepo. --meshes-include 'chairs/**'limits which files are used.- Add
--path-idswhen file names are hashes, or when each object sits in its own folder (<id>/model.obj). It groups the held-out split by object name instead of full path. Without it (the default, kept for reproducibility) such names can all fall into one group, and the split holds out every object. - Never set
--workersabove 8: Blender startups crash. - Eight stages (
index mesh voxel shape ss render split export). The repo is updated after each one, and re-running the identical command resumes. Scale: 12 objects ≈ 16 min on L4. - Success check: the output repo has
ss_latents/,shape_latents/,renders_cond/,metadata.csv,holdout.txt, and a README with per-stage counts. Compare those counts to the number of input meshes; objects that failed a stage are dropped, not fatal.
Step 2: train (two independent jobs; launch both in parallel)
for FLOW in ss shape; do
hf jobs run -d --flavor a100-large --timeout 12h --secrets HF_TOKEN \
cr.hf.co/suvadityamuk/trellis2-finetuning-train:latest \
trellis2-train --config $FLOW \
--latents <user>/my-meshes-trellis2 \
--push-to <user>/my-trellis2-lora --private \
--trackio-space <user>/trellis2-trackio --trackio-private \
--set max_steps=3000 i_save=500
done
- Dry run:
--flavor l4x1 --timeout 1h,--latentsand--push-topointing at the-dryrunrepos, and--set max_steps=20 i_save=10 batch_size=1. The trainer drops incomplete batches, so a tiny dataset withbatch_size=4can yield zero batches. --configtakesss,shape, a local JSON,hf://<repo>/<path>, or a URL. Use--set key=value(JSON-parsed) for one-off changes. See references/configs.md for fields and tuning advice.- Every
i_savesteps,<run-name>/adapter-<step>.safetensors(~37 MB) plusconfig.jsonlands in--push-to. Re-running the same command resumes from the newest checkpoint. - Held-out objects are excluded automatically;
--no-holdouttrains on everything. --subset <dataset repo | dir | file>trains on part of--latents(asubset.txtof sha256s, optional ownholdout.txt). GSO subsets exported from the GSO subset builder Space already have both.- On a 24 GB GPU (
l4x1, A10, 3090/4090) addbatch_size=2 grad_accum=8to--set(same effective batch of 16; smoke-tested on L4). - Reference speed on one A100, 3000 steps: 7.2 h for
ss, 3.6 h forshape. If--timeouthits, just re-run; it resumes.
Monitoring
hf jobs ps # running jobs
hf jobs inspect <id> # status
hf jobs logs <id> | tail -50 # recent output
hf jobs cancel <id>
Live loss, grad norm and LR are at https://huggingface.co/spaces/<user>/trellis2-trackio.
A non-finite loss raises a Trackio alert; stop the job and lower lr.
Step 3: generate meshes with the adapters
See references/inference.md. It uses scripts/generate.py, which
loads both adapters into the base pipeline and writes base-vs-LoRA GLBs to a Hub repo.
Common failures
| symptom | fix |
|---|---|
401/403 on facebook/dinov3-... in a train job | accept the DINOv3 license with the token's account |
prep crashes in render with many workers | --workers 8 or fewer |
| an object becomes many tiny objects | it was split across files; merge to one file per object |
| near-duplicate variants land in both train and holdout | name them <model>_in_<variant> (or _Color_, _Size_), or dedupe first |
| CUDA OOM in training on a 24 GB GPU | --set batch_size=2 grad_accum=8, or use a100-large |
| job killed at timeout | re-run the identical command; both commands resume |
prep log says held out N/N objects ... train sees 0 | re-run the same prep command with --stages split --path-ids (rewrites holdout.txt in place, minutes), or train with --no-holdout |
Trackio warns could not create a remote client for Space | both training jobs tried to create a new Space at once; metrics still upload when the run ends. Avoid it by launching the second job once the first has created the Space |
Running locally instead of HF Jobs
Same commands, different launcher (NVIDIA GPU, Docker with the NVIDIA runtime):
docker run --gpus all --rm --shm-size 16g -e HF_TOKEN \
-v hf-cache:/root/.cache/huggingface -v "$PWD/work":/workspace/work \
cr.hf.co/suvadityamuk/trellis2-finetuning-prep:latest \
trellis2-prep --meshes <user>/my-meshes --push-to <user>/my-meshes-trellis2 --private
For local meshes add -v "$PWD/my_meshes":/meshes:ro and pass --meshes /meshes.
The GitHub repo also has preprocess.py and finetune.py, single-file equivalents of the
two commands that run in the starter image with the repo mounted. Prefer the images unless
the user wants to modify the training code.
Limits
Geometry only; textures come from the base model. Adapters are trained for the 512 pipeline. The holdout split's de-duplication rules were written for GSO-style names.
Sponsors
Details
- Marketplace
- suvadityamuk/trellis2-finetuning
- Category
- Community