Marigold V2
Revisiting Diffusion Transformers for Monocular Depth Estimation
SIGGRAPH Asia 2026 · ACM Transactions on Graphics
Revisiting Diffusion Transformers for Monocular Depth Estimation
SIGGRAPH Asia 2026 · ACM Transactions on Graphics
Monocular depth estimation is a ubiquitous yet highly ill-posed computer vision task, with downstream applications in scene reconstruction, computational photography, and robotics, among others. Despite the field's maturity, recent models still struggle to generalize to out-of-distribution inputs and to produce sharp and detailed depth maps.
In this paper, we revisit Marigold, a set of techniques for repurposing modern image generation and editing models, powered by the diffusion transformer (DiT) architecture, into state-of-the-art monocular depth estimators. Our recipes target single-step inference from pretrained multi-step flow-matching models, with quantization where needed, preserving model capacity while remaining cheap to run. We analyze the artifacts of naïve training and identify two effective remedies: aligning the model's internal representations with semantic features extracted from ground-truth, and adopting a 2-stage fine-tuning protocol built around a novel Sinkhorn-based loss.
The results are crisper, cleaner depth maps that generalize well out-of-distribution, with 16–26% improvement in AbsRel over the previous best on KITTI and ETH3D. Qualitatively, our model resolves fur, foliage, and hair-thin edges that have eluded prior models. Furthermore, Marigold V2 achieves state-of-the-art results when applied to other dense regression tasks, such as surface normals estimation and intrinsic image decomposition.
Photographs on the left, Marigold V2 depth on the right; drag the divider on any example. The model is zero-shot: it was trained only on synthetic data and has never seen these scenes. Depth is shown with the Spectral color map, red for near and blue for far, normalized per image. Look at the places where depth estimators usually fail: single hairs, fur, foliage, wires, railings, and the sky behind them. Marigold V2 keeps them separate from the background instead of blurring them into it.
The same scenes, with a recent method on the left and Marigold V2 on the right, in the same color map. Each example shows the name of its method (Pixel-Perfect Depth, InfiniDepth, Lotus-2, MoGe-3 or Marigold V1.1), and the buttons above it switch to another one. The differences are at the boundaries: flying pixels around thin structures, halos around hair, and edges that do not follow the object.
The same fine-tuning recipe with different supervision gives single-step predictors for other dense targets. Three of them are shown below. Metric depth completion, which fits a small adapter to sparse measurements at test time, is described in the paper.
Ordinary depth stops at the first surface, so a window or a display case becomes a flat sheet. The see-through model is fine-tuned on the final depth layer of LayeredDepth-Syn and predicts the geometry behind the glass. Left: the input, or the regular depth prediction of the same scene; right: see-through depth.
Left: the input or another normals estimator (FE2E, Lotus-2, MoGe-3 or Marigold V1.1); right: Marigold V2. Colors encode the direction of the surface normal, one channel per axis: surfaces facing up are green, surfaces facing the camera are blue, and surfaces facing sideways are red. Look at railings, leaves and folds, where small surfaces keep their own orientation instead of being flattened into their surroundings.
Albedo is the surface color without the lighting: shadows, highlights and shading are removed. Left: the input or another intrinsic decomposition method (RGB-X, IID-in-the-wild or Marigold V1.1); right: Marigold V2. Albedo is predicted in linear color and shown tone-mapped.
Marigold V2 starts from Qwen-Image-Edit-2509, a pretrained image-editing diffusion transformer, and fine-tunes it into a single-step predictor of a dense target: depth, or with different supervision, surface normals, albedo or see-through depth. The transformer is quantized to 4 bit and trained through rank-128 LoRA adapters, so a few days on one consumer GPU are enough. The recipe does not depend on this backbone: the same losses also improve Stable Diffusion 1.5, the U-Net behind the original Marigold, and FLUX.2 klein, so most image foundation models should work. Given an image $I$, the VAE encoder produces a latent $z_I$, the adapted DiT maps it to the target latent in one rectified-flow step, and the VAE decoder turns that into the prediction.
Stage 1 trains only the LoRA adapters. The DiT is conditioned on $z_I$ and a fixed timestep $t=0.5$ and regresses the velocity $v = z_I - z_d$ toward the target latent $z_d$. With the timestep fixed, this rectified-flow objective is a direct latent regression with no trajectory to integrate. Added to it are an $L_1$ loss on the decoded target, an $L_1$ loss on its spatial gradients, and iREPA, which aligns the intermediate features of the DiT with DINOv3 features computed on the target. iREPA preserves the global layout while recovering fine structure such as foliage, fur and hair.
Stage 2 continues from that checkpoint, unfreezes the VAE decoder and adds SinkLoss. The image is tiled into non-overlapping $K\times K$ blocks ($K=5$), and within each block Sinkhorn-Knopp matching softly assigns predicted values to ground-truth values. This relaxes strict pixel-to-pixel supervision where the ground truth is noisy or ambiguous, above all around thin and transparent objects, which removes flying pixels and sharpens discontinuities.
The input is encoded once and passed through the DiT once, at $t=0.5$: $\hat{z}_d = z_I - f_\theta(z_I, t)$, then $\hat{d}=\mathcal{D}_{\mathrm{vae}}(\hat{z}_d)$. There is no iterative denoising schedule. A prediction costs one encoder pass, one transformer pass and one decoder pass, and the 4-bit weights keep the memory footprint small. On a single 32 GB GPU, Marigold V2 runs at 2048×2048 while the other diffusion-based estimators run out of memory:
Only the supervision changes.
Training details, ablations and the remaining results are in the paper and in the GitHub repository.
Zero-shot affine-invariant depth on NYUv2, KITTI, ETH3D, ScanNet and DIODE, after alignment to the ground truth. The table reports AbsRel in percent, lower is better. Marigold V2 has the lowest error on all five benchmarks among the listed methods, with the largest margins on KITTI and ETH3D, where it improves on the previous best by 16–26%. Results for see-through depth, surface normals, albedo and depth completion are in the paper.
Marigold V2 is the fourth model in a line that started from one observation: a pretrained image generator already knows what scenes look like, so it is a better starting point for dense prediction than training from scratch.
Marigold V2 keeps the idea and replaces the rest: a diffusion transformer instead of a U-Net, 4-bit QLoRA instead of full fine-tuning, one fixed-timestep pass instead of a denoising schedule, and SinkLoss and iREPA for fine detail. The same recipe now covers see-through depth, surface normals, albedo and metric depth completion. The earlier models remain available from their project pages.
@article{pavlovic2026marigoldv2,
author = {Pavlovic, Igor and Wandel, Thiemo and Obukhov, Anton and Bartolomei, Luca and Davydov, Andrey and Tosi, Fabio and Poggi, Matteo and S{\"u}sstrunk, Sabine and Dai, Dengxin},
title = {Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation},
year = {2026},
issue_date = {December 2026},
publisher = {Association for Computing Machinery},
volume = {45},
number = {6},
url = {https://doi.org/10.1145/3842528},
doi = {10.1145/3842528},
journal = {ACM Trans. Graph.},
month = dec,
articleno = {204},
numpages = {14}
}