NeurIPS 2026 Test-time guidance · Diffusion

REPA-G: Test-Time Conditioning with Representation-Aligned Visual Features

1Valeo.ai, Paris 2LIGM, CNRS, Univ Gustave Eiffel, ENPC, IP Paris 3LTCI, Télécom Paris, IP Paris
valeo.ai École nationale des ponts et chaussées, IP Paris (IMAGINE lab) Télécom Paris
Standard conditioning label · text
Class condition
⟨Angora Rabbit⟩
Class-conditional sample of an angora rabbitclass label
Text prompt · 72 words

A hyper-realistic, close-up of a fluffy white albino rabbit sitting on active volcanic terrain. Ground is cracked, blackened earth with fissures of bright, molten lava bubbling underneath. The light of lava casts an orange glow onto the rabbit's paws and lower fur. Rabbit has tall, upright ears. Background is dark, blurry volcanic landscape. High contrast, surreal, photorealistic.

Text-to-image sample of a rabbit on a volcanotext prompt
REPA-G (ours) feature tokens
Anchor photo of a white rabbit
Patch features of the anchor (PCA colours) avg. token
REPA-G sample guided by the averaged rabbit tokenREPA-G (ours)averaged features
Lava field with one selected patch
Masked rabbit features + lava patch
REPA-G sample: the rabbit standing in lavaREPA-G (ours)masked + patch
Figure 1. Class labels and text prompts versus REPA-G, all with models trained on ImageNet only. Top: averaging the anchor's feature map into a single token gives a generic “rabbit”. Bottom: the masked rabbit features plus one “lava” patch put the rabbit on a volcano, with lava light on its fur that neither source contains.
TL;DR

Diffusion transformers trained with representation alignment (REPA) learn internal features that live in a DINOv2-like space. REPA-G uses that space at inference: pick feature tokens from any image, and steer the denoising toward them, with no additional parameters and no additional training.

Abstract

While representation alignment with self-supervised models has been shown to improve diffusion model training, its potential for enhancing inference-time conditioning remains largely unexplored. We introduce Representation-Aligned Guidance (REPA-G), a framework that leverages these aligned representations, with rich semantic properties, to enable test-time conditioning from features in generation. By optimizing a similarity objective (the potential) at inference, we steer the denoising process toward a conditioned representation extracted from a pre-trained feature extractor.

Our method provides versatile control at multiple levels of granularity, ranging from patch-level matching via single patches to broad semantic guidance using global image feature tokens. We further extend this to multi-concept composition, allowing for the faithful combination of distinct concepts. REPA-G operates entirely at inference time with no additional training required, offering a flexible and precise alternative to often ambiguous text prompts or coarse class labels. Our approach achieves high-quality, diverse generations on ImageNet and COCO.

Method

Representation-aligned guidance

REPA trains a flow model so that its layer-8 features, after a small projection, match a frozen encoder (DINOv2). At inference, REPA-G turns that alignment into a guidance signal: it measures how well the current features match the target tokens extracted from an anchor image, and adds the gradient of this potential to the score.

REPA-G sampling step, drawn like the REPA figure. Right: the noisy latent goes up through a stack of SiT blocks, whose output gives the SDE drift. Left: a frozen pretrained visual encoder turns the anchor image into target tokens. Features taken after an early block go through the MLP projection and are compared with these tokens by the potential. The gradient of the potential flows back through the MLP and the first blocks only, down to the noisy latent, and is added to the drift before the Euler–Maruyama step. Guidance gradient First blocks only SiT ❄ frozen SiT blockSiT blockSiT blockSiT blockSiT blockSiT block Noisy latent SDE drift Velocity + score Euler–Maruyama Next denoising step MLP Potential Similarity Pretrained visual encoder (DINOv2) ❄ Anchor image
Sampling with REPA-G. We reuse the REPA objective at inference. Features and velocity come from the same forward pass. The backward pass only goes through the first blocks (8 of the 28 in SiT-XL), which keeps the overhead at 26% of unconditional throughput.

Conditioning granularity

The potential compares generated features with the anchor's features. We can condition on the full feature map, on a subset of it (a mask), or on its average, and this choice decides what the sample inherits from the anchor. Pick a mode to see which conditioning tokens are used and what the model generates.

Conditioning tokens
Generated sample
REPA-G sample

Full feature map

Representation space

Properties of the representation space

Feature guidance needs two things: nearby embeddings must mean similar things, and similar embeddings must yield similar conditional densities. A vanilla SiT has neither. A REPA-trained SiT has both, before and after its projection head.

Reference image
Figures 2 and 7. k-means (k = 1,000) on ImageNet in four feature spaces. For the reference image (red frame) we show other members of its cluster. Without alignment, SiT clusters mix unrelated content; the aligned model reproduces the teacher's semantic grouping.

Euclidean embedding and feature interpolation

Euclidean embedding property · log–log
anchor pairsB · ‖Δφ‖² (max slope)A · ‖Δφ‖² (min slope)
Figure 3. Symmetrised KL between conditional densities against squared feature distance, for 283 anchor pairs. Every pair sits between two lines of slope 1 whose ratio is B/A = 2.60: the conditioning space is well conditioned in every direction.
Bilinear interpolation of global tokens
Figure 4. Drag inside the pad (or use the arrow keys) to blend the averaged tokens of four anchor birds. Each cell of the 4 × 8 grid is a sample generated from the bilinearly interpolated token; transitions stay smooth and semantic.
Results

Single-source conditioning

REPA-G reads the model's own internal features, so they must be semantically organized. A SiT trained with the diffusion loss alone lacks such features: guidance degrades its samples. REPA and REPA-E obtain them through the alignment loss. REPA-G is not tied to REPA: it transfers directly to Self-Flow, which learns semantic features with a self-supervised teacher–student objective and no REPA loss.

Features
Tables 1, 2 and 7 (extract). ImageNet 256, no class label, guidance with the model's internal features. FID over 50,000 samples, each guided by a different anchor; similarity is the DINOv2 cosine between each anchor and its sample. Self-Flow is evaluated with full feature maps only.

Generalization beyond REPA-style models

Self-Flow never sees a REPA loss or an external encoder during training: a student predicts the representations of a teacher that receives a less noisy input. Its internal features end up semantic and robust to noise, and that is all REPA-G needs. Full-map guidance on Self-Flow brings FID from 35.51 to 5.49 with a DINOv2 similarity of 0.72 to the anchor, without changing anything in the method.

Figure 11 (extract). Each pair: anchor image, then the REPA-G sample from Self-Flow with full feature-map guidance.
Results

Multi-source composition

Two potentials are summed at every step: one on the anchor's full feature map, and a term pulling some patches toward a single background token. With selective patch alignment, the soft-max lets the model choose where the background goes.

Figure 5 (right). Anchor object (top) and target background (bottom), composed by REPA-G with REPA-E and DINOv2 features. Full features come from the anchor and a single feature patch from the target's background.
Analysis

Baselines, guidance scale and runtime

Comparison with TFG and IP-Adapter

TFG, adapted to feature guidance with DINOv2 as target predictor, must decode a clean estimate and backpropagate through the decoder, DINOv2 and the whole transformer; its samples carry heavy artefacts (FID 75–86). An IP-Adapter trained for 100k steps on REPA-E follows the anchor but keeps its layout less precisely and adds about 150M parameters.

Figure 11. Full-feature-map guidance with each method. The last column applies REPA-G to Self-Flow, which learns semantic internal features without a REPA objective: guidance still transfers.

Effect of the guidance scale

Metric
FullMaskedAverage
Table 8. Single-source guidance on ImageNet across λ. Quality improves then plateaus; the setting is not fragile.
Anchors
Average featuresFull feature map
AnchorAverage featureFull feature map
Figures 9 and 10. Diversity across 10 seeds per anchor (mean ± spread). Higher is more diverse. Samples below use λ = 2,000. COCO anchors are out of distribution for the generator, yet diversity does not drop: REPA-G is not retrieving training images.

Runtime

Throughput on one H100 · 256 × 256, batch 32, images/s
Table 17. Multi-source composition costs nothing extra: all potentials share one set of features. Peak memory grows from 8 to 10 GB (TFG: 52 GB).

Limitations

Strongly conflicting conditions remain hard: incompatible viewpoints or aspect ratios produce implausible blends, one source can dominate, and multi-source composition may need per-term tuning of λ. REPA-G also does not take structural inputs such as depth maps, edges or poses.

Figure 13. Failure cases of multi-source composition. Each triplet: anchor, target, sample.
Citation

BibTeX

@inproceedings{sereyjol2026repag,
  title     = {{REPA-G}: Test-Time Conditioning with Representation-Aligned Visual Features},
  author    = {Sereyjol-Garros, Nicolas and Kirby, Ellington and Letzelter, Victor and Besnier, Victor and Samet, Nermin},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}

Acknowledgements: we acknowledge EuroHPC Joint Undertaking for awarding the project ID EHPC-REG-2024R02-234 access to Karolina, Czech Republic. The code builds on REPA-E and the ADM evaluation suite.