ScaleResfusionResidual Rectified Flow based on Residual Vector Field

Abstract

Method

Residual Rectified Flow

Given an LQ–HQ pair, we define the residual R = x̂₀ − x₀. RRF inserts the weighted residual into the linear path of Rectified Flow while keeping the transport straight:

Substituting R shows that the coefficient of the unknown HQ image vanishes at t* = 1/(1+γ): the state there depends only on the LQ image and Gaussian noise, so inference starts from x_t* = γ/(1+γ) x̂₀ + 1/(1+γ) ε and integrates the learned residual vector field back to t = 0 over a shortened interval. Because resv = v^RF + γR carries no scheduler-specific coefficients, adapting a pre-trained rectified-flow model only requires fitting the compact residual correction, which we realise with LoRA alone.

Why start from the residual?

The reverse ODE is initialization-sensitive: a start closer to the restoration manifold is transported to a terminal state closer to the HQ distribution. Gaussian initialization is task-agnostic and far from the observed content; deterministic one-step LQ→HQ mappings lose the sampling capability of diffusion models; noisy-LQ DDPM sampling is biased toward the degradation without an explicit correction direction.

ScaleResfusion instead starts from a noisy LQ state and follows a residual-oriented transport, giving a closer initialization and more coherent convergence toward the target distribution.

Knowledge-distilled parameter-efficient training

The restored-image generator predicts the residual vector field under RRF. LQ features are injected as conditioning (a ReferenceNet-style feature extractor for SD3 / Z-Image, token concatenation for FLUX.2), and DAPE provides textual constraints. A DMD loss and an optional GAN loss regularise the restored distribution toward natural HQ images. Generation and regularization branches share the same frozen backbone weights; only LoRA adapters are trained.

Results

Quantitative comparison with the default FLUX.2-klein-4B backbone and 4 sampling steps. Red = best, blue = second best. The first group lists GAN-based methods, the second group diffusion-based methods.

Efficiency

Number of function evaluations (NFE) and average inference time per 512×512 image on a single NVIDIA RTX A6000.

Consistent transfer across backbones

RRF adapts pre-trained rectified-flow backbones that differ in architecture and scale — SD3 (2B), FLUX.2-klein (4B / 9B) and Z-Image (6B) — with the same LoRA-only recipe. Results on DRealSR; the upper block is w/o GAN, the lower block w/ GAN.

Backbone	PSNR ↑	SSIM ↑	LPIPS ↓	DISTS ↓	FID ↓
SD3 (2B)	28.77	0.79	0.30	0.23	149.67
FLUX2-Klein (4B)	29.77	0.82	0.25	0.20	118.18
Z-Image (6B)	29.35	0.80	0.28	0.22	132.31
FLUX2-Klein (9B)	30.17	0.82	0.25	0.19	109.28
SD3 (2B), w/ GAN	27.86	0.76	0.32	0.22	146.93
FLUX2-Klein (4B), w/ GAN	28.26	0.78	0.29	0.21	124.03
Z-Image (6B), w/ GAN	28.34	0.78	0.28	0.21	127.40
FLUX2-Klein (9B), w/ GAN	28.77	0.79	0.28	0.19	116.69

Few-step sampling

Separately trained 4-, 2- and 1-step FLUX.2-4B models on DRealSR. More steps improve fidelity, while even the 1-step model remains competitive thanks to the RRF acceleration point.

NFE	PSNR ↑	SSIM ↑	LPIPS ↓	FID ↓
4-step	29.77	0.82	0.25	118.18
2-step	29.03	0.79	0.27	115.43
1-step	28.93	0.78	0.28	117.56

Visual Comparison

DRealSR

RealSR

DIV2K-Val

LSDIR-Val

WebPhoto-Test (in-the-wild face restoration, zero-shot)

User Study

Pairwise model arena: in each round a participant sees an LQ input with the anonymised results of two models and picks the preferred one (or tie / both bad). 48 evaluators contributed 1,756 pairwise comparisons over benchmark images and in-the-wild photos.

Video

BibTeX

@article{shi2026scaleresfusion,
  title   = {ScaleResfusion: Residual Rectified Flow based on Residual Vector Field},
  author  = {Shi, Zhenning and Xu, Chen and Zhang, Junhao and Zhang, Kefei and Liu, Linjie and Zheng, Zhedong and Li, Tao},
  journal = {arXiv preprint arXiv:2607.25275},
  year    = {2026}
}