StatLUT: Statistical Feature-Driven Multimodal 3DLUT Generation for Photorealistic Style Transfer

Abstract

Photorealistic Style Transfer (PST) aims to transfer the color and tonal style of a reference to a content image while strictly preserving its structural integrity. However, existing deep learning-based methods inherently suffer from semantic entanglement caused by pre-trained image encoders, leading to unnatural spatial distortions. Moreover, current pixel-level mapping paradigms often ignore color gamut topology, resulting in color banding, while also lacking the multimodal capability for precise, tone-specific text-driven control.

To address these bottlenecks, we propose StatLUT, an innovative multimodal framework for 3D LUT generation. First, we bypass traditional encoders and introduce a Lab-Extractor to derive spatially-agnostic statistical features, fundamentally decoupling color distributions from structural semantics to ensure artifact-free rendering. Second, we formulate LUT generation as a Transformer-based Seq2Seq translation task, utilizing a Multi-dimensional Residual Mapper (MR-Mapper) to predict topologically smooth 3D LUTs. Finally, to break the unimodal barrier, we propose the H-Diffuser, a lightweight Diffusion Transformer that directly synthesizes statistical features from tone-descriptive text prompts, enabling controllable and robust color grading. Extensive experiments on standard benchmarks demonstrate that StatLUT significantly outperforms state-of-the-art methods in both visual quality and quantitative metrics, pioneering a highly robust and versatile paradigm for multimodal photorealistic style transfer.

Method

Overview of the StatLUT framework. The framework consists of three key components: (1) Lab-Extractor for spatially-agnostic statistical feature extraction, (2) MR-Mapper for Transformer-based 3D LUT prediction, and (3) H-Diffuser for text-driven statistical feature synthesis.

1 Lab-Extractor

Bypasses traditional encoders and derives spatially-agnostic statistical features from the Lab color space, fundamentally decoupling color distributions from structural semantics to ensure artifact-free rendering.

2 MR-Mapper

Formulates LUT generation as a Transformer-based Seq2Seq translation task, utilizing a Multi-dimensional Residual Mapper to predict topologically smooth 3D LUTs with proper color gamut topology preservation.

3 H-Diffuser

A lightweight Diffusion Transformer that directly synthesizes statistical features from tone-descriptive text prompts, enabling controllable and robust color grading beyond the unimodal barrier.

Key Modules

Lab-Extractor. Extracts spatially-agnostic statistical features from the Lab color space, decoupling color distributions from structural semantics to ensure artifact-free rendering.

MR-Mapper (Multi-dimensional Residual Mapper). Formulates LUT generation as a Transformer-based Seq2Seq translation task, predicting topologically smooth 3D LUTs with proper color gamut topology preservation.

Quantitative Results

Comparison with state-of-the-art photorealistic style transfer methods on NAS and PST50 datasets. StatLUT (Img) achieves the shortest distance to the ideal point, indicating an optimal balance between content preservation and style transfer.

Metric	PhotoNAS	NLUT	Neural Preset	D-LUT	SA-LUT	DLUT-VCG	StatLUT (Img)	StatLUT (Text)
Content Sim. ↑	0.477	0.640	0.709	0.700	0.696	0.629	0.707	0.688
Style Sim. ↑	0.557	0.787	0.656	0.562	0.426	0.676	0.746	0.498
Dist. to Ideal ↓	0.681	0.400	0.448	0.517	0.598	0.491	0.386	0.565

Metric	PhotoNAS	NLUT	Neural Preset	D-LUT	SA-LUT	DLUT-VCG	StatLUT (Img)	StatLUT (Text)
Content Sim. ↑	—	0.746	0.774	0.734	0.729	0.713	0.760	0.757
Style Sim. ↑	—	0.849	0.738	0.467	0.650	0.549	0.870	0.536
Dist. to Ideal ↓	—	0.284	0.345	0.544	0.438	0.514	0.259	0.487

Red / blue: best / second-best image-driven results. PhotoNAS results are not available on PST50.

Distance to Ideal Point

Scatter plots showing the trade-off between Content Similarity and Style Similarity on (a) NAS and (b) PST50. The ideal point (1, 1) represents perfect content preservation and style transfer. StatLUT (Img) achieves the shortest distance to the ideal point among all methods.

Qualitative Results

Qualitative comparison on NAS dataset. StatLUT produces more natural color grading with better structural preservation compared to existing photorealistic style transfer methods.

Qualitative comparison on PST50 dataset. StatLUT (Img) achieves superior visual quality by effectively decoupling color distributions from structural semantics.

Applications

StatLUT enables versatile photorealistic style transfer with both image reference and text-driven color grading control.

Multimodal-Driven PST

Multimodal-driven photorealistic style transfer. StatLUT supports both image-reference and text-driven modes. Given a content image, users can either provide a style reference image or describe the desired style with natural language text prompts, achieving flexible and controllable color grading.

Cross-Semantic Style Transfer

Cross-semantic photorealistic style transfer. StatLUT effectively transfers style across semantically different scenes while preserving structural integrity. The statistical feature representation decouples color distributions from content semantics, enabling robust style transfer even when content and style images have different semantic layouts.

Video Style Transfer

Drag the slider to compare Content (left) and Stylized (right). Click to play/pause.

Style

Stylized Content

Style

Stylized Content

Style

Stylized Content

Style

Stylized Content

Style

Stylized Content

Style

Stylized Content

Style

Stylized Content

Style

Stylized Content

Case 1 / 8

Semantic Interference

To evaluate the impact of semantic structure on style transfer, we shuffle the style image using patches of different sizes (destroying spatial layout while preserving color statistics) and measure the deviation from the baseline PST result (unshuffled style). Larger deviation indicates stronger sensitivity to semantic layout. StatLUT shows minimal deviation across all shuffle levels, demonstrating effective decoupling of color distribution from structural semantics.

Semantic interference ablation study. Row 1 (baseline): Content, Style, and PST results from different methods using the original unshuffled style. Rows 2–5: PST results when the style image is shuffled with increasing patch granularity (2×2, 3×3, 4×4, 5×5). Numbers below each result indicate the deviation from the baseline PST result. Larger values indicate greater sensitivity to spatial semantic structure. StatLUT maintains consistently low deviation across all shuffle levels, while other methods show significant degradation as semantic layout is destroyed.

Datasets & Acknowledgments

Training: MSCOCO (cocodataset.org)

Testing: PST50 (HuggingFace) and NAS (GitHub)

We thank the creators of the PST50 and NAS datasets for making their benchmarks publicly available. This work was supported by Honor Device Co., Ltd.

Citation

@article{wang2026statlut,
  title     = {StatLUT: Statistical Feature-Driven Multimodal 3DLUT Generation for Photorealistic Style Transfer},
  author    = {Wang, Yifan and Hao, Zhixiang and Wang, Yu and Zhu, Congchao},
  journal   = {NeurIPS},
  year      = {2026}
}