Render pipeline

This page defines render-stage and CPU/GPU ownership. Update it when an operator changes stage, route, resolution, or color-space behavior.

Operator registry

crates/raw-pipeline/src/ops.rs registers operators and sorts them by (stage, order):

Sensor -> WhiteBalance -> Tone -> Color -> Geometry -> Output

Every operator implements Op and declares one mandatory GpuRoute:

Route	Meaning
Fused	Pointwise Rust and WGSL implementation in the generated process pass
Presence	Texture, clarity, and luma-pyramid infrastructure
Detail	Dedicated noise-reduction or capture-sharpen path
Pass(name)	Dedicated named GPU pass such as retouch, dehaze, or lens blur
Manifest	Persistence-only entry that never runs as a normal operator

There is no default route. A new operator that omits it fails to compile. Registry tests check that named passes exist, manifest operators never become active, and only fused operators expose WGSL snippets.

Fused math remains written in Rust and WGSL. src/ops/shader_parity.rs builds a standalone shader from each fused operator, runs a scene-linear color grid, and compares it with apply_cpu at a relative tolerance of 1e-4. This also checks uniform packing.

CPU flow

cpu::render_with_cancel:

1. Demosaics Bayer or X-Trans mosaics.
2. Blends the AI Denoise result into the demosaiced image when one is supplied.
3. Applies the DNG’s own lens warp when the file carries one.
4. Resolves the camera matrix and DCP baseline.
5. Runs sensor operations, including effective lens corrections.
6. Applies EXIF orientation.
7. Runs WhiteBalance, Tone, Color, and Geometry operators.
8. Runs Output operators.
9. Applies the DCP LookTable and tone curve, destination gamut, curves, and creative 3D LUT.
10. Computes preview metadata and encodes the result.

CPU operators do not skip Detail routes. Noise reduction, capture sharpening, creative sharpening, retouch, dehaze, lens blur, masks, and geometry all have CPU implementations.

Sensor-stage cache

SPATIAL_BOUNDARY = (Tone, -35) separates expensive source-dependent work from display work. CpuRenderer can cache the lower half for previews: demosaic, AI Denoise blend, lens correction, orientation, as-shot white balance and camera matrix, retouch, noise reduction, capture sharpening, and preview reduction. User white balance, exposure, tone, color, geometry, masks, and output continue from the cached image, so a white balance change never re-runs noise reduction.

Quality renders and nonstandard preview modes bypass the cache. tests/cpu_cache.rs compares cache misses and hits with uncached cpu::render.

White balance order

The as-shot white balance and camera matrix run in the WhiteBalance stage. Raw edits store the scene white as an absolute color temperature and tint, using the DNG SDK temperature model: 2000 K to 50000 K, and tint ±150, where one tint unit is 1/3000 Duv and positive tint is magenta. Images without camera color data, and raw edits saved before this model, store a relative shift from the as-shot white instead: temperature ±100 spans ±1.5 stops of color temperature, and tint ±100 spans ±0.03 Duv. Raw images re-render as if the camera had been white balanced at that white, using the camera matrices and DCP forward matrices interpolated for the new white, so the DCP base table and camera color follow the edit. Images without camera color data use a Bradford adaptation in linear sRGB. The adjustment is pointwise and runs in the Tone stage after dehaze, texture, and clarity and before exposure, on both renderers and for mask layers too. Noise reduction, capture sharpening, and the dehaze atmosphere estimate therefore see the as-shot white balance, and a white balance change reuses all of them.

DNG corrections and bad pixels

RAW decode repairs hot and dead pixels first. Every Bayer and X-Trans mosaic is checked against the same-color photosites in its 5×5 neighborhood: a photosite more than twice as bright as all of them (or less than half as bright), by at least 2% of white, is replaced by their median. A point of light spreads into the neighboring photosites of the other colors and a defective photosite does not, so a photosite whose four direct neighbors stand out from their own colors on average by at least 5% of its excess is kept. DNG FixBadPixelsConstant and FixBadPixelsList entries in OpcodeList1 are repaired the same way, whether or not they look defective.

After highlight reconstruction, decode multiplies the mosaic by the DNG’s gain maps and radial vignette from OpcodeList2 and OpcodeList3, per color, without the DNG SDK’s clamp at white so rebuilt highlights survive. Phones store lens shading this way, and drones and some compact cameras add a vignette. Linear DNGs get the same gains per channel after decode. A gain listed after the warp is evaluated where the warp moves each photosite. A WarpRectilinear distortion correction travels with the frame and runs after demosaic and the AI Denoise blend, before user lens corrections, on both renderers. A photo with any of these corrections skips the automatic lensfun profile, so nothing is corrected twice. Other opcodes are skipped, and a malformed one never stops the photo from opening.

Highlight reconstruction

RAW decode rebuilds clipped photosites before demosaic, so both renderers and every later stage see the same data. Each color’s clip point is 98.7% of the level where its photosites pile up, which on some cameras sits above the white level in the file. Where one color in the 3×3 neighborhood is clipped, its value comes from the other two (the cube-root mean of their white-balanced levels, the “inpaint opposed” idea from darktable) plus a per-image color offset measured on unclipped photosites near the clipped areas. Where two or three colors are clipped, the pixel takes the color of the nearest bright unclipped surface: a smooth color field built from the photosites around each clipped area, scaled to the brightness of the colors still known. Small clipped areas take one color throughout. Where all three colors are clipped, or the clipped values are well above what that color allows, the result blends smoothly toward white, as blown areas look in other RAW editors. The estimate never lowers a photosite and can exceed white. Linear DNGs get the same treatment per pixel after decode.

Highlights below 0 then applies one gain to all three channels, driven by the brightest channel on a log scale. The gain is 1 at mid gray and below and reaches the full slider strength four stops above, so bright areas come down without flattening or changing color.

GPU flow

GpuRenderer::render_with_cancel selects a fast path or the presence/detail path. Current order:

Order	Pass	Responsibility
1	Upload and demosaic	Mosaic or RGB input to scene-linear texture
2	AI Denoise	Blend the stored full-resolution AI Denoise result at the chosen strength and detail
3	Sensor	The DNG’s own lens warp, then lens vignette and sensor-space corrections
4	White-balance preparation	As-shot white balance and camera-to-sRGB base
5	Retouch	Heal and clone strokes when active
6	Noise reduction	Color, then luma wavelet shrinkage against a measured noise profile
7	Capture sharpening	RAW-only deconvolution before preview reduction
8	Preview reduction	Shared Lanczos3 target whenever the preview is smaller than the source
9	Dehaze	Bounded atmosphere estimate and guided filter
10	Lens blur	Depth-driven circle-of-confusion gather when Lens Blur is active
11	Presence	Texture, clarity, and shadows pyramid
12	Process	User white balance, DCP HueSatMap, fused operators, and geometry sampling
13	Masks	Component weight and local adjustment blend
14	Sharpen	Global and per-pixel masked amount
15	Effects and output	Vignette, grain, destination gamut, transfer curve, curves
16	DCP finish	LookTable, profile tone curve, curves
17	LUT	.cube 1D shaper and tetrahedral 3D sampling in display sRGB
18	Mask overlay	Optional red coverage overlay
19	Readback and encode	Warning paint, histogram, scopes, and output encoding

The exact encoder grouping can combine adjacent entries. The order and color-space boundaries are the contract.

Portable linear source

A render splits into a sensor stage (passes 1 to 8 plus the dehaze atmosphere estimate) and a display chain (everything after). render_source on either renderer runs only the sensor stage and returns a SourceImage: the as-shot white-balanced, noise-reduced, preview-sized frame as half-float RGB, plus the frame metadata and the atmosphere the display chain needs for dehaze. It always takes the presence path and always estimates the atmosphere, so one source can render any display edit. GpuRenderer::upload_source turns a decoded source back into a texture, and the display chain renders it byte for byte like the frame it came from.

Edits::sensor_stage keeps only the fields that change the source: AI Denoise, noise reduction, capture sharpening, lens, retouch, the DCP selection, and crop and rotation, which set its resolution. It also keeps the Lens Blur depth map id, because a windowed source widens its margin by the largest blur reach whenever a depth map exists. The CPU source is already oriented, so its metadata carries identity orientation and oriented dimensions.

source::encode writes an IESR stream: a versioned binary header that keeps every float bit for bit, then the half-float samples as per-channel planes of high and low bytes, each row delta coded against its left neighbour, then any context grids as raw f32 samples, all compressed with zstd level 1. The header records each context grid’s level, origin and size. source::decode reads it on the server and in the browser with a pure Rust decoder.

When the request’s Accept-Encoding lists zstd, the source endpoint sends the same planes uncompressed inside an IESP stream, compresses the whole body as one zstd frame, and sets Content-Encoding: zstd. The browser then decompresses natively while the body downloads, and the wasm decoder skips its own zstd pass. Browsers that do not advertise zstd get the IESR stream.

Resolution and geometry

Both renderers use geom::preview_ratio and geom::resample_target. A non-quality preview whose cropped source is larger than its output reduces at the spatial boundary. Noise reduction, retouch, and capture sharpening remain at source resolution; dehaze and later work run near preview size.

Creative sharpening measures its radius in source pixels. A reduced preview or a resized export scales the radius by its output ratio, so the fitted view sharpens the same image detail as 100% zoom instead of much coarser structure.

Grain is defined on the source pixel grid. A reduced render averages the grain each output pixel covers, so the fitted view shows what a downscaled full-size export would show: coarse grain keeps its pattern and fine grain softens instead of turning into larger, stronger speckles.

The reduction uses separable Lanczos3 in scene-linear space. Geometry samples fractional source coordinates with the same Catmull-Rom bicubic kernel on CPU and GPU. The reduction ratio comes from the active crop, not the full sensor.

Region of interest

RenderOptions.roi is a normalized rectangle in display space after orientation, perspective, angle, and user crop. The backend composes it into the geometry crop, so lens, masks, and transforms use the normal path. Vignette and grain remap tile coordinates to the full frame to keep their appearance stable.

The editor requests the visible region at the required device-pixel size. It draws the tile over the full preview so overlays and surrounding pixels remain stable while the tile loads. Base, original, and ROI requests use separate render-queue lanes so concurrent views do not cancel one another.

Dehaze resolution

Both paths estimate atmosphere after reducing the long edge to at most 256 pixels. When the short edge is at least 512 pixels, both solve the guided-filter coefficients at quarter resolution, upsample those coefficients, and evaluate transmission against the full guide. CPU and GPU sampling rules must change together.

Negative dehaze adds haze by raising the transmission to a power that grows with the slider, so a pixel moves toward the atmosphere by 1 - t^(3|amount|). Distant or already hazy areas fog first, while clear, dark foreground keeps its contrast until the slider is near its end.

Lens blur

Lens blur is a Tone operator at order -33, after dehaze and before texture and clarity, on the scene-linear image at the same resolution dehaze sees. Mask layers start from the blurred image. It reads the depth raster named by lens_blur.depth_raster_id, a disparity map (255 nearest) in the same scene space as generated masks, through the inverse lens warp.

• Focus: the depth levels whose distance rank overlaps Focal Range stay sharp. Outside that band the signed circle of confusion grows linearly with disparity, and the depth extreme farthest from the band reaches the full Amount, at most 2% of the long edge.
• Blur: both renderers average the image into a grid of power-of-two cells sized so the blur radius spans at most 16 of them, anchored to the oriented frame. Each cell gathers the cells whose aperture covers it; a farther cell never spreads over a nearer one. Bokeh tests offsets against a polygon, and Boost weights bright cells up. The result is upsampled with depth-aware bilinear weights and blended over the sharp pixel by the larger of its own circle of confusion and the spread that reached it, so a blurred foreground also covers sharp pixels behind its edge.
• The GPU source is sensor-oriented and may be a window, so the shaders map each texel to the oriented frame before sampling depth or the grid.

Camera profiles and tone

RAW rendering has three profile outcomes:

• A matched or explicitly selected DCP supplies camera matrices, optional HueSatMap and LookTable, and an optional profile tone curve.
• Default Color supplies no tables and uses the built-in tone curve.
• Flat skips the profile stage and uses only the camera matrix.

Non-RAW input is always Flat. Neutral RAW rendering carries no hidden content-dependent exposure. Matched DCP baseline exposure applies only through its explicit profile control.

The DCP HueSatMap runs in the Tone stage right after user white balance and before exposure and every user tone and color operator, matching the DNG order. Sliders therefore act on the profile-rendered color. The table looks up values from before the baseline exposure gain, so that gain does not shift which slice of a value-dependent table a pixel reads. Mask layers apply the table to each layer before blending, and range masks sample color after it. Range masks read an edge-preserving smoothed copy of that display color (a self-guided filter per channel, radius 1/1024 of the long edge), so noise cannot speckle the selection.

Working textures use linear scene-referred sRGB primaries except while DCP tables operate in linear ProPhoto. The profile tone curve precedes output gamut conversion. Curves then act on the display-encoded values, so curve coordinates match what the screen shows. A luma lift that would push a channel past white desaturates toward its target luminance instead of clipping. The creative LUT runs last in display-referred sRGB. For Display P3 output, colors are first gamut-mapped into sRGB, graded, and then converted back to P3, so a LUT gives the same look in either output space. While a LUT is active, P3 colors outside sRGB clip to the sRGB boundary.

Output color and warnings

RenderOptions.output_color_space selects sRGB or Display P3. The working space stays unchanged; the primary matrix, gamut mapping, transfer curve, and matching ICC profile apply at output. Gamut mapping keeps Oklab hue and lightness and reduces chroma until the color fits the output space. Lightness above white maps to white.

Clipping and gamut classes travel through the GPU display texture’s alpha channel until readback. Any pass added after tone must preserve or recompute alpha. CPU and GPU classify the undithered, post-LUT display result. Warnings are preview-only and never enter edits or exports.

Export final stage

raw_pipeline::finish::final_stage is a CPU step between the render and the encoder. Only exports use it. It runs after either renderer has produced display-encoded pixels, so CPU and GPU exports stay identical from there on by construction.

1. Enlarge. With RenderOptions.enlarge set and a crop smaller than max_edge, the renderer finishes at the crop’s own size and a Lanczos3 pass (fast_image_resize, 8- or 16-bit) scales the result up to max_edge. Downscaling stays inside the renderers through max_edge, so every edit operator, grain and sharpening included, runs at the resolution it would without the enlarge.
2. Output sharpening. RenderOptions.output_sharpen runs a luma-only unsharp mask (finish/sharpen.rs) on the final pixels, after any resize. The Gaussian radius follows the medium: fixed for screen, scaled by print ppi against a 300 ppi reference for glossy and matte paper, clamped to 0.5–4 px. Each result is clamped to its 3×3 neighborhood, so edges cannot overshoot into halos. It is separate from the Detail panel’s creative sharpening, which runs inside the renderers.
3. Watermark. RenderOptions.watermark composites a decoded PNG (finish/watermark.rs) last, so it is never resampled with the photo or sharpened. Its size and inset are fractions of the photo’s short edge; it is placed at one of nine anchors and blended with its own alpha times the chosen opacity. The PNG is treated as sRGB and converted through linear light for Display P3 exports.

Masks

Masks are manifest operators with dedicated render paths. Components evaluate in scene space and stay anchored through lens and geometry changes. Generated masks are stored as ordinary r8 rasters, so model inference does not run during preview or export.

CPU builds local layer images and blends them once. GPU computes component weights and blends each layer through dedicated mask passes. Masked sharpening uses a per-pixel amount map on both paths.

Add an operator

1. Choose its stage and CPU implementation.
2. Declare its GpuRoute.
3. Register it in default_registry().
4. Add manifest round-trip coverage when it persists data.
5. Add fused shader parity or dedicated CPU/GPU parity coverage when it changes pixels.
6. Update this page when ownership, order, resolution, or color space changes.