HDR transcoding and tone mapping on a home media server

Tone mapping squeezes a wide luminance range into a narrow one, keeping the detail in highlights and shadows. It’s needed when the file carries HDR and the display or the playback path wants SDR. Where your server does it, and whether it should at all, decides what a 4K film costs you.

1. What the server reads as HDR

Two signals, both taken from the probe and not from the filename:

Counts as	When
HDR10	The transfer function is SMPTE ST.2084 (the PQ curve), or the file carries mastering-display or content-light-level side data, which stays reliable even where the transfer characteristic went missing
HLG	The transfer function is ARIB STD-B67, the broadcast curve

Dolby Vision has no branch of its own in the streaming decision. The scanner reads its profile and level, and your library statistics count those files under a dynamic range of their own on the Statistics tab. Most Dolby Vision rips carry an HDR10 base signalling PQ, so the file converts as HDR10 and the per-scene metadata stays uninterpreted.

2. When the server converts

Client	What happens
Desktop and mobile apps	They tell the server they can open the file themselves, so the session takes the direct-file route and the bytes leave as they sit on disk. The HDR test is never reached
Modern browsers	The profile covers H.264 through AV1 at up to 4K and 25 Mbps. HDR10 is added where the display shows HDR and the browser decodes 10-bit HEVC; anywhere else an HDR film is tone mapped unless the viewer asks otherwise
AirPlay receivers	They keep HDR10 and take HEVC, so an Apple TV gets the range as authored
Chromecast default receiver	It takes H.264 and AAC only, so an HDR title cast from a phone is converted for the codec and for the range

Every conversion you skip keeps the quality intact and leaves a core free for a session that has no choice.

3. Which backend does the tone map

Backend	Tone mapping runs on
NVIDIA (NVENC or CUDA)	The graphics hardware, through libplacebo on a Vulkan device chosen independently of the encoder: discrete adapters first, then integrated, with software rasterisers and virtual devices excluded
Intel Quick Sync	Its video-processing unit. On a 4K HDR title, thirty seconds took 3.43 seconds there against 27.4 seconds through libplacebo on the same integrated chip
VAAPI on Linux (Intel and AMD)	The video-processing unit, while the decoded frames are still on the device. After a software decode fallback the processor chain runs and the result is uploaded for the encoder
AMD AMF on Windows	The processor, with the encode on the card. The same card on a Linux host converts on the GPU through VAAPI
Apple VideoToolbox on macOS	The processor, with the encode on the hardware

Auto takes the first backend that validates, in the order NVENC, Quick Sync, VAAPI, VideoToolbox, AMF, then on Linux a codec-backed V4L2 device.

To check or override it, an administrator opens Settings, the Quality tab, Server transcoding. The Info & Diagnostics tab lists what your machine validated as chips under Hardware acceleration backends, and a backend that never passed its runtime probe isn’t offered.

4. The picture stays the same across backends

The operator is Hable everywhere. The software chain converts to linear light at a nominal peak of a hundred nits, tone maps with Hable and no desaturation, then returns to BT.709, and libplacebo gets the same Hable operator so the two match. Moving the server from the processor to the graphics card therefore doesn’t change the grade, and a measurement from before the move still means something after it.

5. When a card drops out mid-session

1. A hardware session that dies before its first segment is relaunched with software decode, the hardware encoder kept and tone mapping on the processor.
2. If the second launch dies the same way, the backend is marked tainted. The session restarts fully in software, and every later session in that process goes straight to libx264.

The log names each step, so search it:

Log line	Meaning
Hardware decode failed at startup on {backend}; retrying with software decode	The first step
Hardware acceleration {backend} tainted	The second step. The flag lives in memory, so restarting the server clears it and the backend is probed again
GPU HDR tonemap support detected: False	The libplacebo filter or the Vulkan device never came up
A warning that a named Vulkan device corrupts its frame transfers	That device failed the check that verifies them

6. Measure with real media

Resolution, source codec, bit depth, burnt-in subtitles and output bitrate all move the cost of a transcode. HDR with burn-in is the heaviest thing the server does, because the overlay wants the frame back in system memory and only NVIDIA and Quick Sync can keep the hardware encoder through it. On any other backend that combination drops the session into software, and a library of disc rips with bitmap subtitles is where you’ll meet it. The disc guide covers the cost of picking one of those tracks.

1. Pick a 4K HDR file that stands for your library, a film you actually watch.
2. Play it through the client and the network you’ll actually use.
3. Let it run long enough for the encoder to settle, ideally the whole film.

A capability probe only tells you a device is there. Sustained load shows whether playback holds up.

Order of preference: direct HDR playback first, then an accelerated tone-map path that has survived a real session, then the processor, which goes everywhere and wants generous headroom at high resolution.

Frequently asked questions