Complete Guide: Implementing DNG Output for Handheld Super-Resolution

[Shadetail]

Complete Guide: Implementing DNG Output for Handheld Super-Resolution

This guide documents how to successfully implement 16-bit DNG output for the Handheld Multi-Frame Super-Resolution project. The implementation preserves correct colors, white balance, and lens correction while maintaining full RAW editing flexibility in Adobe Camera Raw and RawTherapee.

Difficulty Level: Intermediate
Time Required: 2-4 hours
Platform: Linux (Ubuntu/WSL2 tested), adaptable to macOS

This guide is generated by Claude Code Sonnet 4.5, effectively summarizing the 10h it took us to implement this alongside assistance of ChatGPT5 Pro. Note that AIs may have made incorrect assumptions or claims in places, so take it with a grain of salt. But final result is a working implementation, was worth burning 25$ of API credits on it Hope it saves you some time.

---

Table of Contents

1. Overview
2. Prerequisites Installation
3. Understanding the Challenge
4. Implementation Steps
5. The Critical Fix: AnalogBalance
6. Code Changes
7. Testing Your Implementation
8. Troubleshooting
9. Technical Deep Dive

---

Overview

What We're Building

The goal is to save the super-resolution output as a 16-bit DNG file that:

• ✅ Opens correctly in Adobe Camera Raw and RawTherapee
• ✅ Has accurate colors (no pink/magenta tint)
• ✅ Defaults to "As Shot" white balance
• ✅ Has working Temp/Tint sliders (not maxed out)
• ✅ Includes lens distortion correction
• ✅ Preserves full camera color profile
• ✅ Maintains 16-bit depth and dynamic range

Why This Is Challenging

The super-resolution algorithm works best with white-balanced RGB data, but DNG LinearRaw format expects un-white-balanced camera-native data. This mismatch causes color casts and metadata issues if not handled correctly.

Our solution uses DNG's AnalogBalance tag to declare the baked-in white balance, allowing ACR to process the file correctly.

---

Prerequisites Installation

1. Install ExifTool

ExifTool is required for reading and writing DNG metadata.

Ubuntu/Debian/WSL2:

sudo apt update
sudo apt install -y libimage-exiftool-perl
exiftool -ver  # Should show version 12.40 or higher

macOS:

brew install exiftool

Verify installation:

which exiftool  # Should return a path

2. Compile dng_validate (Adobe DNG SDK)

The dng_validate tool is required to finalize DNG files. Adobe provides source code but not pre-compiled binaries.

Install Build Dependencies

Ubuntu/Debian/WSL2:

sudo apt install -y build-essential gcc g++ make unzip libjpeg-dev

macOS:

xcode-select --install
brew install jpeg

Download and Compile

We'll use the community makefile from abworrall/go-dng which provides Linux build support:

cd ~/projects  # or your preferred location

# Download DNG SDK with community makefile
git clone https://github.com/abworrall/go-dng.git
cd go-dng/sdk

# Compile dng_validate
make

# Verify compilation
ls -lh bin/dng_validate  # Should show ~25MB binary
./bin/dng_validate        # Should show usage information

Add to PATH (optional but recommended):

# Add to ~/.bashrc or ~/.zshrc
export PATH="$HOME/projects/go-dng/sdk/bin:$PATH"

# Reload shell
source ~/.bashrc

Alternative: Compile Adobe's Official SDK

If you prefer Adobe's official SDK:

# Download from Adobe
wget https://download.adobe.com/pub/adobe/dng/dng_sdk_1_6.zip
unzip dng_sdk_1_6.zip
cd dng_sdk_1_6/dng_sdk

# You'll need to create your own Makefile or use CMake
# The abworrall/go-dng approach is simpler for Linux

3. Fix JPEG Preview Issue (Optional but Recommended)

The compiled dng_validate may fail on large images with "No JPEG encoder" error when trying to generate previews. Since previews aren't essential for our use case, we can disable them:

Patch the source code:

cd go-dng/sdk/dng_sdk_1_6/dng_sdk/source

# Edit dng_validate.cpp line 401
# Change: for (uint32 previewIndex = 0; previewIndex < 2; previewIndex++)
# To:     for (uint32 previewIndex = 0; previewIndex < 0; previewIndex++)

# Using sed:
sed -i 's/previewIndex < 2/previewIndex < 0/' dng_validate.cpp

# Recompile
cd ../..
make clean
make

This disables preview generation, avoiding the JPEG encoder requirement while still producing valid DNGs.

---

Understanding the Challenge

The Color Space Problem

The super-resolution algorithm in this project was designed/tested with white-balanced RGB data. When we tried to save as camera-native (un-white-balanced) DNG, we encountered:

• Pink/magenta tint throughout the image (especially in shadows)
• Raised blacks that couldn't be fixed with simple adjustments
• Only 3% improvement even after applying per-channel offset corrections

Why "Just Remove WB" Didn't Work

The Pixel3 Night Sight algorithm (which this implements) expects white-balanced data. Processing un-white-balanced Bayer data through the super-resolution pipeline introduced color artifacts that couldn't be corrected in post.

The Solution: White-Balanced LinearRaw with AnalogBalance

We bake white balance into the pixel data (which the SR algorithm needs), but use DNG's AnalogBalance tag to tell ACR:

• "I already applied these gains to the stored data"
• ACR applies (AnalogBalance)^-1 internally to return to camera-native space
• Then proceeds normally through its color pipeline

Result: Perfect colors with full RAW editing flexibility!

---

Implementation Steps

Step 1: Modify White Balance Application

File: handheld_super_resolution/utils_dng.py

Location: Lines 180-204 in load_dng_burst() function

Ensure white balance IS applied during Bayer data loading:

#### Performing white balance and normalizing into 0, 1
# IMPORTANT: Applying white balance during loading
# The super-resolution algorithm was designed/tested with white-balanced data
# We'll save this as white-balanced LinearRaw (with AsShotNeutral=1 1 1)

if np.issubdtype(type(ref_raw[0, 0]), np.integer):
    # Black level correction, normalization, and white balance
    ref_raw = ref_raw.astype(DEFAULT_NUMPY_FLOAT_TYPE)
    for i in range(2):
        for j in range(2):
            channel = CFA[i, j]
            ref_raw[i::2, j::2] = (ref_raw[i::2, j::2] - black_levels[channel]) / (white_level - black_levels[channel])
            # CRITICAL: Apply white balance
            ref_raw[i::2, j::2] *= white_balance[channel] / white_balance[1]

    ref_raw = np.clip(ref_raw, 0.0, 1.0)

Do the same for raw_comp array (lines 197-204).

Step 2: Calculate AnalogBalance in save_as_dng()

File: handheld_super_resolution/utils_dng.py

Location: After line 249 in save_as_dng() function

Add code to calculate AnalogBalance from the white balance gains:

raw = rawpy.imread(ref_dng_path)
black_levels = raw.black_level_per_channel[:3]  # RGB only

# Get the white balance gains that were applied during loading
# These need to be declared in AnalogBalance so ACR knows they're already baked in
camera_wb = raw.camera_whitebalance[:3]  # [R, G, B]
analog_balance = [camera_wb[0]/camera_wb[1], 1.0, camera_wb[2]/camera_wb[1]]  # Normalize to G=1

print(f"Camera white balance applied: R={analog_balance[0]:.6f}, G={analog_balance[1]:.6f}, B={analog_balance[2]:.6f}")
print(f"Setting AnalogBalance={analog_balance[0]:.6f} {analog_balance[1]:.6f} {analog_balance[2]:.6f}")
print(f"Setting AsShotNeutral=1 1 1 (pixels already white-balanced)")

new_white_level = 2**16 - 1

Step 3: Set AnalogBalance and AsShotNeutral in Metadata

Location: Around line 367 in save_as_dng(), after dng_validate runs

Replace the metadata fixing section with:

# Fix metadata after dng_validate (it resets some values)
black_level_rgb = f"{black_levels[0]} {black_levels[0]} {black_levels[0]}"
white_level = "65535 65535 65535"

# Format AnalogBalance for exiftool
analog_balance_str = f"{analog_balance[0]} {analog_balance[1]} {analog_balance[2]}"

# CRITICAL: Set AnalogBalance to the WB gains we applied, and AsShotNeutral to "1 1 1"
# AnalogBalance tells ACR: "these gains were already applied to the stored data"
# AsShotNeutral tells ACR: "neutral is at equal RGB values after removing AnalogBalance"
# This allows ACR's Temp/Tint model to work normally without pinning the tint slider
# Per DNG spec: CameraToXYZ_D50 = FM · D · (AB·CC)^-1, where AB^-1 undoes the baked gains

cmd = f'''{EXIFTOOL_PATH} -n -overwrite_original \
    -IFD0:BlackLevel="{black_level_rgb}" \
    -WhiteLevel="{white_level}" \
    "-DNGBackwardVersion=1 2 0 0" \
    -BaselineExposure=0 \
    -AnalogBalance="{analog_balance_str}" \
    -AsShotNeutral="1 1 1" \
    -AsShotWhiteXY= \
    -BlackLevelDeltaH= \
    -BlackLevelDeltaV= \
    -XMP:ColorTemperature= \
    {outpath.as_posix()}'''
os.system(cmd)

Step 4: Copy Lens Correction Opcodes

Location: After profile data copying, before cleanup

Add code to extract and inject lens correction opcodes:

# Copy lens correction opcodes (OpcodeList3 contains WarpRectilinear for distortion correction)
# Extract OpcodeList3 from source DNG and inject into output DNG
# This allows ACR to apply lens distortion/CA/vignetting correction
if copy_lens_correction:
    opcode3_file = outpath.parent / 'opcodelist3.bin'
    cmd = f'{EXIFTOOL_PATH} -b -SubIFD:OpcodeList3 {ref_dng_path} > {opcode3_file.as_posix()} 2>/dev/null'
    os.system(cmd)

    # Check if OpcodeList3 was extracted (file size > 10 bytes)
    if opcode3_file.exists() and opcode3_file.stat().st_size > 10:
        # Inject into IFD0 of output DNG
        cmd = f'{EXIFTOOL_PATH} -overwrite_original -n \'-IFD0:OpcodeList3<={opcode3_file.as_posix()}\' {outpath.as_posix()}'
        os.system(cmd)
        print(f"Copied OpcodeList3 (lens correction) from source DNG ({opcode3_file.stat().st_size} bytes)")
        # Clean up
        if opcode3_file.exists():
            opcode3_file.unlink()
    else:
        print("No OpcodeList3 found in source DNG (lens correction unavailable)")
else:
    print("Lens correction disabled (--copy_lens_correction=False)")

Step 5: Add Command-Line Flag (Optional)

File: run_handheld.py

Add a flag to enable/disable lens correction:

# Around line 113, after other arguments
## DNG Output Options
parser.add_argument('--copy_lens_correction', type=str2bool, default=True,
                   help='Copy lens correction opcodes from source DNG (WarpRectilinear distortion correction)')

Update the save_as_dng() call (around line 199):

save_as_dng(handheld_output, ref_img_path, outpath, copy_lens_correction=args.copy_lens_correction)

Update the function signature in utils_dng.py:

def save_as_dng(np_img, ref_dng_path, outpath, copy_lens_correction=True):

---

The Critical Fix: AnalogBalance

Understanding the Problem

When we baked white balance into the pixels but set AsShotNeutral="1 1 1", ACR thought the data was un-white-balanced. ACR then tried to "undo" white balance that wasn't there (from its perspective), pushing the Tint slider to -150 (maximum green correction) to compensate.

The Solution

The DNG specification provides AnalogBalance specifically for this situation:

From DNG Spec 1.6:

"AnalogBalance defines the gain (scaling) that has been applied to the stored raw values."

How It Works

The DNG color pipeline formula:

CameraToXYZ_D50 = FM · D · (AB·CC)^-1

Where:

• FM = ForwardMatrix (white-balanced camera → XYZ D50)
• D = White balance diagonal matrix (from AsShotNeutral)
• AB = AnalogBalance (gains already applied to stored data)
• CC = CameraCalibration

By setting AnalogBalance = [2.203, 1.0, 1.687] (the white balance gains we applied), the (AB)^-1 term in the pipeline undoes those gains internally, allowing ACR's white balance model to work normally.

Before and After

Before (without AnalogBalance):

• Tint slider: -150 (maxed out, unusable)
• Colors: Correct after hitting "As Shot" WB
• Problem: No headroom to adjust greener

After (with AnalogBalance):

• Tint slider: 0 (centered, full range available)
• Colors: Correct with "As Shot" WB
• Adjustment range: Full green ↔ magenta control

---

Code Changes

Complete save_as_dng() Function

Here's the complete function with all fixes applied:

def save_as_dng(np_img, ref_dng_path, outpath, copy_lens_correction=True):
    '''
    Saves a RGB numpy image as DNG with proper white balance handling.

    Parameters
    ----------
    np_img : numpy array
        RGB image (white-balanced, linear, 0.0-1.0 range)
    ref_dng_path : Path
        Path to source DNG for metadata extraction
    outpath : Path
        Output save path
    copy_lens_correction : bool, optional
        Whether to copy lens correction opcodes (OpcodeList3) from source DNG.
        Default is True. Set to False to disable lens distortion correction.
    '''
    assert np_img.shape[-1] == 3

    # Get metadata from source DNG
    raw = rawpy.imread(ref_dng_path)
    black_levels = raw.black_level_per_channel[:3]  # RGB only

    # Calculate AnalogBalance from applied white balance
    camera_wb = raw.camera_whitebalance[:3]  # [R, G, B]
    analog_balance = [camera_wb[0]/camera_wb[1], 1.0, camera_wb[2]/camera_wb[1]]

    print(f"Setting AnalogBalance={analog_balance[0]:.6f} {analog_balance[1]:.6f} {analog_balance[2]:.6f}")
    print(f"Setting AsShotNeutral=1 1 1 (pixels already white-balanced)")

    # Convert to 16-bit
    new_white_level = 2**16 - 1
    np_img_corrected = np.clip(np_img, 0.0, 1.0)

    black_level_uniform = black_levels[0]
    np_int_img = np.empty_like(np_img_corrected, dtype=np.float32)
    for c in range(3):
        np_int_img[:,:,c] = black_level_uniform + np_img_corrected[:,:,c] * (new_white_level - black_level_uniform)

    np_int_img = np.clip(np_int_img, 0, 2**16 - 1).astype(np.uint16)

    # Save as TIFF
    save_as_tiff(np_int_img, outpath)

    tmp_path = outpath.parent / 'tmp.dng'
    if os.path.exists(tmp_path):
        os.remove(tmp_path)

    # Copy metadata from source DNG using exiftool
    # (Initial metadata copy - see full implementation for complete exiftool commands)

    # Run dng_validate to finalize DNG
    output_base = outpath.with_suffix('').as_posix()
    cmd = f'{DNG_VALIDATE_PATH} -16 -dng {output_base} {tmp_path.as_posix()}'
    ret = os.system(cmd)

    if ret != 0:
        warnings.warn(f'dng_validate returned non-zero exit code: {ret}')

    # Fix metadata after dng_validate
    black_level_rgb = f"{black_levels[0]} {black_levels[0]} {black_levels[0]}"
    white_level = "65535 65535 65535"
    analog_balance_str = f"{analog_balance[0]} {analog_balance[1]} {analog_balance[2]}"

    # Set AnalogBalance and AsShotNeutral
    cmd = f'''{EXIFTOOL_PATH} -n -overwrite_original \
        -IFD0:BlackLevel="{black_level_rgb}" \
        -WhiteLevel="{white_level}" \
        "-DNGBackwardVersion=1 2 0 0" \
        -BaselineExposure=0 \
        -AnalogBalance="{analog_balance_str}" \
        -AsShotNeutral="1 1 1" \
        -AsShotWhiteXY= \
        -BlackLevelDeltaH= \
        -BlackLevelDeltaV= \
        -XMP:ColorTemperature= \
        {outpath.as_posix()}'''
    os.system(cmd)

    # Copy profile data (ProfileHueSatMapData, ProfileLookTableData, etc.)
    # (See full implementation for binary profile data extraction/injection)

    # Copy lens correction opcodes
    if copy_lens_correction:
        opcode3_file = outpath.parent / 'opcodelist3.bin'
        cmd = f'{EXIFTOOL_PATH} -b -SubIFD:OpcodeList3 {ref_dng_path} > {opcode3_file.as_posix()} 2>/dev/null'
        os.system(cmd)

        if opcode3_file.exists() and opcode3_file.stat().st_size > 10:
            cmd = f'{EXIFTOOL_PATH} -overwrite_original -n \'-IFD0:OpcodeList3<={opcode3_file.as_posix()}\' {outpath.as_posix()}'
            os.system(cmd)
            print(f"Copied OpcodeList3 (lens correction) from source DNG ({opcode3_file.stat().st_size} bytes)")
            if opcode3_file.exists():
                opcode3_file.unlink()

    # Cleanup
    tif_path = outpath.with_suffix('.tif')
    if os.path.exists(tif_path):
        os.remove(tif_path)

---

Testing Your Implementation

Basic Test

# With default settings (scale=1, lens correction enabled)
python run_handheld.py --impath test_burst --outpath output.dng

# With super-resolution (2x upscale)
python run_handheld.py --impath test_burst --outpath output.dng --scale 2

# Without lens correction
python run_handheld.py --impath test_burst --outpath output.dng --copy_lens_correction False

Verify Metadata

exiftool -n -s -PhotometricInterpretation -AsShotNeutral -AnalogBalance \
         -ColorMatrix1 -ForwardMatrix1 -ProfileHueSatMapData1 \
         -OpcodeList3 -BaselineExposure -BlackLevel -WhiteLevel output.dng

Expected output:

PhotometricInterpretation       : 34892
AsShotNeutral                   : 1 1 1
AnalogBalance                   : 2.203123573 1 1.687498808
ColorMatrix1                    : [matrix values]
ForwardMatrix1                  : [matrix values]
ProfileHueSatMapData1           : (large array of 8100 float values)
OpcodeList3                     : (Binary data XXX bytes, use -b option to extract)
BaselineExposure                : 0
BlackLevel                      : 800 800 800
WhiteLevel                      : 65535 65535 65535

Test in Adobe Camera Raw

1. Open the DNG - Should open without errors
2. Check White Balance - Should default to "As Shot"
3. Check Tint Slider - Should be near 0, NOT at -150
4. Adjust Tint - Should be able to move in both directions
5. Check Colors - Should be neutral, no pink/magenta cast
6. Enable Profile Corrections - Lens distortion correction should apply (if enabled)

Test in RawTherapee

1. Open the DNG - Should open without errors
2. Check "From Metadata" checkbox - Should be available in Lens Correction panel
3. Colors - Should be neutral without manual corrections

---

Troubleshooting

Issue: Pink/Magenta Tint in Output

Cause: White balance not being applied during loading, or AnalogBalance not set correctly.

Solution:

• Verify white balance IS applied in load_dng_burst() (lines 193 in utils_dng.py)
• Verify AnalogBalance is calculated and set in save_as_dng()
• Check console output for "Setting AnalogBalance=..." message

Issue: Tint Slider Maxed at -150

Cause: AnalogBalance not set, or set incorrectly.

Solution:

• Check that AnalogBalance values match the applied white balance gains
• Verify the ExifTool command includes -AnalogBalance="{analog_balance_str}"
• Run exiftool -AnalogBalance output.dng to verify it's set

Issue: dng_validate Fails with "No JPEG encoder"

Cause: dng_validate trying to generate JPEG preview but lacking JPEG support.

Solution:

• Apply the patch to skip preview generation (see Prerequisites section)
• Or ensure libjpeg-dev is installed before compiling

Issue: ACR Can't Open the File

Cause: Missing required metadata tags or invalid DNG structure.

Solution:

• Ensure DNGBackwardVersion >= 1.2.0.0 for profile support
• Verify PhotometricInterpretation=34892 (LinearRaw)
• Check that dng_validate ran successfully (return code 0)
• Remove SIMPLE.dng approach - don't remove ColorMatrix/ForwardMatrix

Issue: Colors Correct but Blacks Raised

Cause: This is a minor artifact of the white-balanced approach.

Solution:

• Adjust blacks slider in ACR (trivial fix)
• Or implement per-channel offset removal (advanced, only 3% improvement)

Issue: Lens Correction Not Applying

Cause: OpcodeList3 not copied, or not present in source DNG.

Solution:

• Check console output for "Copied OpcodeList3..." message
• Verify source DNG has OpcodeList3: exiftool -OpcodeList3 source.dng
• If source lacks OpcodeList3, lens correction isn't available
• Try with different source DNGs that include lens correction

Issue: ProfileHueSatMapData Not Copying

Cause: ExifTool needs -m flag to decode large binary arrays.

Solution:

• Ensure profile extraction commands use -m flag
• Extract to binary files first, then inject
• Check file sizes (should be >100KB for ProfileHueSatMapData)

---

Technical Deep Dive

Color Space Explained

Question: Are we in a display-referred color space after baking in white balance?

Answer: No. We're in white-balanced, linear, camera-native RGB.

What This Means:

1. Camera-Native Primaries

   • Still using the camera sensor's unique color primaries
   • NOT sRGB, Adobe RGB, or ProPhoto RGB
   • Color gamut is camera-specific

2. White-Balanced

   • Channels scaled to make neutral equal RGB values
   • Still in camera's native color space
   • AnalogBalance declares this scaling

3. Scene-Referred

   • Linear (proportional to scene luminance)
   • No gamma curve applied
   • Not display-ready
   • Preserves full dynamic range

The DNG Color Pipeline

When ACR opens your DNG:

1. Stored pixels (white-balanced camera RGB, linear)
         ↓
2. Apply (AnalogBalance)^-1  [undoes baked WB internally]
         ↓
3. Apply white balance D (from AsShotNeutral=1 1 1)
         ↓
4. Apply CameraCalibration matrix
         ↓
5. Apply ForwardMatrix → XYZ D50
         ↓
6. Convert XYZ D50 → ProPhoto RGB (ACR working space)
         ↓
7. Apply ProfileHueSatMapData (hue/saturation in ProPhoto)
         ↓
8. Apply ProfileLookTableData (3D LUT in ProPhoto)
         ↓
9. Convert to user's working space (sRGB/Adobe RGB/ProPhoto)
         ↓
10. Apply tone curve, exposure adjustments, etc.
         ↓
11. Convert to output space with gamma for display

Key Insight: AnalogBalance makes step 2 return the data to un-white-balanced camera space, then ACR proceeds normally through the rest of the pipeline.

Why Other Approaches Failed

Approach 1: Camera-Native (No WB)

• Tried: Don't apply WB, keep data in true camera space
• Result: Pink/magenta tint throughout image
• Why: Super-resolution algorithm doesn't handle unbalanced channels correctly

Approach 2: Camera-Native + Offset Removal

• Tried: Remove per-channel offsets (R=0.0007, G=0.0018, B=0.0015)
• Result: Only 3% improvement, tint persists
• Why: Color corruption is systematic, not just an offset issue

Approach 3: White-Balanced without AnalogBalance

• Tried: Bake WB, set AsShotNeutral="1 1 1"
• Result: Tint slider maxed at -150, unusable
• Why: ACR thinks data is un-WB'd, tries to "undo" WB that it thinks should be there

Approach 4: White-Balanced with AnalogBalance ✅

• Implementation: Bake WB, set AnalogBalance to gains, AsShotNeutral="1 1 1"
• Result: Perfect colors, centered tint slider, full editing range
• Why: Correctly declares the processing applied to stored data

Lens Correction Details

OpcodeList3 contains:

• WarpRectilinear opcode for geometric distortion correction
• Applied by ACR after linear mapping (early in pipeline)
• Works for both scale=1 and scale=2 (surprisingly!)

Why It Works for scale=2:

• The WarpRectilinear parameters are relative/normalized
• Geometric transformations scale appropriately with resolution
• ACR applies correction after reading pixel data but before further processing

Disabling Lens Correction:

python run_handheld.py --impath test_burst --outpath output.dng --copy_lens_correction False

---

Summary

What We Achieved

1. ✅ 16-bit DNG output with full dynamic range preservation
2. ✅ Correct colors without pink/magenta tint
3. ✅ Proper white balance defaulting to "As Shot"
4. ✅ Usable Temp/Tint sliders with full adjustment range
5. ✅ Lens distortion correction embedded and working
6. ✅ Complete camera profile (ColorMatrix, ForwardMatrix, ProfileHueSatMapData, ProfileLookTableData)
7. ✅ Works with scale=1 (demosaicing) and scale=2 (super-resolution)
8. ✅ Compatible with Adobe Camera Raw and RawTherapee

Key Insights

1. Super-resolution algorithms may require white-balanced data
2. AnalogBalance is the correct way to declare baked-in gains
3. DNG is flexible - white-balanced LinearRaw is valid and useful
4. Lens correction opcodes are more robust than expected
5. Color space remains camera-native despite white balance

Resources

• DNG Specification 1.6: https://helpx.adobe.com/camera-raw/digital-negative.html
• ExifTool Documentation: https://exiftool.org/
• Adobe DNG SDK: https://helpx.adobe.com/camera-raw/digital-negative.html#dng_sdk_download
• Community DNG SDK Makefile: https://github.com/abworrall/go-dng
• Original Project: https://github.com/Jamy-L/Handheld-Multi-Frame-Super-Resolution

---

Also attaching the chatlog of correspondence between Claude Code 4.5 Sonnet and ChatGPT 5 Pro, as it contains more techanical details and precise solutions to toughest problems we ran into:
ChatGPT5Pro - DNG profile metadata chatlog.md

[Shadetail]

Preserving Detail

After trying to use this with my Sony RX100VII DNGs, most of my results were coming out blurry with no detail, compared to the detail I'd see by stacking the images in photoshop and averaging them. After another day of digging and it turned out there are two separate issues, and after addressing them I'm now getting results that are better than manual stacking and averaging.

When I tested the outputs of the attached Samsung raws, they were better than photoshop stacking, but when I tried it with my Sony raws, I was getting worse results. Primary difference seems to be that my raws tend to be more underexposed and noisier, and that default setup just wasn't meant to handle that due to:

1. White balance corrupting the noise model
2. Accumulated robustness denoiser adding extra smoothing

Problem 1: White Balance Noise Model Bug

The code was applying white balance to raw data but not updating the noise coefficients. This broke everything at low light.

What was happening:

The DNG NoiseProfile tag (0xC761) defines noise as σ²ₙ = α·I + β for unprocessed raw data. The code multiplied pixels by white balance gains (e.g., R=2.39, G=1.0, B=2.09), but kept using the original α and β values.

When you multiply pixel intensities by gain g, noise variance scales as:

• α' = g·α
• β' = g²·β

For Sony RX100VII at ISO 4000, this meant:

• Original α averaged: 0.002057
• Original β averaged: 0.0000213
• Should be after WB: α = 0.003751, β = 0.0000782

That's a 1.8× underestimation of α and 3.7× underestimation of β. The algorithm thought the data was way cleaner than it actually was.

The fix:

I modified super_resolution.py to automatically apply the white balance correction. It now:

1. Reads per-channel NoiseProfile values from the DNG
2. Reads white balance gains from the DNG
3. Applies α' = g·α and β' = g²·β automatically
4. Uses the corrected values

The correction is automatic and sensor-agnostic. It should work correctly for any camera/ISO combination.

Manual override still available:

If you want to provide your own values:

python run_handheld.py --impath burst_folder --outpath output.dng \
    --alpha 0.003751 --beta 0.0000782

When verbose mode is enabled, you'll see the correction being applied:

White balance correction applied:
  WB gains: R=2.395, G=1.000, B=2.094
  Original α: 0.002057, corrected: 0.003751
  Original β: 0.00002131, corrected: 0.00007818

Problem 2: Accumulated Robustness Denoiser

Even with correct noise values, output was slightly better but still blurry. Tracked it down to the "accumulated robustness denoiser" being enabled by default.

This is a separate smoothing layer (merge mode) that runs during reference frame accumulation. It's described in section 2.3 of the IPOL paper.

Just turning off robustness mask completely would uncover many seams between tiles, but only disabling this resulted in minimal seams.

The fix:

Changed handheld_super_resolution/params.py line 79:

'merge':{'on':False,  # Was True, now False

Or override via API:

custom_params = {
    'accumulated robustness denoiser': {'merge': {'on': False}}
}

There's no CLI flag for this, you have to either modify params.py or use the Python API.

Robustness Power Parameter

I also added a power parameter that applies r^power to the robustness mask values, this is optional, and I keep it disabled by default, but it allows to navigate the tradeoff between remaining few visible seams and having detail in those areas. Bumping this up to 10 or above, seems like it effectively replaces few remaining seams with noisy pixels from the single image, which can be desirable depending on image and how visible the seams are.

Implementation:

Added to handheld_super_resolution/params.py line 50:

'robustness' : {
    'on':True,
    'power': 1.0,
    'tuning' : { ... }
},

Modified handheld_super_resolution/merge.py line 446 to apply the transform:

local_r_powered = local_r ** robustness_power
val[channel] += c*w*local_r_powered
acc[channel] += w*local_r_powered

Added parameter propagation in super_resolution.py lines 347-351:

if 'robustness' not in params["merging"].keys():
    params["merging"]["robustness"] = {}
if 'power' in params["robustness"].keys():
    params["merging"]["robustness"]["power"] = params["robustness"]["power"]

Added CLI flag in run_handheld.py:

parser.add_argument('--robustness_power', type=float, default=1.0,
    help='Robustness mask power (r^power). Higher values reject uncertain pixels more aggressively.')

Full Working Example

Python API with all fixes:

from handheld_super_resolution import process
from handheld_super_resolution.utils_dng import save_as_dng
import numpy as np
import glob

custom_params = {
    'scale': 1,
    'robustness': {'on': True, 'power': 1.0},
    'accumulated robustness denoiser': {'merge': {'on': False}},
    'post processing': {'on': False}
}

output_img, _ = process("burst_folder", {'verbose': 1}, custom_params)
output_img = np.nan_to_num(output_img)
output_img = np.clip(output_img, 0, 1)

ref_path = glob.glob("burst_folder/*.dng")[0]
save_as_dng(output_img, ref_path, "output.dng", copy_lens_correction=True)

CLI with all fixes:

python run_handheld.py --impath burst_folder --outpath output.dng \
    --robustness_power 1.0

Files Modified

• handheld_super_resolution/params.py - Added power parameter, disabled denoiser
• handheld_super_resolution/merge.py - Applied power transform
• handheld_super_resolution/super_resolution.py - Automatic WB correction, parameter propagation
• handheld_super_resolution/utils_dng.py - Return white_balance from load_dng_burst
• run_handheld.py - CLI exposure for robustness_power (alpha/beta already existed)
• README.md - Documented the new parameter

With these fixes the difference is night and day on high-ISO, low exposure:

You can also notice the little seam at top of the bird's beak, that's the tradeoff for getting this detail, but power parameter can be increased to get rid of it.

It's totally possible there are better ways of achieving these results, as I basically have no idea what I'm doing here, just tinkering with the stuff in an attempt to make it usable for my use case. Having the power of computational photography in an actual camera feels like a superpower. Many thanks to authors for this implementation!

[Jamy-L]

Thank you for the long report, I will investigate carefully ! I was able to install dng_validate on WSL using your method and to make it work. There was a small detail missing with the JPEG Preview Issue, but I managed to take care of that. I have update the readme accordingly, hoping it will help other people.

I wasn't yet able to test the rest in detail, but my first remarks are:

• I am surprised to a see a bug with the whitebalance in the dng, because the whitebalance is manually undone here before saving. If I understand right, you propose instead to keep the whitebalance, but set proper image tags so that other softwares take into account that WB is already applied. It sounds cleaner and I should indeed make it like that instead. I will dig a bit to try to understand why the result is different

• Thank you for pointing out that these tags were missing, I will add them. I wasn't aware that they were useful

• The remark on the noise profile makes a lot of sense. I am a bit careful though, because I know that the dng specification is not really precise about the precise meaning of these values, so whether or not they are already white balance-corrected is not 100% clear to me after all. I will make experiments on my side and correct it I can confirm that it must indeed be corrected.

• This final denoising mask can indeed make some part of the image very blurry. It activates at parts where the local robustness mask is the less than the max frame count parameter, which is 8 by default. Maybe I should disable it totally by default. When it comes to the robustness mask, I am glad to see that you can improve your results with a but of tweaking ! Note that the robustness function is basically a soft step function between 0 and 1, so you should be able to achieve something similar by tweaking the s and t parameters instead of adding a new one. Don't hesitate to visualise the accumulated robustness mask, you could see how many frames are locally merged, which is helpful to tweak s, t and M. By adding more frames, you may have something with even less noise

[Shadetail]

Also had JPEG preview issue, worked around it by just disabling jpeg preview since I didn't need it, but I'm glad that you managed to take care of it properly.

Awesome to hear that you're updating readme with some of this info!

I won't pretend to understand any of this better than you, but since I have the chat history of the entire debugging session at hand from the linked GPT discussion that I linked above, I fed all of that context back into ChatGPT 5 Pro and asked it to try to help you with those points you mentioned, so I attached the reply in case you find it useful:

GPT5Pro-HHMFSR-Reply.md

What I would add is that I would agree with disabling that final denoising mask by default, at least from experience with my raws. I tried to process a few more bursts with my changes, and have been getting great results so far.

Unrelated to that, the only remaining issue I'm seeing is that I get lifted blacks in my output dng, but that's easily fixed in practice by bringing down the blacks slider in ACR. I might try to debug that as well, and will post here if I find the solution.

I also uploaded the raw burst that I used for testing, in case you need any lower exposure noisy Sony raws to test with: https://drive.google.com/drive/folders/1WqzVuhhUwaH7irWzar7GxGE7s8yzgl07?usp=sharing

And thanks for the tips around robustness mask params! That sounds like a better way of going about it.

[Jamy-L]

Thank you for the burst, it came very to implement the fix ! I have fixed the whitebalance noise-profile issue as well. I have updated how the blackLevel tag is handled before writing the dng, to use the full bit range: maybe it can fix your black level issue.

I also updated the default settings and the rule for selecting the kernels : you may get better results with this. Lastly, if the image has a high iso, it would normally require to use large patches but the size is limited with the current code base due to implementation issues. I am currently refactoring and improving some section of the code to handle this (and make it faster !)

[Shadetail]

I tested the updates that you made, and with the defaults now I'm getting even better output than before, the details are as good with the tiny occasional seams now fully gone, and black level is also fixed now! Thanks a lot for the updates!