ComfyUI-ImgPatchEditor

A ComfyUI node for patching images using a difference detection algorithm. It detects changes between two images (original and patched), creates a mask of the differences, and merges them back together with configurable blending.

Features

• Automatic Patch Detection: Compares two images to find changed regions.
• Automatic Patch Alignment: Automatically aligns the edited image to the original, even if it's scaled or shifted.
• Configurable Masking: Control region size, minimum area, tolerance, blur, and mask expansion.
• Multiple Detection Algorithms: Choose between difference, l2_norm, and ssim.
• Batch Support: Process multiple image pairs in a batch, chaining the edits.
• Debug Outputs: Provides multiple outputs for visualizing the process, including a heatmap, raw mask, and debug overlay.

Installation

1. Clone this repository into your ComfyUI/custom_nodes/ directory.
2. Install dependencies:

   pip install -r requirements.txt

Usage

Add the Image Patch Editor node (Category: ImgPatch) to your workflow.

Inputs

• original_image: The base image.
• edited_image: The image with the changes.
• detection_alg: The algorithm used to detect differences.
  • difference: Simple pixel-wise difference.
  • l2_norm: L2 norm of the difference in a local region.
  • ssim: Structural Similarity Index Measure.
• region_size: Size of the blocks for difference detection.
• stride: Step size for the sliding window.
• min_area_size: Minimum size of a changed area to be kept in the mask.
• tolerance: Sensitivity of the difference detection. Higher values detect more changes.
• blur_radius: Radius of the Gaussian blur applied to the final mask.
• mask_expansion: Expands or shrinks the mask. Positive values expand, negative values shrink.
• edit_max_alpha: Maximum alpha for the edited image overlay.
• allow_rescale: If true, rescales the edited image to match the original's size.

Outputs

• merged_image: The original image with the patched areas from the edited image.
• final_mask: The final mask used for merging. This mask is the result of all post-processing, including mask_expansion and blur_radius. White indicates areas taken from the edited image, and black indicates unchanged areas from the original image.
• heatmap: A visualization of the differences between the two images. Brighter areas indicate larger differences.
• raw_mask: The initial binary mask created from the heatmap before any post-processing like expansion or blurring. White indicates detected changes, black indicates no change.
• debug_overlay: The original image with a red overlay highlighting the detected changes (based on the sized_mask before blurring).

Patch Matching

The node can automatically align the edited image to the original, even if it has been scaled or shifted. This is particularly useful when working with images of different resolutions or when the edited image is a cropped or slightly moved version of the original.

This automatic alignment is achieved through feature matching. Here's a brief overview of the process:

1. Feature Detection: The node uses the ORB (Oriented FAST and Rotated BRIEF) algorithm to identify keypoints and descriptors in both the original and edited images.
2. Feature Matching: The descriptors are then matched to find corresponding points between the two images.
3. Transform Estimation: Using the matched points, a robust transformation model is estimated using RANSAC (Random Sample Consensus). This model, a ScaleTranslationTransform, only considers uniform scaling and translation, ignoring rotation and shear. This makes it ideal for aligning images that have been resized or shifted.
4. Image Warping: The edited image is then warped using the estimated transformation to align it perfectly with the original image before the difference detection and merging process begins.

If the alignment process fails (e.g., not enough reliable matches are found), the node falls back to simply resizing the edited image to the dimensions of the original.

Workflow: Chaining Image Edits

A powerful feature of this node is its ability to chain multiple image modifications. The ImgPatchNode can take a batch of edited images and apply them sequentially. The output of one patch operation becomes the input for the next. This allows for iterative refinement of an image.

This is particularly useful for workflows involving generative AI models for image editing (e.g., Qwen-VL-Plus, Klein). Instead of regenerating the entire image at each step, you can apply small, targeted edits. The ImgPatchEditor detects only the significant changes and applies them as a patch. This has two main advantages:

1. Quality Preservation: It avoids the quality degradation that can occur with repeated full image generation or compression cycles. The original image quality is maintained in the areas that are not changed.
2. Efficiency: Processing only the changed parts of an image is faster than processing the entire image.

By tuning the detection parameters, you can control precisely how much of the generated change is applied, making it a flexible tool for creative image editing.

Tuning the Patch Detection

The effectiveness of the patch detection depends on a few key parameters:

• detection_alg: The algorithm used to compare the images. ssim is often a good choice for perceptual similarity, while difference and l2_norm are faster but less sophisticated.
• tolerance: This is the most critical parameter. It controls the sensitivity of the difference detection. A lower tolerance will only pick up very significant changes, while a higher tolerance will include more subtle differences. Start with a low value and increase it until you get the desired result.
• min_area_size: This helps to filter out small, noisy changes. Increase this value to ignore small artifacts.
• region_size: The size of the blocks to compare. Larger blocks can be faster but might miss fine details.

By experimenting with these settings, you can fine-tune the patch detection to fit your specific use case.

Examples

Simple Workflow

Download This example demonstrates the core editing capabilities of the ImgPatchEditor node. It applies two different patches to an original image. The first patch is the same size as the original, showing a straightforward edit. The second patch is a smaller, cropped section. The node automatically detects the correct scale and location of this smaller patch and applies it seamlessly.

Qwen Image Edit Workflow

Download)

This workflow showcases a practical application of the node for inpainting, specifically for logo removal using a generative model like Qwen Image edit or Flux.2.Klein The ImgPatchEditor takes the output from the Qwen model and intelligently extracts only the changes (the inpainted area where the logo was). It then merges this change back into the original image. This ensures that only the targeted area is modified, preserving the quality of the rest of the image and avoiding the subtle degradation that can occur when re-encoding the entire image.