Get pixel data

[vicdoval]

Hello!

Im trying to get the color of one particular pixel from a PImage but there is not PImage.get(x,y) neither loadPixels neither getNative.
Is there any workaround to acces this data?

Thanks in advance!

[neilcsmith-net]

Moving to discussions ...

This is a little complicated because there are multiple types of PImage that wrap the underlying Processing type. It also seems there's a bug with some types since the update to Processing 4 which I need to take a look at.

What type of component are you using? video:custom or video:gl:p2d/p3d?

Where is the PImage from - an input port or a file?

You can get access to the underlying Processing PImage in a video GL component by calling.

var pimage = image.find(processing.core.PImage.class).orElse(null);

It should never be null in this case, however the pixel data doesn't seem to work with an image loaded from a file. I need to look at that.

There is a simpler way to get access to the underlying PGraphics of the drawing surface.

var pgraphics = g.unwrap();

I've used var here because of the name conflicts with the wrapping type. Hope that helps. If you share a bit more about what you're trying to do I might be able to give more pointers.

[vicdoval]

Hi Neil, thank you for your fast response.
I have some particle system running on a video:gl:p2d component and I would like the particles to react to a webcam sourced image (from a video:capture node)
Neither image.find or g.unwrap gives me acces to loadPixels or get(x,y)

[neilcsmith-net]

OK, I think I've found a solution for this. We need to work around the fact that the PImage inputs from another node are actually a PGraphics. In PraxisLIVE we get the illusion of writing multiple sketches by directing the drawing in each node to a specific PGraphics. If we try to access the pixels of another graphics during draw, that code breaks.

The only solution I can think of right now is to use an offscreen scratch buffer that PraxisLIVE manages, draw the camera to that, and then read the pixels from it. The following graph code should show the general idea - I also added and exposed the read-only value as hex for testing -

@ /video root:video {
  .renderer OpenGL
  .width 800
  .height 600
  .fps 60.0
  @ ./screen video:output {
    .meta [map graph.x 574 graph.y 51]
    .always-on-top true
    .show-cursor true
  }
  @ ./p2d video:gl:p2d {
    .meta [map graph.x 420 graph.y 51 expose [array colour]]
    .code {
    @In PImage camera;
    @OffScreen PGraphics2D scratch;

    @P @ReadOnly String colour;

    @Override
    public void draw() {
        scratch.beginDraw();
        scratch.image(camera, 0, 0);
        var cam = scratch.find(processing.core.PGraphics.class).orElseThrow();
        cam.loadPixels();
        int col = cam.get(width / 2, height / 2);
        colour = Integer.toHexString(col & 0xFFFFFF);
        scratch.endDraw();
        noStroke();
        fill(col >> 16 & 0xFF, col >> 8 & 0xFF, col & 0xFF);
        rect(0, 0, width, height);
    }
}
  }
  @ ./capture video:capture {
    .meta [map graph.x 245 graph.y 51 graph.minimized true]
  }
  @ ./start-trigger core:start-trigger {
    .meta [map graph.x 44 graph.y 51]
  }
  ~ ./start-trigger!out ./capture!play
  ~ ./p2d!out ./screen!in
  ~ ./capture!out ./p2d!camera
}

If you only need one specific pixel, you could optimize this by using a 1x1 image to limit the amount of data that needs to be read back. Likewise, you could scale the image.

    @In PImage camera;
    @OffScreen(width = 1, height = 1) PGraphics2D scratch;
    
    @P @ReadOnly String colour;

    @Override
    public void draw() {
        scratch.beginDraw();
        int x = width / 2;
        int y = height / 2;
        scratch.image(camera, 0, 0, 1, 1, x, y, x + 1, y + 1);
        var cam = scratch.find(processing.core.PGraphics.class).orElseThrow();
        cam.loadPixels();
        int col = cam.get(0, 0);
        colour = Integer.toHexString(col & 0xFFFFFF);
        scratch.endDraw();
        noStroke();
        fill(col >> 16 & 0xFF, col >> 8 & 0xFF, col & 0xFF);
        rect(0, 0, width, height);
    }

Hope that helps you get going with this!

[neilcsmith-net]

Actually, just after writing that I thought of another way! 😄 We can unwrap the graphics and call beginDraw() on it again to fix things. So this should also work -

    @In PImage camera;

    @P @ReadOnly String colour;

    @Override
    public void draw() {
        var cam = camera.find(processing.core.PImage.class).orElseThrow();
        cam.loadPixels();
        int col = cam.get(width / 2, height / 2);
        colour = Integer.toHexString(col & 0xFFFFFF);
        g.unwrap().beginDraw();
        noStroke();
        fill(col >> 16 & 0xFF, col >> 8 & 0xFF, col & 0xFF);
        rect(0, 0, width, height);
    }