Add dithering methods

[roshavagarga]

Changes:

1. Changed all dither names to typically used informal/formal ones.
2. Removed decimal points from matrices - these seemed to serve no purpose.
3. Renamed and rearranged all dithers - publication date to modification and error diffusion vs ordered.
4. Added 10 new dithers - Fan, Shiau–Fan, Shiau–Fan 2, Sierra, Sierra (Two-row), Bayer (3x3), Bayer (8x8), Cluster Dot (4x4), Halftone (8x8), Void and cluster (14x14).
5. Switched from dashes to en dashes for names, wherever applicable.

New list of dithering methods:

1. Floyd–Steinberg - 1976
2. Atkinson - Floyd–Steinberg modification, 1980s
3. Sierra (Filter Lite) - Floyd–Steinberg modification, 1990
4. Fan - Floyd–Steinberg modification, 1993/1994
5. Shiau–Fan - Floyd–Steinberg modification, 1993/1994
6. Shiau–Fan 2 - Floyd-Steinberg modification, 1993/1994
7. Jarvis–Judice–Ninke - 1976
8. Stucki - Jarvis–Judice–Ninke modification, 1981
9. Burkes - Stucki modification, 1988
10. Sierra - Jarvis–Judice–Ninke modification, 1989
11. Sierra (Two-row) - Sierra modification, 1990
12. Bayer 2x2
13. Bayer 3x3
14. Bayer 4x4
15. Bayer 8x8
16. Ordered (3x3)
17. Cluster Dot (4x4)
18. Halftone (8x8)
19. Void and cluster (14x14)

Sources for above:
https://community.wolfram.com/groups/-/m/t/1383824
https://momentsingraphics.de/BlueNoise.html
https://surma.dev/things/ditherpunk/
https://tannerhelland.com/2012/12/28/dithering-eleven-algorithms-source-code.html
https://en.wikipedia.org/wiki/Dither
https://www.researchgate.net/figure/Error-diffusion-scheme-for-different-dithering-algorithms-Floyd-Steinberg-a-Jarvis_fig1_347225976
https://www.visgraf.impa.br/Courses/ip00/proj/Dithering1/ordered_dithering.html
https://beyondloom.com/blog/dither.html
https://www.cs.princeton.edu/courses/archive/fall00/cs426/lectures/dither/dither.pdf
http://caca.zoy.org/study/part3.html
https://cs.wellesley.edu/~pmetaxas/ei99.pdf
https://cs.wellesley.edu/~pmetaxas/pdcs99.pdf
https://www.iro.umontreal.ca/~ostrom/varcoeffED/SIGGRAPH01_varcoeffED.pdf
https://web.archive.org/web/20190316064436/http://www.efg2.com/Lab/Library/ImageProcessing/DHALF.TXT
https://core.ac.uk/reader/327118630
https://github.com/robertkist/libdither/blob/main/src/libdither/dither_errordiff_data.h
https://bisqwit.iki.fi/jutut/kuvat/ordered_dither/error_diffusion.txt
https://devlog-martinsh.blogspot.com/2011/03/glsl-8x8-bayer-matrix-dithering.html

[roshavagarga]

@rebane2001 Just a quick heads up - I am not a developer/programmer, but the above seemed simple enough to implement.

I don't have a way of testing any of the above, but I imagine that the changes I made should improve certain things as far as code readability, and other things as far as actual images/previews - I believe there were 1-2 matrices that had errors on top of the fluff I removed.

The dither methods I added are all pre-2020 and do not have active patents, thus are free to use. Since they are derivations of others present in the list, they should also offer some improvements.

If anything was changed erroneously and you think I can fix it - please highlight the issue and I will try my best.

[roshavagarga]

Added another dither method - Stevenson–Arce (1985), hexagonal, 7x4 matrix.

I don't plan on adding anything else at this point, unless requested to.

[roshavagarga]

Fixed Stevenson-Arce (duplicate uniqueID), added Bayer 3x3 and 8x8 (added +1 to every point compared to typical matrices, as previously done).

Current issues:

1. Broken matrices, unclear exactly why, but related to removed lines/buffers somehow:

• Floyd–Steinberg
• Atkinson
• Sierra (Filter Lite)
• Fan
• Shiau–Fan

Note: Did not bug out all changed matrices, again - unclear why.

2. Removing decimals off divisors changes dither result.

[Emix33]

I can answer some of the questions/issues that you ran into.

Regarding 1:
Removing lines from the matrices breaks them because all matrices used for non ordered dither are assumed to be fixed size (5x3). This is because we need to know which pixels relative to the original pixel we need to shift the error to.
So when you have a matrix like:
[0,0,X,0,0]
[0,0,0,0,0]
[0,0,0,0,0]
X always refers to the origin pixel. (That is also why the first 3 pixels in this type of matrix should be 0, as you cant diffuse an error to pixels you already have processed)
If you want to use matrices with different sizes/origin pixel you need to adjust the algorithm used for applying the error diffusion. (Ordered Dithering uses a different algorithm and is thus not affected)

Regarding 2:
Javascript has both integers (-1, 0, 1, 2 ...) and floats (0.5, 1.2, -3.7). When you have a division with at least one float: 7.5 / 2.5 you would get 3.0 and when you have 8 / 3.0 you would get 2.6666 (not exact, but close enough). When you try to divide an integer by another integer: 8 / 3 you would get another integer (always rounded down, even if up would be closer) so in this case 2. This is obviously extremely far off from the original value and will change the result massively expecially when the value is used in further calculations.

[roshavagarga]

@Emix33 Thank you for the information! That does explain a lot.

Now, excuse these follow-up questions, but I am curious:

1. Per your explanation, either the algorithm needs to be changed, or the previously-removed buffer rows/columns will need to be added back in - do you think the results would differ? As in, which method would provide better/more accurate dithering.

I will fiddle with the matrices on my own, but per the current setup, it seems that 5x2 matrices still work, despite removing the last row - I think I compared these and they were giving the same output, but will recheck.

2. If I'm understanding you correctly, the best option quality-wise would be to leave the divisors as floats, but have the matrices be done as integers, right? Again, I'm going off my bad memory, but I think comparisons showed that Burkes and Jarvis were giving the same results despite me changing the matrices to integers and removing their buffer rows.

3. Stevenson-Arce seemingly works for me, but if it is being read as a 5x3 (rather than the 7x4 it is), then some of it is not being handled/used properly, correct?

That would explain some obvious dithering patterns I saw, which I chalked up to the matrix itself, rather than to an error on my part.

4. Performance-wise, for the website/calculations, would it be better to do the buffer rows/columns, or implement a new algorithm - I've noticed slower calculations when running locally and wasn't sure why, though that may be because the matrices I was using weren't 5x3.

Again, thank you for the input, anything else would be greatly appreciated.

[roshavagarga]

With the above commits, all dithers now work, added a few more ordered ones via derived matrices from the Libcaca study.

Added:

• Cluster Dot 4x4
• Halftone 8x8
• Void-and-cluster 14x14 (unclear if it should be done on the fly or via set matrix, but it seemingly gives good results)

Fixed:

• Floyd–Steinberg
• Atkinson
• Sierra (Filter Lite)
• Fan
• Shiau–Fan

Other:

• Added back padding/buffer row and columns where needed - does not change dither result, but speeds up calculations.

Issues:

1. Unclear if the current way of handling is accurate or not. Specifically, there is a high chance that Stevenson–Arce is inaccurate, as its matrix size is above 5x3, thus possibly only part of it is handled? If it's not that simple to fix - can be dropped, although even if it is inaccurate currently, it gives interesting results in some cases.
2. If void-and-cluster method needs to be done on-the-fly/per-image, matrix is a stopgap and an external script can be included to do the calculations - there are free blue noise filters + scripts across Github and in the links above.

[roshavagarga]

Removed:

• Stevenson–Arce

The matrix I used was off, but even if it hadn't been wrong, it would have given bad results, as it was created for hexagonal pixels, rather than square ones, thus is would have given bad results.

This should clean up the last issues - PR is ready for review/merging.

[EFHIII]

I want to bring up two points:
1 - As far as I can tell, none of these algorithms are using superior color spaces for color comparison. To my knowledge, the HCT delta function is the current best one. If I'm not mistaken, the current code uses CIELab.
2 - Among the suggested algorithms, there aren't any multi-pass algorithms. Even just a fairly simple simulated annealing based dithering algorithm can dither better than an error diffusion algorithm.

Examples:
Using this image:

This is the output the current version of MapartCraft gives using a 7 color palette (TNT, Orange Wool, Yellow Wool, Emerald, Blue Wool, Black Wool, White Wool), no dithering, and no staircasing:

This is what it would look like using HCT:

This is the Floyd-Steinberg output the current version of MapartCraft gives:

This is what can be achieved using HCT with a simulated annealing based dithering algorithm:

I used a small palette to make the differences more pronounced, but here's with the everything palette (no staircasing):
For reference, the original again:

Current MappartCraft no dithering:

HCT:

Current MappartCraft Floyd-Steinberg:

HCT and simulated annealing:

[roshavagarga]

@EFHIII This fork (https://mike2b2t.github.io/mapartcraft/) has implemented the above change + more color spaces by what I remember, if you want to test around.

[mms0316]

@EFHIII, I understand that what you requested is something entirely new, so should be out of scope of this PR:

• I understand you're requesting a new dithering method, Simulated Annealing, perhaps what's described in baeldung.com/cs/simulated-annealing
• I understand you're requesting for a new color space, perhaps what's described as Google Material Design's HCT: https://github.com/material-foundation/material-color-utilities/blob/main/typescript/hct/hct.ts

It seems that rebane isn't actively working on this repo anymore, so I'd say that anyone that tries to tackle this suggestion to start a PR in https://github.com/mike2b2t/mapartcraft fork. In there, @roshavagarga's additional dithering algorithms of this PR were already applied, as already mentioned.

As for the new color spaces in mike2b2t's fork, the Better Color selection was changed from a checkbox (on/off) to a selection including CIE76 and CIEDE2000 color algorithms, but HCT is not in there. Just as a comment, after realizing that the Better Color algorithm (which seems to have been redstonehelper's modification of CIE76 D50 in their original code) wrongly skewed dark blue to purple, I fiddled with CIEDE2000 and it corrected that case way better.

Going back to HCT, which I wasn't aware of until now, @EFHIII, could you please confirm if the color difference is calculated using the Euclidian distance of L* (from L* a* b* color space), and a* and b* (from CAM16-UCS space)? This is what I understood looking up what HCT is about. If so, it's fairly straightforward to implement and I could add that.

And as for Simulated Annealing, which I also wasn't aware of until now, the algorithm looks to use a probability function, so it's non-deterministic - the same input image may render different results every time it is applied. This is just a comment that this could be bad depending on an user's case, unless someone fiddles with the algorithm and finds out a good deterministic approach

[EFHIII]

Yes, from what I understand, HCT's delta is the Euclidean distance of [a*, b*, L*] with a* and b* from CAM16 and L* from Lab*.

Simulated annealing is not inherently deterministic, but algorithms that take advantage of the same concepts can be deterministic or you could just use a set-seed for the RNG if all you care about is determinism.

There's a lot of nuance into how exactly to tune such an algorithm which felt out of scope for initially bringing things up, but even poorly tuned can be better than error diffusion.

I just looked at the fork and while it does have other color spaces, they still look much worse than HCT to me.

Here's CIEDE2000 D50:
no dithering (7 color palette):

Floyd dithering:

no dithering (full palette):

Floyd dithering:

[roshavagarga]

Although out of scope for this PR, I should also note that certain improvements can be made to this PR too. as I only had and still have the capability to apply matrices. I'd love to hear your opinion on the info below, @mms0316.

By what I remember, I simply applied a static 14x14 matrix for the void and cluster method, which may not be the best way to go about it - there's a fair bit of blue noise generation library attempts on github that may be included in mapartcraft so that a real blue noise matrix is created and used. I'm guessing this would yield better results, though I may be wrong.

Further interesting reads and examples can be found here. I'd personally single out Ostromoukhov's Variable Error Diffusion and the related Zhou-Fang dithering method (more information and implementation on these two here) as seemingly good dither methods.

As far as I remember, serpentining is also enabled by default - having the option to turn it on and off may be useful. I can't really remember if it was turned on for error diffusion only or for ordered too.

[mms0316]

@roshavagarga, well, from what I understood about Void-and-Cluster, the blue noise generation is not static - it depends on the input image (according to the voids / lack of pixels, and clusters / presence of pixels). In dark clusters, the noise generated would have a dense amount of black pixels.

And, because the algorithm talks about generating only a series of zeroes / ones, from my understanding that algorithm is only useful for greyscale images:

• If you use L* a* b* color space, on matrixes for a* (green-red) and b* (blue-yellow), the matrix relating to green-red would add green-red to the entire image (and same thing for blue-yellow). I would image applying the matrix for L* would be fine however.
• If you use RGB color space, same thing, all colors would be added to the entire image.

As a exemplifying scenario, from my understanding, an input image with black, grey or white background and a colored vase in the center would have an output image with colored noises throughout all the background.

Also, I understand that an entirely white image or an entirely black image as input would have noise as output.

Now, if there was a way that blue noise generated wouldn't be entirely discrete (I could see that each pixel in the blue noise only has zeros and ones, nothing in-between, and the pattern generated goes throughout the entire image, not allowing for holes where nothing is applied), I understand that such issue for colored images wouldn't occur.

As for Ostromoukhov, that looks interesting, but it's worth mentioning the coefficients were made for greyscale images (in L* a* b* color space, those coefficients would fit well the L* coordinate). As for a* (green-red) and b* (blue-yellow) coordinates, I have no idea if it would work with the same coefficients.
Note: The example images show that the error matrix have green and red, but I think that's just a problem with the toLinear() function, which is not using L* a* b*

As for Zhou-Fang, besides the point of greyscale images, I also have no idea how well that would go with a* and b*. Also, just a point that the algorithm is non-deterministic.

As for serpentine path, that could be a separate "Scan line" option, as it could be used in any error-diffusion algorithm. I'm not sure if it has any use for ordered dithering though.

[roshavagarga]

@mms0316 Interesting, coincidentally greyscale/monochrome art is what I use rebane for the most.

So if void-and-cluster is implemented correctly and generates per-image blue noise, it would be useful for more natural/accurate representations of greyscale images? Or, maybe it would present an improved output for monochrome images that only use 2x basic colors (i.e. white and black)? Guessing the same for Ostromoukhov/Zhou-Fang, though by what you said I'm guessing the latter would be hard to implement.

Serpentine has no use for ordered dithers, only for error diffusion - I'm going off of memory that somebody mentioned it was already baked into the mapartcraft code but only for error diffusion dithers. My idea is that, in certain cases - serpentine dithering removes obvious patterns, but in others it introduces them (especially monochrome images), so having the option to turn it on or off for error diffusion patterns would be useful.

[mms0316]

@roshavagarga, to me it's a stylistic choice. On one hand, void-and-cluster gives a film grain effect everywhere and you'd rarely find "worm artifacts" (dithering patterns like what was exemplified in Ostromoukhov's paper in figure 2a). On the other hand, Floyd-Steinberg doesn't spread any errors if the colors already match the palette, so those areas should comparatively be sharper (and, if you use Atkinson, which spreads less errors, it would sharpen even more). Ostromoukhov / Zhou-Fang would be somewhere in the middle in both worlds.

As for serpentine, it's not implemented in mapartcraft. Everything is in getMapartImageDataAndMaterials() in components\mapart\workers\mapCanvas.js, and the creation of canvasImageData.data is always done in positive directions.
Maybe you were talking about Ostromoukhov, which requires serpentine in the paper, and is implemented in https://observablehq.com/@jobleonard/variable-coefficient-dithering

[roshavagarga]

@mms0316 Thank you for the explanation! I was hoping these would be easier or good to implement, but they'll probably never be a thing, alongside another personal favorite, Riemersma dithering.

I guess I remembered wrong - I vaguely remember somebody mentioning it was implemented. Afaik, serpentine being an option to enable/disable would be a boon, because it can make dither patterns less visually distracting in some cases.

[EFHIII]

I have several advanced algorithms implemented here:
https://efhiii.github.io/dithering

Depending on your color palette needs, you may need to choose "custom palette" and input the palette as comma separated hex colors.