Add support to load KTX2 compressed texture

[jessearmandse]

• KTX2 texture loading with libktx on iOS
• KTX2 texture loading with libktx on macOS
• Universal framework for libktx
• Load texture into SCNMaterial
• Test compressed textures from external 3D models

[jessearmandse]

@chetan-set

I added a sample model Cube which supposed to look like this when the texture is loaded:

But, it looks like this when it's loaded with the changes in this PR:

While office corridor scanned model, can be loaded with metal texture, model loading is done in Sample/iOS/GameViewController.swift:

let sceneSource = try GLTFSceneSource(named: "art.scnassets/OfficeCorridor/glTF/Office_Corridor_compressed.gltf")

[chetan-set]

I take it that the texture is KTX compressed? What happens when the texture is not KTX compressed?

[jessearmandse]

This is with the original GLTF model (PNG textures):

Then loaded the ktx2 compressed textures with glTF Viewer

Another thing is the validator of this viewer give warnings about image/ktx2 but it can load the textures:

From glTF-Validator

{
    "uri": "Cube.gltf",
    "mimeType": "model/gltf+json",
    "validatorVersion": "2.0.0-dev.3.8",
    "validatedAt": "2023-12-04T07:22:46.398Z",
    "issues": {
        "numErrors": 3,
        "numWarnings": 2,
        "numInfos": 4,
        "numHints": 0,
        "messages": [
            {
                "code": "UNSUPPORTED_EXTENSION",
                "message": "Cannot validate an extension as it is not supported by the validator: 'KHR_texture_basisu'.",
                "severity": 2,
                "pointer": "/extensionsUsed/0"
            },
            {
                "code": "VALUE_NOT_IN_LIST",
                "message": "Invalid value 'image/ktx2'. Valid values are ('image/jpeg', 'image/png').",
                "severity": 1,
                "pointer": "/images/0/mimeType"
            },
            {
                "code": "VALUE_NOT_IN_LIST",
                "message": "Invalid value 'image/ktx2'. Valid values are ('image/jpeg', 'image/png').",
                "severity": 1,
                "pointer": "/images/1/mimeType"
            },
            {
                "code": "UNUSED_MESH_TANGENT",
                "message": "Tangents are not used because the material has no normal texture.",
                "severity": 2,
                "pointer": "/meshes/0/primitives/0/attributes/TANGENT"
            },
            {
                "code": "UNUSED_OBJECT",
                "message": "This object may be unused.",
                "severity": 2,
                "pointer": "/images/0"
            },
            {
                "code": "UNUSED_OBJECT",
                "message": "This object may be unused.",
                "severity": 2,
                "pointer": "/images/1"
            },
            {
                "code": "IO_ERROR",
                "message": "Resource not found (Cube.bin).",
                "severity": 0,
                "pointer": "/buffers/0/uri"
            },
            {
                "code": "IO_ERROR",
                "message": "Resource not found (Cube_BaseColor.ktx2).",
                "severity": 0,
                "pointer": "/images/0/uri"
            },
            {
                "code": "IO_ERROR",
                "message": "Resource not found (Cube_MetallicRoughness.ktx2).",
                "severity": 0,
                "pointer": "/images/1/uri"
            }
        ],
        "truncated": false
    },
    "info": {
        "version": "2.0",
        "generator": "VKTS glTF 2.0 exporter",
        "extensionsUsed": [
            "KHR_texture_basisu"
        ],
        "extensionsRequired": [
            "KHR_texture_basisu"
        ],
        "resources": [
            {
                "pointer": "/buffers/0",
                "mimeType": "application/gltf-buffer",
                "storage": "external",
                "uri": "Cube.bin"
            },
            {
                "pointer": "/images/0",
                "storage": "external",
                "uri": "Cube_BaseColor.ktx2"
            },
            {
                "pointer": "/images/1",
                "storage": "external",
                "uri": "Cube_MetallicRoughness.ktx2"
            }
        ],
        "animationCount": 0,
        "materialCount": 1,
        "hasMorphTargets": false,
        "hasSkins": false,
        "hasTextures": true,
        "hasDefaultScene": true,
        "drawCallCount": 1,
        "totalVertexCount": 36,
        "totalTriangleCount": 12,
        "maxUVs": 1,
        "maxInfluences": 0,
        "maxAttributes": 4
    }
}

[chetan-set]

Ah the gltf itseld seems invalid with 3 errors:

Resource not found (Cube.bin)
Resource not found (Cube_BaseColor.ktx2)
Resource not found (Cube_MetallicRoughness.ktx2)

I strongly suspect that's the issue here

[jessearmandse]

Yep, the validator complains, but the three-gltf-viewer could load it.

I found what I missed here 7b4c432, it's because I didn't set the texture sampler, I noticed a hint when one of the cube face has the correct texture, but not all the other 5 faces.

[chetan-set]

So I take it the problem is now fixed?

[jessearmandse]

For this particular context of loading KTX2 texture, yes. For the invalid GLTF format is a bit out of scope as the exported GLTF that I tested here comes from Khronos compressor itself, and likely the online validator is out of date. While with 3D scanner we can change the code, not so much for other exporters.

[chetan-set]

Can you share the invalid GLTF file with me?

[jessearmandse]

Here's a few, but all of these are available from glTF-Sample-Models by compressing it with glTF-Compressor:

Cube.zip
BoxTextured.zip

[jessearmandse]

Those asset are actually valid with some warnings, it's just I need to drop all the files *.gtlf, *.bin and *.ktx2, json spec, binary, and textures:

{
    "uri": "BoxTextured.gltf",
    "mimeType": "model/gltf+json",
    "validatorVersion": "2.0.0-dev.3.8",
    "validatedAt": "2023-12-05T08:49:11.406Z",
    "issues": {
        "numErrors": 0,
        "numWarnings": 2,
        "numInfos": 2,
        "numHints": 0,
        "messages": [
            {
                "code": "UNSUPPORTED_EXTENSION",
                "message": "Cannot validate an extension as it is not supported by the validator: 'KHR_texture_basisu'.",
                "severity": 2,
                "pointer": "/extensionsUsed/0"
            },
            {
                "code": "VALUE_NOT_IN_LIST",
                "message": "Invalid value 'image/ktx2'. Valid values are ('image/jpeg', 'image/png').",
                "severity": 1,
                "pointer": "/images/0/mimeType"
            },
            {
                "code": "UNUSED_OBJECT",
                "message": "This object may be unused.",
                "severity": 2,
                "pointer": "/images/0"
            },
            {
                "code": "IMAGE_UNRECOGNIZED_FORMAT",
                "message": "Image format not recognized.",
                "severity": 1,
                "pointer": "/images/0"
            }
        ],
        "truncated": false
    },
    "info": {
        "version": "2.0",
        "generator": "COLLADA2GLTF",
        "extensionsUsed": [
            "KHR_texture_basisu"
        ],
        "extensionsRequired": [
            "KHR_texture_basisu"
        ],
        "resources": [
            {
                "pointer": "/buffers/0",
                "mimeType": "application/gltf-buffer",
                "storage": "external",
                "uri": "BoxTextured0.bin",
                "byteLength": 840
            },
            {
                "pointer": "/images/0",
                "storage": "external",
                "uri": "CesiumLogoFlat.ktx2"
            }
        ],
        "animationCount": 0,
        "materialCount": 1,
        "hasMorphTargets": false,
        "hasSkins": false,
        "hasTextures": true,
        "hasDefaultScene": true,
        "drawCallCount": 1,
        "totalVertexCount": 24,
        "totalTriangleCount": 12,
        "maxUVs": 1,
        "maxInfluences": 0,
        "maxAttributes": 3
    }
}