Skip to content

sc 4.5 new chunks#228

Open
Frozandero wants to merge 56 commits intoMarkemp:release/v2.0from
Frozandero:feature/sc-4.5-chunk-type-names
Open

sc 4.5 new chunks#228
Frozandero wants to merge 56 commits intoMarkemp:release/v2.0from
Frozandero:feature/sc-4.5-chunk-type-names

Conversation

@Frozandero
Copy link

@Frozandero Frozandero commented Jan 5, 2026

This is a PR for making cgf-converter work for SC 4.5. It is a starting point after I experimented a lot during the weekend. It does export the meshes. However I don't think it is release ready for now. I am just creating this PR as a reference point.

FYI I have no idea about animations, if they work or not. My focus was getting the meshes converted.
Additionally wolf and f7c mkii, did export too but couldn't load their materials using the SC blender addon. Very much possible that these changes are not enough to make everything work.

Also seems to not fix the USD exporter yet, I will also work on that but I am not sure if SCBP importer would support it so I can only test it with singular assets.

Markemp and others added 30 commits November 30, 2025 08:56
Implements support for individual .caf animation files (vs .dba databases):
- ChunkController_829: Complete implementation for compressed format (was stub)
- ChunkController_830: Uncompressed CryKeyPQLog format (28 bytes/keyframe)
- ChunkController_831: Compressed format with separate rotation/position tracks
- CafAnimation model class for aggregated animation data
- Wildcard pattern expansion for chrparams (*.caf, cinematics/*.caf)
- USD SkelAnimation generation from CAF bone tracks

CAF files use logarithmic quaternion encoding for rotations, which is
converted to standard quaternions during parsing. Controller IDs are
CRC32 hashes of bone names for skeleton matching.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
ChunkController_829/831 fixes:
- Correct data layout: rotation values -> times -> position values -> times
- Fix time format enum: 0=float, 1=uint16, 2=byte (was inverted)
- Add alignment between data sections when TracksAligned is set
- Handle PositionKeysInfo: if 0, position shares rotation's time keys
- Remove non-existent Flags field from 829 (14-byte header, not 18)

SmallTree quaternion decompression fix:
- Clamp sqrt input to prevent NaN when sqrsumm > 1.0 due to precision issues
- Applied to all 4 SmallTree quaternion types

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Implements parsing for Star Citizen's #ivo animation format:
- New chunk types: IvoCAFData (0xA9496CB5), IvoAnimInfo (0x4733C6ED),
  IvoDBAData (0x194FBC50), IvoDBAMetadata (0xF7351608)
- ChunkIvoCAF_900 parses #caf blocks with bone controllers and
  compressed keyframe data (SmallTree48/64BitQuat formats)
- Fixed wildcard path expansion in chrparams to handle patterns
  like "*/*.caf" for recursive subdirectory searches
- Added chunk factory registrations for new #ivo animation types
- Integration tests for aloprat creature model with CAF animations

The aloprat skeleton now exports with 9 separate animation USDA files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Raw uint16 time keys were being used directly as frame numbers, causing
animations to have 60000+ frames. Now normalizes time keys to frame
indices (0 to keyframe count-1).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…erer

Fixes IndexOutOfRangeException when converting Star Citizen .skin files
(e.g., aloprat.skin) where MatID could exceed SubMaterials array length.

Also adds bounds checking for normals array access to prevent crashes
when indices reference out-of-bounds normal data.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Bones influence wrong vertices (e.g., R_UpperArm affects head vertices).
Added investigation notes for future debugging session.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The bone indices in CryEngine skinning data (IntSkinVertex.BoneMapping.BoneIndex)
are indices into the CompiledBones array, but USD's skel:jointIndices expects
indices into the joints array built from jointPaths. Since BuildJointPaths()
walks the bone hierarchy depth-first, it produces a different ordering than
the linear CompiledBones array.

Added _compiledBoneIndexToJointIndex mapping in CreateSkeleton() and use it
in AddSkinningAttributes() to remap bone indices before outputting to USD.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The MaterialIndices array has NumberOfMeshSubsets entries (one per mesh subset),
not NumberOfNodes entries. Using materialTable[index] where index is a node index
caused ArgumentOutOfRangeException when NumberOfNodes > NumberOfMeshSubsets.

Fixed by getting the material ID directly from the mesh subset (subsets[0].MatID)
when the node has geometry, which is the correct source for this data.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
When a material file has submaterials with the same name (after cleaning),
USD import would fail with "Duplicate prim" error. Added deduplication
by tracking created material names and skipping duplicates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
MTL files can use Map="Normalmap" for normal maps, but the parser only
handled "Bumpmap" and "Normal". Added "Normalmap" as an alias.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Some materials may have null SubMaterials array. Skip these gracefully
instead of throwing NullReferenceException.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Multiple mesh subsets can use the same material, creating duplicate
GeomSubset prims with the same name. Now collect and merge face indices
per material, creating one GeomSubset per unique material.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Reorganize test structure so manual tests can be excluded when running
unit and integration tests from Test Explorer. The ManualTests namespace
allows selective test execution without needing test.runsettings.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Star Citizen 4.1 uses version 901 of the IvoDBAMetadata chunk.
Format is identical to version 900.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Star Citizen 4.1 uses version 901 of the IvoDBAMetadata chunk.
Key difference from v900: string table starts 4 bytes earlier,
overlapping with the last entry's Padding field.

Added debug logging to manual tests to help diagnose chunk parsing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Updated all axis transformation functions to use consistent mapping:
CryEngine (X, Y, Z) → glTF (X, Z, -Y)

This ensures glTF models face +Y in Blender after import, matching
USD exports. The fix includes:
- SwapAxesForPosition: (x, y, z) → (x, z, -y)
- SwapAxesForTangent/Quaternions: consistent axis transformation
- SwapAxes(Matrix4x4): correct T × M × T⁻¹ conjugation
- Use matrix-based node transforms instead of TRS decomposition
- Add per-subset vertex extraction for Ivo format skinning

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Additive animations store bone transforms as deltas from the rest pose,
not absolute positions. When exported directly, Blender collapses the
skeleton because all translations are near-zero.

Changes:
- Add IsAdditive property to CafAnimation class
- Read CA_ASSET_ADDITIVE flag (0x001) from ChunkGlobalAnimationHeaderCAF
- Check AssetFlags.Additive in DBA MotionParams
- Convert additive deltas to absolute transforms during USD export:
  - absolute_rotation = restRotation * additiveRotation
  - absolute_translation = restTranslation + additiveTranslation
- Add BuildRestRotationMapping() for additive conversion

Fixes pilot joystick/throttle animations that were previously broken.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Version 0x801 stores only B2W (boneToWorld) matrix, not W2B like assumed.
The fix computes W2B by inverting B2W, matching the Lumberyard loader behavior.
This corrects armature alignment issues for ArcheAge .chr files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
CryEngine matrices store translation in column 4 (M14, M24, M34), not row 4.
BuildRestTranslationMapping was using .Translation which reads M41/M42/M43,
returning zeros for all bones. This broke CAF animations and additive DBA
animations that rely on rest pose translations.

Additionally, CAF position data is absolute local transforms, not deltas.
The code was incorrectly adding rest translation to animation positions,
causing characters to float/levitate during animation.

Changes:
- Fix translation extraction to use M14/M24/M34 in BuildRestTranslationMapping
- Fix CAF position handling: use animation data directly, not rest + delta
- Add separate RotationKeyTimes/PositionKeyTimes to BoneTrack for 829 controllers
- Add validation warnings to unvetted chunks (900, 901, 829)
- Fix ChunkController_829 header padding alignment
- Document animation data flow in DEVNOTES.md for future chunk implementations

Tested with MWO pilot (800+905) and Armored Warfare chicken (901+829).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Armored Warfare chicken walk/idle animations confirmed working in Blender.
Updated DEVNOTES.md vetted chunks table and removed warning from chunk class.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Chicken_Usd test for skeleton/armature validation
  - Verifies 15 joints with correct hierarchy
  - Checks bindTransforms, restTransforms, and mesh skinning
- Add Chicken_WalkAnimation_Usd test for CAF animation export
  - Tests ChunkController_829 animation pipeline
  - Regression test for pelvis translation (prevents levitation bug)
  - Validates timeSamples, frame range, and joint count

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ArcheAge to working animations list (uses ChunkCompiledBones_801 + .cal)
- Add ChunkCompiledBones_801 to vetted chunks table
- Update status to include MWO, Armored Warfare, and ArcheAge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ChunkController_905 now correctly handles two DBA storage modes:
- Standard mode: positive offsets relative to start of track data
- In-place streaming mode: negative offsets relative to END of track data

Key fixes:
- Detect in-place streaming via negative offset in keyTimeOffsets[0]
- Handle different data layout: padding -> animations -> controllers -> track data
- Add 4-byte alignment before track data in in-place mode
- Use rest translation for bones without position animation (fixes collapsed skeleton)

Also adds DBA wildcard pattern resolution (*.dba) in chrparams.

Tested with KCD2 pig skeleton (51 animations) and MWO pilot.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Animation Support section to CLAUDE.md covering DBA, CAF, and CAL formats
- Document critical implementation details (matrix convention, rest translation, additive)
- Add Star Citizen #ivo animations as next development target in DEVNOTES.md
- Minor code style cleanup in CalFile.cs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add CAF/CAL animation export to glTF (WriteCafAnimations method)
- Fix split DDS texture paths (.dds.1 → .dds.dds bug)
- Add -ut unsplit texture support for glTF
- Fix NoDraw materials not being added to materials array

CAF animations from .caf files and .cal animation lists are now
exported to glTF alongside existing DBA animation support.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Markemp and others added 14 commits December 19, 2025 19:39
For Ivo format files with ObjectNodeIndex mappings, create a unified
hierarchy where meshes are attached directly to skeleton bone nodes
instead of creating separate parallel geometry/skeleton hierarchies.

This ensures geometry moves with the skeleton when animated.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Initialize ObjectNodeIndex to -1 (not set) instead of 0
  This prevents traditional .chr files from being incorrectly
  treated as Ivo format, which caused random geometry to be
  attached to bone nodes

- Use single shared skin named after model (not "barrel/skin")

- Unified skeleton/mesh hierarchy for Ivo format files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix DBA path resolution: resolve relative paths against ObjectDir
- Add Ivo DBA animation extraction (ChunkIvoDBAData/ChunkIvoDBAMetadata)
- Support Ivo DBA in both CreateAnimations() and ExportAnimationFiles()
- Add SampleIvoPosition/SampleIvoRotation for keyframe interpolation

Ivo format DBA files use different chunk types than standard DBA files.
The chrparams binary XML loading was working, but the DBA path wasn't
being resolved and the renderer didn't know how to extract animations
from Ivo DBA chunks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add CreateIvoSkeletonNodes() method that builds the skeleton bone Xform
hierarchy first, then attaches meshes directly to their corresponding
bones with identity transforms.

Previously, meshes were nested in a node hierarchy (barrel inside body),
causing double transforms: once from scene graph parenting and once from
skeletal skinning. The barrel appeared at doubled position in Blender's
pose/object mode.

Now for Ivo format with skinning:
- Bone Xforms form the hierarchy (matching skeleton joints)
- Meshes are direct children of their corresponding bone Xforms
- Mesh transforms are identity (all positioning via skeletal skinning)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Place skinned meshes at SkelRoot level (siblings of Skeleton) instead of
under bone Xforms. This avoids double-transforms where meshes would
inherit bone Xform transforms via scene graph parenting in addition to
skeletal skinning transforms.

Meshes now have identity transforms with all positioning coming purely
from skeletal skinning (skel:jointIndices/Weights).

Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix DBA block parsing: next block starts after controller headers,
  not at blockEnd (was only finding 1 of N animation blocks)
- Ivo DBA positions use rest + delta (confirmed working with lasercannon)
- Ivo DBA rotations use rest * delta (WIP, needs rotation-only test case)
- Add metadata StartPosition/StartRotation access for future use
- Add diagnostic logging for animation values
- Update glTF animation handling to match USD approach

Rotation handling still needs validation with a rotation-only animation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Analyzed and documented the 6 new chunk types introduced in Star Citizen 4.5:
- IvoAssetMetadata (0xBE5E493E): Asset GUIDs/metadata (128 bytes)
- IvoLodDistances (0x9351756F): LOD switching thresholds
- IvoLodMeshData (0x58DE1772): LOD1-4 mesh data (can be 50MB+)
- IvoBoundingData (0x2B7ECF9F): Bounding/animation data
- IvoChunkTerminator (0xE0181074): EOF marker in .cgam files
- IvoMtlNameVariant (0x83353533): Material name variant (rare)

All chunks safely skipped as they contain metadata or LOD data
not needed for LOD0 rendering.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Frozandero
Copy link
Author

image

meshes do export now, however I am still seeing issues with some materials (clipper interior is plain white). Wolf doesn't load its textures at all.

These might be due to the fact that I am not running "export missing" on the sc blender addon but as I mentioned in my reply, this change seems to slow down the logic somewhere.

I tried pressing export missing, on clipper here, it said 241 files need exporting but it was very slow and blender stopped responding by the 7th item and I gave up. I can look into it more during the next weekend possibly.

But I published so that people can check for themselves and improve on it or just use it as inspiration and throw it away completely.

@Frozandero
Copy link
Author

Some more images:
image
image

@Markemp
Copy link
Owner

Markemp commented Jan 6, 2026

Can you change the target for this PR to release/v2.0 instead of the animation branch? That way I can merge it down to mine and keep the changes separate. And thanks a ton for doing this!

@Frozandero Frozandero changed the base branch from feature/caf-animation-support to release/v2.0 January 6, 2026 01:29
@Frozandero
Copy link
Author

I did change it but it still builds on the CAF because I don't think my changes are standalone compared to it I did make some changes to existing chunks in that branch to skip padding bytes in some cases that were causing read errors on the files I was trying to convert. I can look into it more, Saturday.

And again it seems incomplete to me, however I lack the older SC files to test the "changed behaviour" compared to older version to know what is missing. Maybe some stuff were already broken before the 4.5 changes so idk.

@Markemp
Copy link
Owner

Markemp commented Jan 6, 2026

Ok, makes sense. I'll go over it and get some feedback in a bit. 👍🏽

@Frozandero Frozandero marked this pull request as ready for review January 11, 2026 15:23
@Markemp
Copy link
Owner

Markemp commented Jan 11, 2026

I'm taking a look at this now, but I think I gave you bad advice asking to change the target to release/2.0. You have a lot of the files that I'm working on in feature/caf-animation-support, so it's making the PR a bit difficult to parse. Would it be possible to make the branch straight off of release/2.0 and just include the SC ones you were working on? That way I can merge it down into my animation branch and that should be able to handle all the animations.

This is my fault for letting the caf animation branch fester for so long. I need to finish that up and get it into the release branch, but been struggling with the SC animations. I'll probably make a separate feature branch for that, so that I don't get too far behind on the current work. Sorry for the change in direction.

@Markemp
Copy link
Owner

Markemp commented Jan 11, 2026

Oh wait. I can just merge mine into the release branch and that should make this PR more readable. Let me do that.

@Markemp
Copy link
Owner

Markemp commented Jan 11, 2026

I've merged my animation work into release/v2.0. Can you rebase or merge the latest release/v2.0 into your branch? That should reduce the diff to just your SC 4.5 format changes and make review much easier.

@Frozandero
Copy link
Author

Frozandero commented Jan 11, 2026

Rebase done

This reverts commit 1fe0a75.
@Frozandero
Copy link
Author

Frozandero commented Jan 11, 2026

Sidenote, I was testing the USD export today to see and found there is some issue related to bones still, probably a trivial fix. Though starfab workflow uses .dae so I only thought of testing it today.

~\test\Data\Objects\Spaceships\Ships\DRAK\clipper\exterior\drak_clipper.cga
  AggregateException: One or more errors occurred. (One or more errors occurred. (An item with the same key has already been added. Key: ____/_/_))
    AggregateException: One or more errors occurred. (An item with the same key has already been added. Key: ____/_/_)
      ArgumentException: An item with the same key has already been added. Key: ____/_/_
         at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
         at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
         at CgfConverter.Renderers.USD.UsdRenderer.GetBindTransforms(List`1 jointPaths, Dictionary`2 bonePathMap)
         at CgfConverter.Renderers.USD.UsdRenderer.CreateSkeleton(Dictionary`2& controllerIdToJointPath, List`1& jointPaths, Dictionary`2& bonePathMap, Int32[]& compiledBoneIndexToJointIndex)
         at CgfConverter.Renderers.USD.UsdRenderer.GenerateUsdObject()
         at CgfConverter.Renderers.USD.UsdRenderer.Render()
         at CgfConverter.Program.RunRenderersAndThrowAggregateExceptionIfAny(IEnumerable`1 renderers)
[INF:Program] Finished. Rendered 0 file(s)
[ERR:Program] Failed to convert 1 file(s).

I am planning to check the material id issues that cause some parts to have no materials but that will probably be its own thing (Likely a thing that can be fixed by the new IvoMtlNameVariant chunk but I couldn't find the logic there, which is why I reverted it for now, to not clutter the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants