Skip to content

Add support for handling cloud reparse points#44

Merged
Schamper merged 10 commits intofox-it:mainfrom
JSCU-CNI:ntfs-cloud-reparse-point
Sep 9, 2025
Merged

Add support for handling cloud reparse points#44
Schamper merged 10 commits intofox-it:mainfrom
JSCU-CNI:ntfs-cloud-reparse-point

Conversation

@JSCU-CNI
Copy link
Contributor

@JSCU-CNI JSCU-CNI commented Aug 6, 2025

Attempts to fix #43 by adding a MftRecord.is_cloud_file method. Does not differentiate between offline-available and online-backed files.

For example we have the following OneDrive files:

image

Note that synced text files will have their regular streams, while created-online.txt will be empty:

image

Copy link
Member

@Schamper Schamper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have test data available for unit tests?

@JSCU-CNI
Copy link
Contributor Author

JSCU-CNI commented Aug 7, 2025

Added a cloud MftRecord in f769b87. How would you propose we mock a valid Mft instance with this cloud record so we can properly test Mft._get_path()?

@JSCU-CNI JSCU-CNI requested a review from Schamper August 7, 2025 13:53
@Schamper
Copy link
Member

Added a cloud MftRecord in f769b87. How would you propose we mock a valid Mft instance with this cloud record so we can properly test Mft._get_path()?

Can't really think of a super clean way, maybe:

  • Create a small partition and configure OneDrive to use a folder on that partition, include the entire MFT as test file
  • Tediously mock the MFT with different mocked records and return values based on mocked paths or segment numbers

@JSCU-CNI
Copy link
Contributor Author

JSCU-CNI commented Sep 1, 2025

@Schamper could you enable LFS for the dissect.ntfs repository?

@JSCU-CNI
Copy link
Contributor Author

JSCU-CNI commented Sep 1, 2025

@Schamper Added an NTFS blob with OneDrive folder in ed2deb4 (compressed 1.6 MB). We can start using LFS and cleanup the tests/data folder in another PR.

@Schamper
Copy link
Member

Schamper commented Sep 1, 2025

LFS has been enabled.

@JSCU-CNI
Copy link
Contributor Author

JSCU-CNI commented Sep 1, 2025

Thanks, moved to LFS in 5f08eda.

Copy link
Member

@Schamper Schamper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional question, is there any structure we can parse, like we do with symlink and mount points?

if self.tag == IO_REPARSE_TAG.SYMLINK:
self.tag_header = c_ntfs._SYMBOLIC_LINK_REPARSE_BUFFER(data)
elif self.tag == IO_REPARSE_TAG.MOUNT_POINT:
self.tag_header = c_ntfs._MOUNT_POINT_REPARSE_BUFFER(data)

@JSCU-CNI
Copy link
Contributor Author

JSCU-CNI commented Sep 2, 2025

@Schamper Additional question, is there any structure we can parse, like we do with symlink and mount points?

Yes, it seems like there is a Cloud/OneDrive-related extension on the c_ntfs._REPARSE_DATA_BUFFER structure (just like c_ntfs._SYMBOLIC_LINK_REPARSE_BUFFER and c_ntfs._MOUNT_POINT_REPARSE_BUFFER). Further research in cldflt.sys and NTFS is required to properly decode this. At a quick glance it does not seem to contain significant data for the functioning of a cloud reparse point.

The only public reference I could find is https://github.com/nc-36/ntfs3g-onedrive-plugin/blob/master/src/onedrive.c#L77-L85, which seems to be incorrect for the values in our test data.

@JSCU-CNI JSCU-CNI requested a review from Schamper September 2, 2025 15:26
@Schamper
Copy link
Member

Schamper commented Sep 3, 2025

@JSCU-CNI can you create an issue to further reverse that?

@JSCU-CNI
Copy link
Contributor Author

JSCU-CNI commented Sep 3, 2025

See #45.

@JSCU-CNI JSCU-CNI requested a review from Schamper September 3, 2025 13:42
@JSCU-CNI JSCU-CNI requested a review from Schamper September 3, 2025 14:36
@JSCU-CNI JSCU-CNI requested a review from Schamper September 3, 2025 15:34
Schamper
Schamper previously approved these changes Sep 9, 2025
@codecov
Copy link

codecov bot commented Sep 9, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.41%. Comparing base (ea6a0ae) to head (0dd3d83).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #44      +/-   ##
==========================================
+ Coverage   86.72%   88.41%   +1.69%     
==========================================
  Files          11       11              
  Lines        1250     1226      -24     
==========================================
  Hits         1084     1084              
+ Misses        166      142      -24     
Flag Coverage Δ
unittests 88.41% <100.00%> (+1.69%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Schamper Schamper merged commit 35d785f into fox-it:main Sep 9, 2025
25 checks passed
@JSCU-CNI JSCU-CNI deleted the ntfs-cloud-reparse-point branch September 9, 2025 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for IO_REPARSE_TAG.CLOUD*

2 participants