Filesystem API

[S-S-X]

Implement both fake and real file systems.

Fake file system API being default while real file system API can be enabled with configuration.

• Basic engine functions.
  ☑️ Works for me (some stuff still missing).
• Instead of configuration, maybe introduce new command line flag. This could reduce fatal mistakes.
  ☑️ Added --real-filesystem command line option, defaults to fake file system.
• For real fs, handle all paths relative to something.
  Maybe some ./spec/filesystem/ as a root?
  ❓ this might not be that good idea, it would add some protection about mistakes in mod code but could also be confusing.
  • Mod paths returned by core.get_modpath(modname) for fake fs.
  • World path returned by (core.get_worldpath() for fake fs.
  • Mod data path returned by (core.get_mod_data_path() for fake fs (since 5.9.0).
  • Restrict fake fs access to other paths.
• For fake fs, during initialization:
  Load directories and files under ./spec/filesystem/ to allow writing static content for testing.
  Read only: writing should replace in memory contents instead of actual files or directories.
  ☑️ Added mineunit:fs_copy(src, dst) to copy files from real fs into fake fs.
  ☑️ Added path tracking with automated initialization from real file fs.
  Comes with few known corner case bugs:
  • Recursive directory initialization can silently wipe already tracked files from fake fs.
  • At root level in some cases it is not checked if target is file or directory.
• Sane core.get_worldpath() and core.get_modpath()
  ❓ worldpath is currently first configured fixture lookup path (so defaults to spec/fixtures).
  ☑️ modpaths are configurable already.
• Related io & os stuff.
  • io.open and its friends + file objects.
  • os.rename & os.remove.
  • Maybe also no-op all related io & os unless explicitly enabled by loading fs module?
    Maybe do this when easier module loading has been added?
    ❌No. Wont do. Restored fs to keep loading it with core module.
• A lot of manual testing (in addition to basic automated test sets).
  ❓ some testing done, not sure what would be enough.
  Would need some mods that do some heavy complicated io & os stuff.
  Advtrains passes all the tests already with simple railway ndb (including some routes/signals/etc. simple stuff).

Adds few Mineunit utilities (fs_* are only available when fake file system module is loaded):

• dump(thing) - luassert version of dump, used in case engine module isn't loaded.
• mineunit:builtin(name) - get builtin Lua module that was overridden by Mineunit.
• mineunit:fs_copy(src, dst) - copy files from real file system (fixtures) into fake file system, recursive when src is directory.
• mineunit:fs_getfile(path) - retrieve file from fake file system.
• mineunit:fs_raw() - retrieve raw fake file system storage object, for internal debugging. Should probably be removed.

[S-S-X]

Tests depends on new mineunit runner functionality ee6c57d which cannot be used in tests until merged to master branch.

• Done at mineunit:subscribe(what, callback) #110

[S-S-X]

Tests are getting toward very high coverage for fake fs though still some things without any tests.
Most of it should work.

Will break certain test cases where mod is reading / writing into its own source files, should probably add some cow to solve this.
Though requiring mineunit:fs_copy(".", ".") would also work just fine, just a bit wasteful especially with large mods.

[S-S-X]

To do: rebase to v0.15 (currently @engine-version-compat-0.15) and release either with or after v0.15

Old base b82f55f to master c414270 to engine-version-compat-0.15 c4760ef

[github-actions]

Test coverage 53.52% in 33/39 files (higher is better).

Dynamic debug coverage 28.61% in 25/39 files (lower is better).

Click for detailed source code test coverage report

File                                      Hits Missed Coverage
--------------------------------------------------------------
game/constants.lua      12   0      100.00%
voxelmanip.lua          214  2      99.07%
settings.lua            375  17     95.66%
game/privileges.lua     64   3      95.52%
craftmanager.lua        119  13     90.15%
core.lua                84   13     86.60%
assert.lua              191  32     85.65%
fs.lua                  266  51     83.91%
formspec.lua            166  34     83.00%
auth.lua                39   10     79.59%
craft.lua               93   24     79.49%
itemstack.lua           121  34     78.06%
entity.lua              64   22     74.42%
player.lua              332  130    71.86%
metadata.lua            177  80     68.87%
config.lua              63   35     64.29%
game/features.lua       21   12     63.64%
globals.lua             94   55     63.09%
init.lua                131  97     57.46%
game/register.lua       209  158    56.95%
common/serialize.lua    55   44     55.56%
world.lua               83   85     49.40%
common/vector.lua       70   73     48.95%
server.lua              72   102    41.38%
game/auth.lua           38   68     35.85%
common/misc_helpers.lua 148  269    35.49%
game/misc.lua           42   87     32.56%
nodetimer.lua           12   25     32.43%
game/item.lua           142  331    30.02%
lib/json.lua            51   146    25.89%
common/chatcommands.lua 17   49     25.76%
game/chat.lua           150  593    20.19%
deprecation.lua         3    12     20.00%
scwd.lua                0    1      0.00%
protection.lua          0    12     0.00%
print.lua               0    70     0.00%
http.lua                0    60     0.00%
default/functions.lua   0    354    0.00%
common/after.lua        0    26     0.00%

Mineunit stdout:

++ Executing suite spec/assert_spec.lua
++ Executing suite spec/auth_spec.lua
++ Executing suite spec/core_spec.lua🢆 Running tests from core_spec.lua:38 ▷ Mineunit core deprecation prints error
E:	test deprecation message
🢆 Running tests from core_spec.lua:38 ▷ Mineunit core deprecation prints warning
W:	test deprecation message

++ Executing suite spec/craft_spec.lua
++ Executing suite spec/debug_spec.lua
W:	Tests skipped: Debug hooks not available.

++ Executing suite spec/formspec_spec.lua🢆 Running tests from formspec_spec.lua:109 ▷ Mineunit formspec Mineunit:Form provides fields table
E:	Form:fields() is experimental and UNSTABLE. Its behavior and interface WILL BE CHANGED.

++ Executing suite spec/fs_spec.lua🢆 Running tests from fs_spec.lua:366 ▷ Mineunit fake io io.open failures io.open(r) without file
W: (fake io) could not open file 'io.open r without file'.
🢆 Running tests from fs_spec.lua:366 ▷ Mineunit fake io io.open failures io.open(rb) without file
W: (fake io) could not open file 'io.open rb without file'.
◌
++ Executing suite spec/itemstack_spec.lua◌
++ Executing suite spec/metadata_spec.lua
++ Executing suite spec/player_bare_spec.lua
++ Executing suite spec/player_spec.lua
++ Executing suite spec/voxelmanip_spec.lua
296 successes / 0 failures / 0 errors / 2 pending : 4.389312 seconds

Pending → spec/fs_spec.lua @ 695
Mineunit fake io interface has no internal properties
spec/fs_spec.lua:695: Internal properties of File not hidden

Pending → spec/itemstack_spec.lua @ 20
ItemStack constructor fails ItemStack()
spec/itemstack_spec.lua:20: Waiting for strict mode implementation

W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.

[github-actions]

Test coverage 53.59% in 35/41 files (higher is better).

Dynamic debug coverage 28.76% in 27/41 files (lower is better).

Click for detailed source code test coverage report

File                                      Hits Missed Coverage
--------------------------------------------------------------
game/constants.lua      12   0      100.00%
voxelmanip.lua          214  2      99.07%
settings.lua            375  17     95.66%
game/privileges.lua     64   3      95.52%
craftmanager.lua        119  13     90.15%
lib/file.lua            109  12     90.08%
core.lua                84   13     86.60%
assert.lua              191  32     85.65%
formspec.lua            166  34     83.00%
auth.lua                39   10     79.59%
craft.lua               93   24     79.49%
itemstack.lua           121  34     78.06%
fs.lua                  136  39     77.71%
entity.lua              64   22     74.42%
player.lua              332  130    71.86%
lib/fs.lua              60   25     70.59%
metadata.lua            177  80     68.87%
config.lua              63   35     64.29%
game/features.lua       21   12     63.64%
globals.lua             94   55     63.09%
init.lua                131  97     57.46%
game/register.lua       209  158    56.95%
common/serialize.lua    55   44     55.56%
world.lua               83   85     49.40%
common/vector.lua       70   73     48.95%
server.lua              72   102    41.38%
game/auth.lua           38   68     35.85%
common/misc_helpers.lua 148  269    35.49%
game/misc.lua           42   87     32.56%
nodetimer.lua           12   25     32.43%
game/item.lua           142  331    30.02%
lib/json.lua            51   146    25.89%
common/chatcommands.lua 17   49     25.76%
game/chat.lua           150  593    20.19%
deprecation.lua         3    12     20.00%
scwd.lua                0    1      0.00%
protection.lua          0    12     0.00%
print.lua               0    70     0.00%
http.lua                0    60     0.00%
default/functions.lua   0    354    0.00%
common/after.lua        0    26     0.00%

Mineunit stdout:

++ Executing suite spec/assert_spec.lua
++ Executing suite spec/auth_spec.lua
++ Executing suite spec/core_spec.lua🢆 Running tests from core_spec.lua:38 ▷ Mineunit core deprecation prints error
E:	test deprecation message
🢆 Running tests from core_spec.lua:38 ▷ Mineunit core deprecation prints warning
W:	test deprecation message

++ Executing suite spec/craft_spec.lua
++ Executing suite spec/debug_spec.lua
W:	Tests skipped: Debug hooks not available.

++ Executing suite spec/formspec_spec.lua🢆 Running tests from formspec_spec.lua:109 ▷ Mineunit formspec Mineunit:Form provides fields table
E:	Form:fields() is experimental and UNSTABLE. Its behavior and interface WILL BE CHANGED.

++ Executing suite spec/fs_spec.lua🢆 Running tests from fs_spec.lua:366 ▷ Mineunit fake io io.open failures io.open(r) without file
W: (fake io) could not open file 'io.open r without file'.
🢆 Running tests from fs_spec.lua:366 ▷ Mineunit fake io io.open failures io.open(rb) without file
W: (fake io) could not open file 'io.open rb without file'.
◌
++ Executing suite spec/itemstack_spec.lua◌
++ Executing suite spec/metadata_spec.lua
++ Executing suite spec/player_bare_spec.lua
++ Executing suite spec/player_spec.lua
++ Executing suite spec/voxelmanip_spec.lua
296 successes / 0 failures / 0 errors / 2 pending : 4.487096 seconds

Pending → spec/fs_spec.lua @ 695
Mineunit fake io interface has no internal properties
spec/fs_spec.lua:695: Internal properties of File not hidden

Pending → spec/itemstack_spec.lua @ 20
ItemStack constructor fails ItemStack()
spec/itemstack_spec.lua:20: Waiting for strict mode implementation

W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.
W: (real io) could not open file 'luacov.stats.out'.