Proposal: have cptofs support a simple script file that instructs it on what to copy into/out of an image

[copumpkin]

#296 mentions that LKL can take some time to initialize which makes it impractical to call cptofs in a loop to put different things into it. #298 alleviates some of that pain, but doesn't handle cases like the following:

1. Create folder /path/a/b/c in the image
2. Copy the host's /another/path/foo, /another/path/bar, /another/path/baz into /path/a/b/c
3. Copy the host's /yet/another/path/meow, /yet/another/path/woof into /path/a

If this sounds like it might get too gnarly, it might instead make sense to adapt the lkl-hijack.sh script to support filesystem operation redirection and then run scripts involving cp under it.

[tavip]

Can you try disabling btrfs / raid6 / crypto stuff in .config and see if now calling cptofs multiple times is feasible?

Implement fs support in the hijack library (which is quite simple) would not solve the problem. We need to support multiple processes with the same instance of LKL, to avoid multiple initializations. This would require creating a ptrace based "hijacker", adding support for separate user/kernel address space, and probably more things that I don't see now.

In the previous incarnation of LKL (based on 2.6 kernel) we implemented lkl-lua which is a LUA based script engine. You can check it out here: https://github.com/lkl/lkl-lua

Between the two, probably the first options is better for the long term, but the second might be easier to implement.

[thehajime]

for the multiple process support with a single LKL instance, we may (also) have a system call proxy, as rump kernel does (my RFC patch #255 partially includes). ptrace may do too, but be known slow.

(off topic, sorry)

I think I wanted to have a road map (set of goals) for this project, since we've been almost more than 1> year but still no update on LKML. having many numbers of wish list is really great and healthy for those kind of community, but upstreaming code should be more simple at the first stage.

So my suggestion is to issue the v1 patch to the RFC emails with addressing issue raised during the discussion. I don't see any benefit to add ptrace-based hijack regarding with this point (simple patch).

for your reference: people had good (and some bad) impressions, and at least a couple of ppl are waiting for the upstreamed code.

• https://lwn.net/Articles/662953/
• https://lwn.net/Articles/685416/

[lurch]

It might be nice to re-use the initramfs format (or maybe just a subset, as appropriate) used by gen_init_cpio (see https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt for more info) as that also gives you much more fine-grained control than "just copying files" into the FS.

  dir /dev 755 0 0
  nod /dev/console 644 0 0 c 5 1
  nod /dev/loop0 644 0 0 b 7 0
  dir /bin 755 1000 1000
  slink /bin/sh busybox 777 0 0
  file /bin/busybox initramfs/busybox 755 0 0
  dir /proc 755 0 0
  dir /sys 755 0 0
  dir /mnt 755 0 0
  file /init initramfs/init.sh 755 0 0

[copumpkin]

That would be very cool! Not sure how hard though: is a parser and processor of that format already in the kernel in a reusable way?

…

On Jan 31, 2017, at 02:48, Andrew Scheller ***@***.***> wrote: It might be nice to re-use the initramfs format (or maybe just a subset, as appropriate) used by gen_init_cpio (see https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt for more info) as that also gives you much more fine-grained control than "just copying files" into the FS. dir /dev 755 0 0 nod /dev/console 644 0 0 c 5 1 nod /dev/loop0 644 0 0 b 7 0 dir /bin 755 1000 1000 slink /bin/sh busybox 777 0 0 file /bin/busybox initramfs/busybox 755 0 0 dir /proc 755 0 0 dir /sys 755 0 0 dir /mnt 755 0 0 file /init initramfs/init.sh 755 0 0 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[lurch]

It looks like all the details are in http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/usr/gen_init_cpio.c?id=HEAD - not sure how re-usable any of it is though?
I just thought it might be a useful file-format to start from, instead of re-inventing the wheel ;-) (although looking in detail, it seems this format doesn't (yet) support recursive directory copies, or different mtimes for different files)

[dezgeg]

I have been working on SWIG bindings for liblkl.so, e.g. here's a WIP proof-of-concept cptofs implemented in Python: https://github.com/dezgeg/linux/blob/lkl-swigpy/tools/lkl/python/cptofs.py. Still WIP and hacky but probably won't end up too bad.