Extend child-support

[lhaagsma]

This PR adds a --list-children flag to target-query similar to fox-it/acquire#239.

Test output looks like this:

$ target-query --list-children <target>
Processing target: local (hostname=pve)
- [#0]: type=proxmox, path=/etc/pve/qemu-server/101.conf
- [#1]: type=proxmox, path=/etc/pve/qemu-server/102.conf
- [#2]: type=proxmox, path=/etc/pve/qemu-server/103.conf
- [#3]: type=proxmox, path=/etc/pve/qemu-server/100.conf
- [#4]: type=proxmox, path=/etc/pve/qemu-server/104.conf

Additionally, added index to the target-info plugin.

$ target-info <target>
<snip>
Children
- <Child #="0" type="proxmox" path="/etc/pve/qemu-server/101.conf">
- <Child #="1" type="proxmox" path="/etc/pve/qemu-server/102.conf">
- <Child #="2" type="proxmox" path="/etc/pve/qemu-server/103.conf">
- <Child #="3" type="proxmox" path="/etc/pve/qemu-server/100.conf">
- <Child #="4" type="proxmox" path="/etc/pve/qemu-server/104.conf">
<snip>

Aims to close: #1132 and #1134

[Schamper]

Can you add a unit test?

And @JSCU-CNI not sure if the # key name breaks anything?

[JSCU-CNI]

I think this could primarily cause trouble in processing dissect records further down the line.

For example, Elasticsearch should work with special characters as field names (it is discouraged: https://www.elastic.co/docs/reference/ecs/ecs-guidelines), however I am not certain if those fields can then be easily queried using Elastic DSL or ES|QL.

Guidelines for field names
Field names must be lower case
Combine words using underscore
No special characters except underscore

It seems like Splunk does not allow special characters in field names: https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureindex-timefieldextraction.

Field name syntax restrictions
You can assign field names as follows:

Valid characters for field names are a-z, A-Z, 0-9, . , :, and _.
Field names cannot begin with 0-9 or _ . Splunk reserves leading underscores for its internal variables.
Avoid assigning field names that match any of the [default field names](http://docs.splunk.com/Documentation/
Splunk/9.4.2/Data/Aboutdefaultfields).
Do not assign field names that contain international characters.

I think renaming # to something else will cause less trouble in the long term.

[lhaagsma]

The # or index (index is reserved in Splunk..) is mainly useful for cli purposes and seems pretty 'relative' to the target. Might not be too valuable to include in the records.
Instead i'm looking into retrieving the child.name from config files which seems more valuable.

[JSCU-CNI]

To expand on this topic, it would be nice if child information output (either using target-query --list-children or target-info [--children] supports nested children. Currently there is no way to rebuild the nested structure of multiple children (e.g. hypervisor -> VM -> Docker container).

[lhaagsma]

So i'm struggling a bit on how to do this in target-info.
I've added (and i'm about to commit an initial version) some code to target-info but the overall structure of target-info does not make it easy to pass arguments to get_target_info and/or print_target_info.
I have added code --list-children and --list-children-recursive to the other tools (query/shell) and will also be added to acquire.

[lhaagsma]

Latest commit adds "--list-children" and "--list-children-recursive" to target-query and target-shell.
A start was made for target-info but looking for guidance how to properly pass an optional parameter for recursive to the get_child_info function. @JSCU-CNI / @Schamper ?

target-shell was expanded to allow for --child to select a (sub)child from a system. example: target-shell --child 4.1 local or target-shell --child 1 /path/to/target/4

Example output for --list-children-recursive:

Processing hostname=pve, path=local (-recursive=True)
 [#0]: type=proxmox, name=wireguard, path=/etc/pve/qemu-server/101.conf
 [#1]: type=proxmox, name=proxy, path=/etc/pve/qemu-server/102.conf
 [#2]: type=proxmox, name=vm1, path=/etc/pve/qemu-server/103.conf
 [#3]: type=proxmox, name=vm2, path=/etc/pve/qemu-server/100.conf
 [#4]: type=proxmox, name=docker-host, path=/etc/pve/qemu-server/104.conf
 -[#4.0]: type=docker, name=angry_sammet, path=/var/lib/docker/image/overlay2/layerdb/mounts/xxx
 -[#4.1]: type=docker, name=changedetectionio-playwright-chrome-1, path=/var/lib/docker/image/overlay2/layerdb/mounts/xxx
 -[#4.2]: type=docker, name=docker1, path=/var/lib/docker/image/overlay2/layerdb/mounts/xxx
 -[#4.3]: type=docker, name=docker2-grafana-1, path=/var/lib/docker/image/overlay2/layerdb/mounts/xxxx
 -[#4.4]: type=docker, name=docker3-mosquitto-1, path=/var/lib/docker/image/overlay2/layerdb/mounts/xxxx
 -[#4.5]: type=docker, name=docker4-1, path=/var/lib/docker/image/overlay2/layerdb/mounts/xxx
 -[#4.6]: type=docker, name=docker5-database-1, path=/var/lib/docker/image/overlay2/layerdb/mounts/xxx
 [#5]: type=proxmox, name=VMWarez, path=/etc/pve/qemu-server/106.conf
 [#6]: type=proxmox, name=Win11hyperrrrrV, path=/etc/pve/qemu-server/105.conf
 -[#6.0]: type=hyper-v, name=VirtualOffspring, path=C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\<guid>.vmcx
 --[#6.0.0]: type=virtualbox, name=test_vm, path=/home/user/VirtualBox VMs/test_vm/test_vm.vbox
 -[#6.1]: type=wsl, name=kali-linux, path=C:\Users\user\AppData\Local\wsl\{guid}\ext4.vhdx
 -[#6.2]: type=wsl, name=Debian, path=C:\Users\user\AppData\Local\wsl\{guid}\ext4.vhdx
 -[#6.3]: type=wsl, name=name_name, path=C:\Users\user\AppData\Local\wsl\{guid}\ext4.vhdx

Additional the ChildTargetRecord was expanded to include the name field. This name field is now parsed for the following hypervisors:

• child/docker.py
• child/esxi.py - not tested yet
• child/hyperv.py
• child/parallels.py - not tested yet
• child/proxmox.py
• child/qemu.py - not tested yet
• child/virtualbox.py
• child/virtuozzo.py - not tested yet
• child/vmware_workstation.py - not tested yet
• child/wsl.py
• child/colima.py

I currently do not have a working setup for all hypervisors so not al code has been tested to work on real (live) installations/systems. But all tox tests are currently passing.

Todo:

• Add support for colima.py
• Commit acquire --list-children(-recursive). (dependency on dissect.target)
• Extend unit tests to check for child names.
• Improve the error handling of the various _get_child_name functions within the child plugins. Any advice @Schamper ?

[lhaagsma]

@Zawadidone any chance you could have a look/share how to retrieve the container name for Colima?
Additionally, if you happen to have a test environment for virtuozzo and could validate that my changes work that would be great :)

[Zawadidone]

I don't have a test environment for Virtuozzo, but I think using the <container-uuid> fixes it.

Thanks for the fix for Colima.

[lhaagsma]

Requesting review.

[codspeed-hq]

CodSpeed Performance Report

Merging #1133 will not alter performance

_{Comparing lhaagsma:list-chldren (6bce98d) with main (2d9d144)}

Summary

✅ 8 untouched

[codecov]

Codecov Report

❌ Patch coverage is 70.28112% with 74 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.63%. Comparing base (2d9d144) to head (6bce98d).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
dissect/target/tools/utils.py	34.09%	29 Missing ⚠️
dissect/target/target.py	56.00%	11 Missing ⚠️
dissect/target/plugins/child/esxi.py	42.85%	4 Missing ⚠️
dissect/target/plugins/child/hyperv.py	81.81%	4 Missing ⚠️
dissect/target/plugins/child/parallels.py	50.00%	4 Missing ⚠️
dissect/target/plugins/child/proxmox.py	69.23%	4 Missing ⚠️
dissect/target/plugins/child/qemu.py	50.00%	4 Missing ⚠️
dissect/target/plugins/child/virtuozzo.py	73.33%	4 Missing ⚠️
dissect/target/plugins/child/wsl.py	71.42%	2 Missing ⚠️
dissect/target/tools/dd.py	0.00%	2 Missing ⚠️
... and 5 more

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #1133    +/-   ##
========================================
  Coverage   80.63%   80.63%            
========================================
  Files         374      375     +1     
  Lines       33060    33221   +161     
========================================
+ Hits        26658    26788   +130     
- Misses       6402     6433    +31     

Flag	Coverage Δ
unittests	80.63% <70.28%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lhaagsma]

So i'm struggling a bit on how to do this in target-info. I've added (and i'm about to commit an initial version) some code to target-info but the overall structure of target-info does not make it easy to pass arguments to get_target_info and/or print_target_info. I have added code --list-children and --list-children-recursive to the other tools (query/shell) and will also be added to acquire.

@JSCU-CNI Any input on this would still be valuable.

[JSCU-CNI]

Thanks for the ping @lhaagsma. What input would you like from us specifically? Personally - from an end-user point of view - I would expect --children to act recursively, if any recursive children exist. Feel free to refactor target-info as you see fit for this functionality.

[Schamper]

@JSCU-CNI I changed this from names to name, seemed to be an oversight in naming since it's seemingly always a singular name, but let me know if I'm overlooking anything.

[JSCU-CNI]

This is intentionally called names since that is how Docker, Podman (and iirc other) container platforms tag their container name (see for example output of docker ps -a or podman ps -a).

https://docs.docker.com/reference/cli/docker/container/ls/#format
https://docs.podman.io/en/v5.0.2/markdown/podman-ps.1.html#format-format

From the dissect perspective I guess it makes sense to make the field singular if it is not a string[] field.

[Schamper]

I didn't know that, but I think that's silly 😄. In the linked example of Docker there's a redis,webapp/db container names though, do you know how that occurs, and if we support that already?