Runtime Loader Shim

what

why

how

normal execution

flowchart TB
  exec["`<code>exec <span style="color: yellow;">path/to/binary</span></code>`"]

  bin_load@{ shape: procs, label: "<code>mmap</code> from <code><span style="color: yellow">binary</span></code>"}

  interp@{ shape: hex, label: "has
  <code><span style="color: orange;">PT_INTERP</span></code>?" }

  interp_load@{ shape: procs, label: "<code>mmap</code> from <code><span style="color: lime">ld.so</span></code>"}

  bin_entry@{ shape: stadium, label: "➡️ <code><span style="color: yellow">binary</span></code>'s entrypoint" }

  interp_entry@{ shape: stadium, label: "➡️ <code><span style="color: lime">ld.so</span></code>'s entrypoint" }

  interp_run@{ shape: procs, label: "resolve <code>dylib</code>s
  apply relocations" }

  %% edges:

  exec --> bin_load --> interp

  interp -- "`no`" --> bin_entry

  interp -- "`yes
  <code><span style="color: lime;">/lib64/ld-linux-x86-64.so.2</span></code>`" --> interp_load

  interp_load --> interp_entry --> interp_run --> bin_entry

  style interp fill:#444

Loading

explicit ld.so invocation

flowchart TB
  exec["`<code>exec <span style="color: lime">ld.so</span></code> <code><span style="color: yellow;">path/to/binary</span></code>`"]

  bin_load@{ shape: procs, label: "<code>mmap</code> from <code><span style="color: yellow">binary</span></code>"}

  interp_load@{ shape: procs, label: "<code>mmap</code> from <code><span style="color: lime">ld.so</span></code>"}

  bin_entry@{ shape: stadium, label: "➡️ <code><span style="color: yellow">binary</span></code>'s entrypoint" }

  interp_entry@{ shape: stadium, label: "➡️ <code><span style="color: lime">ld.so</span></code>'s entrypoint" }

  interp_run@{ shape: procs, label: "resolve <code>dylib</code>s
  apply relocations" }

  %% edges:

  exec --> interp_load --> interp_entry --> bin_load --> interp_run --> bin_entry

Loading

with loader shim

flowchart TB
  exec["<code>exec <span style="color: yellow;">path/to/binary</span></code>
  (patched w/<code><span style="color: red">loader shim</span></code>)"]

  bin_load@{ shape: procs, label: "<code>mmap</code> from <code><span style="color: yellow">binary</span></code>"}

  interp@{ shape: hex, label: "has
  <code><span style="color: orange;">PT_INTERP</span></code>?" }

  bin_entry@{ shape: stadium, label: "➡️ <code><span style="color: yellow">binary</span></code>'s entrypoint
  (<code><span style="color: red">loader shim</span></code>)" }

  under_rtld@{ shape: hex, label: "under <code><span style="color: lime;">ld.so</span></code>?" }

  %% edges:

  exec --> bin_load --> interp

  interp -- "`no`" --> bin_entry --> under_rtld

  under_rtld -- "no" --> runfiles_lookup --> rtld_exec
  under_rtld -- "yes" --> rtld_bin_entry

  style interp fill:#444
  style under_rtld fill:#444

  subgraph rtld_exec_sub ["."]
    direction TB
    rtld_exec["`<code>exec <span style="color: lime">ld.so</span></code> <code><span style="color: yellow;">path/to/binary</span></code>`"]

    rtld_bin_load@{ shape: procs, label: "<code>mmap</code> from <code><span style="color: yellow">binary</span></code>"}

    rtld_load@{ shape: procs, label: "<code>mmap</code> from <code><span style="color: lime">ld.so</span></code>"}

    rtld_bin_entry@{ shape: stadium, label: "➡️ <code><span style="color: yellow">binary</span></code>'s entrypoint
    (<code><span style="color: yellow">original</span></code>)" }

    rtld_entry@{ shape: stadium, label: "➡️ <code><span style="color: lime">ld.so</span></code>'s entrypoint" }

    rtld_run@{ shape: procs, label: "resolve <code>dylib</code>s
    apply relocations" }

    runfiles_lookup@{ shape: event, label: "<code style="color: red">runfiles lookup</code>
    for <code style="color: lime"> ld.so</code>" }

    under_rtld@{ shape: hex, label: "under <code><span style="color: lime;">ld.so</span></code>?" }

    %% edges:

    rtld_exec --> rtld_load --> rtld_entry --> rtld_bin_load --> rtld_run --> bin_entry
  end

Loading

demo?

Caution

Hacky, coupled to Bazel 8, x86-64 only for now.

Important

As of this writing, Bazel's experimental hermetic linux sandbox does not set up procfs appropriately; this causes ld.so to fail to canonicalize program paths which then causes shared object resolution (i.e. for DT_NEEDED entries with $ORIGIN and such) to fail.

1. Set up an RBE service of your choosing to use an empty container image for action execution
   • the bundled example workspace's --config=rbe is set up for buildbuddy w/the rrbutani/empty container image
     • be sure to create an account, get an API key, and place it in bzl/.buildbuddy-api-key.bazelrc
2. Run bazel build //:out --config=rbe in bzl
   • if you're using nix + direnv, Bazel 8.4.2 will be fetched for you; if not you'll need to grab it yourself