Ignore SHF_ALLOC sections in VM memory map

[kesyog]

We've run into an issue where symbols in output sections marked INFO in our linker script show up in our bloaty output with any of -d symbols, -d sections,symbols, or -d segments,symbols.

I think the root cause is that:

• Our executable's VM memory map has valid SHF_ALLOC regions with low addresses near 0
• In lld, non-SHF_ALLOC sections always get 0 sh_addr.
• AFAICT, once bloaty has built up a range map of the memory space, it will assign symbols to a section purely based on the symbol's address. The address of our non-SHF_ALLOC INFO section overlaps with our SHF_ALLOC section near 0, which confuses Bloaty.

Would it be reasonable to check each symbol's section header index (st_shndx) and discard it from VM space if the corresponding section is not SHF_ALLOC?

[EricRahm]

@kesyog Thanks for the report! Your analysis and proposed fix sound about right. It would be helpful if you could provide an example binary that reproduces this for you or a simplified example of sources, linker script, and build commands so that we can repro locally.

[kesyog]

I threw together a small reproducer: https://github.com/kesyog/bloaty-469.

For more context, our use case is an embedded project that stores a Pigweed tokenization database as metadata in the linked executable.

[EricRahm]

This is super helpful, thanks for putting together the reproducer!

Just confirming what you noted:

 % readelf -S build/app
There are 30 section headers, starting at offset 0x1d10:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .text             PROGBITS         0000000000000000  00001000
       00000000000000ba  0000000000000000  AX       0     0     16
<snip>
  [25] .metadata         PROGBITS         0000000000000000  000016c0
       0000000000000024  0000000000000000   W       0     0     16
<snip>

So we see .text and .metadata both have an 0x0 address. From the example we're adding items to .metadata section via an annotation:

// These metadata symbols are placed in an INFO output section
static int metadata1 __attribute__((used, section(".metadata"))) = 6;
static int metadata2 __attribute__((used, section(".metadata"))) = 5;
static int metadata3[5] __attribute__((used, section(".metadata")));

And the linker script is explicitly keeping that section around and marking it INFO:

SECTIONS {
  .text :
  {
    *(.text.main)
  } > REGION1

  /* Note: address is ignored by lld. will always start at 0x0 */
  .metadata 0xFFFFFFFF (INFO) :
  {
    KEEP(*(.metadata))
  }
}

[haberman]

Ignoring non-SHF_ALLOC sections in the memory map makes sense to me, but I think we do that already, don't we?

bloaty/src/elf.cc

Line 1059 in f9a1183

auto vmsize = (header.sh_flags & SHF_ALLOC) ? size : 0;

I guess the more unusual thing is to have entries in the symbol table that point to a non-SHF_ALLOC section. In that case, I agree it makes sense to leave them out of the VM map and only add them to the file map. I presume that we do want them in the file map, so we can properly represent the bytes in the file that they occupy?

To do that properly, we would need to have them in the base map even though they overlap with other sections in VM space. This is necessary because the base map is how we translate a VM address to a file offset.

I think this would require giving Bloaty's internal representation of a VM address an explicit "segment", as I proposed here: #153 (comment) The fact that two different features could potentially benefit from this direction gives me more confidence that this would be a good move.

[EricRahm]

You're right for sections we have a VM size of zero:

~/bloaty/build/main/bloaty build/app -d sections -n 0
    FILE SIZE        VM SIZE
 --------------  --------------
<snip>
   0.4%      36   0.0%       0    .metadata
<snip?
 100.0%  9.14Ki 100.0%  1.75Ki    TOTAL

For symbols the items in the .metadata section are given a VM size:

 ~/bloaty/build/main/bloaty build/app -d symbols -n 0
    FILE SIZE        VM SIZE
 --------------  --------------
<snip>
   2.7%     249   2.7%      48    metadata1
<snip>
   0.6%      54   1.1%      20    metadata3
<snip>
   0.4%      38   0.2%       4    metadata2
   0.4%      36   0.0%       0    [section .metadata]
<snip>
 100.0%  9.14Ki 100.0%  1.75Ki    TOTAL

That's more or less coming from here:

bloaty/src/elf.cc

Lines 905 to 916 in f9a1183

	string_view name = strtab_section.ReadString(sym.st_name);
	uint64_t full_addr =
	ToVMAddr(sym.st_value, index_base + sym.st_shndx, is_object);
	if (sink && !(capstone_available && disassemble)) {
	sink->AddVMRangeAllowAlias(
	"elf_symbols", full_addr, sym.st_size,
	ItaniumDemangle(name, sink->data_source()));
	}
	if (table) {
	table->insert(
	std::make_pair(name, std::make_pair(full_addr, sym.st_size)));
	}

For the non-alloc case we can probably just use sink->AddFileRange, I have a proof-of-concept that does that:

~/bloaty/build/469/bloaty build/app -d sections,symbols -n 0
    FILE SIZE        VM SIZE
 --------------  --------------
<snip>
   2.0%     188  10.5%     188    .text
    40.4%      76  40.4%      76    __do_fini
    34.0%      64  34.0%      64    __do_init
    25.5%      48  25.5%      48    main
<snip>
   0.4%      36   0.0%       0    .metadata
    55.6%      20   NAN%       0    metadata3
    33.3%      12   NAN%       0    metadata2
    11.1%       4   NAN%       0    metadata1
<snip>
 100.0%  9.14Ki 100.0%  1.75Ki    TOTAL

So I think we can solve this with a slightly simpler approach.

[haberman]

I have a proof-of-concept that does that:

Interesting, how does it work? How are you mapping the symbol address to a file range?

[EricRahm]

Here's a snippet from a WIP PR:

            if (sym.st_shndx < SHN_LORESERVE) {
              ElfFile::Section symbol_section;
              elf.ReadSection(sym.st_shndx, &symbol_section);
              if (!(symbol_section.header().sh_flags & SHF_ALLOC)) {
                uint64_t offset =
                    symbol_section.header().sh_offset +
                    (sym.st_value - symbol_section.header().sh_addr);
                sink->AddFileRange("elf_symbols", name, offset, sym.st_size);
                continue;
              }
            }

07ce312#diff-b1862056e7e3999739ef8b076030deb100253c7d9dcc9ecacb8975acec6b1b6cR906-R917

[EricRahm]

@kesyog If you have time it'd be great to see if #471 fixes your issue locally.

[kesyog]

Thanks, that worked great for me locally, both with the reproducer and in my team's actual project.

Unrelated to this issue, but it looks like bloaty.h is missing a #include <list>. Compilation failed for me without fixing that (with clang).

[EricRahm]

Unrelated to this issue, but it looks like bloaty.h is missing a #include <list>. Compilation failed for me without fixing that (with clang).

Ah that's probably a regression from #467, I'll get a fix landed for that. Thanks for reporting!