BinExport2 backend

[williballenthin]

closes #1755

don't merge before v7.0.

TODO:

• add tests
  • Android/ELF/aarch64
  • Android/ELF/arm32
  • Android/ELF/amd64 -- out of scope for now?!
  • Windows/PE/i386
  • Windows/PE/amd64 -- out of scope for now?!
  • Ghidra BinExport2
  • IDA Pro BinExport2 -- out of scope for now?!
  • Binary Ninja BinExport2 -- out of scope for now?!
• add additional features:
  • instruction offset
  • instruction nzxor
  • instruction indirect call
  • basic block tight loop
  • function calls to
  • function loop
  • function recursive call
  • function name
• fix Ghidra: capture data references google/binexport#123 and remove code here
• fix Ghidra: capture string references google/binexport#124 and remove code here
• fix Ghidra: Implement full expression trees google/binexport#78 and remove code here

Checklist

• No CHANGELOG update needed

• No new tests needed

• No documentation update needed

[mike-hunhoff]

@williballenthin let me know if there is a reason we were previously using 0 here for the base address - it was causing read errors that prevented bytes from being extracted properly. I've updated the code to use the base address retrieved from the BinExport file.

[williballenthin]

In the existing design, an AddressSpace describes a module (PE/ELF) loaded into its preferred address. For PE, this is image base, and for ELF it's 0x0 because there's no preferred address.

I think it's moderately important for the AddressSpace to depend only on the PE/ELF, to minimize the amount of context/concerns. It might be nice to be able to use the code in another project, for example. The changes you introduced seem to rely on the BinExport data, which I'm initially hesitant of.

It might make more sense for the AddressSpace to use module RVAs everywhere, rather than preferred VAs. shrug

In any case, users of this class need to do a little address arithmetic before they attempt a read. This is expected. Maybe the documentation should be extended.

I do recall I waffled a bit during the implementation of this class, and I'm open to further discussion.

Thoughts?

[williballenthin]

Perhaps we could introduce a method rebase that shifts an address space from one place to another. Then we can load from a PE/ELF to its preferred location and then rebase to what's found in the BinExport. Since it's only a matter of fixing up the range addresses, it's cheap.

[mike-hunhoff]

Perhaps we could introduce a method rebase that shifts an address space from one place to another. Then we can load from a PE/ELF to its preferred location and then rebase to what's found in the BinExport. Since it's only a matter of fixing up the range addresses, it's cheap.

This sounds like a good option to me. My original change stemmed from capa not doing the arithmetic that you mentioned above before read resulting in read failures for ELF files (and inability to emit bytes features in most cases). Adding a rebase method would remove from AddressSpace users the burden of knowing what arithmetic and when to do it to help reduce bugs like this in the future.

[mike-hunhoff]

We don't handle name demangling (see #2112)

[mr-tz]

good, will update the reference in the code - I don't think this is an important feature for now

[mike-hunhoff]

💯

[mike-hunhoff]

@mr-tz tight loop is implemented - is this a bug?

[mr-tz]

it doesn't work yet for this sample - I hope it's an easy fix in the capa basic block extractor code

[mike-hunhoff]

#2113