Call Graph Discovery in Binary Programs from Unknown Instruction Set Architectures

EasyChair Preprint no. 11183

Abstract

This study addresses the challenge of reverse engineering binaries from unknown instruction set architectures, a complex task with potential implications for software maintenance and cyber-security. We focus on the tasks of detecting candidate call and return opcodes for automatic extraction of call graphs in order to simplify the reverse engineering process. Empirical testing on a small dataset of binary files from different architectures demonstrates that the approach can accurately detect specific opcodes under conditions of noisy data. The method lays the groundwork for a valuable tool for reverse engineering where the reverse engineer has minimal a priori knowledge of the underlying instruction set architecture.

Keyphrases: binary analysis, program call graph, reverse engineering, unknown CPU architecture

@Booklet{EasyChair:11183,
  author = {Håvard Pettersen and Donn Morrison},
  title = {Call Graph Discovery in Binary Programs from Unknown Instruction Set Architectures},
  howpublished = {EasyChair Preprint no. 11183},

  year = {EasyChair, 2023}}