Code attestation for monitor compromise detection

Abstract

Runtime monitors are programs that observe a system’s execution to detect deviations from expected behaviour. This makes them valuable in security contexts, where they can be used for the detection of malicious activity. However, their effectiveness depends on the assumption that the monitor itself has not been compromised. From an attacker’s point of view, the monitor poses a direct obstacle to evading detection, making it a high-value target. An attacker with sufficient privileges may modify the monitor’s in-memory code to ensure that malicious activity goes undetected. Without strong assurances that the monitor remains secure and untampered, the reliability of the monitoring process, and thus the security of the entire system is undermined. This work addresses this challenge by designing and implementing a remote code attestation mechanism. Remote code attestation is a cryptographic technique in which proofs describing the state of executing code are periodically generated and sent to an external verifier for validation. The proposed solution adopts a challenge–response protocol, where the monitor acts as a prover and responds to unpredictable challenges with attestations of its current code state. Any deviation from the expected state results in a verification failure, enabling timely detection of tampering. Through the development of a prototype and its evaluation under multiple tampering scenarios, we demonstrate that the mechanism reliably detects in-memory code modifications. From our testing, we deduce that full attestation introduces moderate overhead, while an optimisation based on pseudorandom traversal can reduce this cost. This enables flexible trade-offs between performance and security, making remote code attestation both a practical and effective mechanism for detecting tampering of runtime monitors.