Pushing Security Left By Mutating Byte Code

By Gaurav Gogia on 08 Sep 2022 @ Nullcon
πŸ“Š Presentation πŸ“Ή Video πŸ”— Link
#secure-coding #application-hardening #static-analysis #secure-development
Focus Areas: πŸ” Application Security , βš™οΈ DevSecOps , 🦠 Malware Analysis

Presentation Material

Abstract

Building secure software takes a lot of time and effort. Developers have to go through many steps to ensure that their software is not vulnerable to any attacks. If the product is valuable enough, such that it may attract highly skilled attackers then they even have to go the extra mile by making their code harder to reverse engineer. These products include but are not limited to video games, antivirus, firewall, intrusion detection systems, animation, video making, and other such highly-priced software.

Developers tend to minify, obfuscate and encrypt their code to make sure no one else can reverse engineer it and use it the way it was not intended in the first place. They even go so far as to write inline or embedded assembly code to make sure that their source code changes every time it’s run so that its real algorithm is harder to figure out.

But adding those levels of protection is not an easy task. A lot of time and effort is required in learning and implementing those techniques, the same time can be spent being productive in the implementation of features or working on more business-oriented processes. Mutant here is a proposed solution for the above problem. It provides a new high-level programming language to write out-of-the-box secure code and a compiler that can help protect existing binaries.

AI Generated Summary

Mutant is an experimental programming language and toolchain designed to embed security directly into the compilation process, shifting protection left in the software development lifecycle. Its core technique involves encrypting both compiled bytecode on disk and individual instructions in memory during program execution. Constants and stack data remain encrypted at rest and are only decrypted momentarily when needed by the CPU, then re-encrypted, aiming to prevent static binary analysis and memory dumping attacks.

Key features demonstrated include cross-compilation to multiple platforms (Windows PE, Linux ELF, Android/ARM) from a single source, and a custom virtual machine (MVM) that executes encrypted bytecode. A live demo compared a mutant-compiled binary containing a hard-coded secret (“42”) with a C++ equivalent. Strings analysis of the mutant binary on disk revealed no plaintext secret, and memory capture using the AVML tool during execution also failed to expose the value, whereas the C++ binary exposed it in both contexts. The encryption uses a deterministic, on-the-fly key generation based on XOR operations, claimed to incur minimal overhead (one to two clock cycles per instruction).

The approach is presented as a proof-of-concept with significant limitations. The generated binaries are large (~30 MB) due to bundling the runtime. The security relies solely on obfuscation through encryption; the open-source, deterministic key scheme could be reverse-engineered to bypass protections. The language lacks support for common operations like networking or file I/O and is not integrated with existing ecosystems (LLVM, JVM). Practical implications suggest that while such a model could eventually allow developers to write business logic without manually implementing cryptography, mutant currently requires adoption of a new language and is not battle-tested for complex applications. Future work mentioned includes bytecode compression, dynamic stack handling, and creating plugins for established compilers to avoid requiring a new language.

Disclaimer: This summary was auto-generated from the video transcript using AI and may contain inaccuracies. It is intended as a quick overview β€” always refer to the original talk for authoritative content. Learn more about our AI experiments.