Explore the internals of applications built using native ahead-of-time (AOT) compilation.

The .NET 7 platform debuted a new deployment model: forward native compilation. When a .NET application is compiled natively using the AOT method, it becomes a self-contained native executable, equipped with its own minimal runtime to control code execution.

The execution time is very low and in .NET 8 you can create standalone C# applications that are less than 1 MB in size. For comparison, the size of a native AOT Hello World application in C# is closer to the size of a similar application in Rust than in Golang, while being one-sixth the size of a similar application in Java.

In addition, for the first time in history, .NET programs are distributed in a file format other than what is defined in ECMA-335 (i.e., as instructions and metadata for the virtual machine), namely, distributed as native code (file format PE/ELF/Mach-O) with native data structures, just like, for example, in C++. This means that none of the .NET reverse engineering tools created in the last 20 years work with native forward compilation.

Unfortunately, due to these two aspects (compactness and complexity of reverse engineering), native AOT compilation is popular among malware authors, as evidenced, for example, by these articles:

Here we will try to talk a little about how to adapt reverse engineering to new conditions.

Preparing Ghidra and native debuggers

I will repeat the idea from the introduction: native AOT compilation does not work with the file formats that are used in CLR virtual machines to store a program and its metadata. Tools to read the VM file format are useless when working with native executables for AOT. It remains to use tools designed for reverse engineering of arbitrary native code, in particular, native debuggers (WinDBG/VS/x64dbg on Windows, lldb/gdb on Unix-like systems) and code analysis frameworks (Ghidra, IDA, Binary Ninja, etc.) .d.).

Since in native AOT mode the program is compiled into a single executable file without dependencies, the amount of available metadata is significantly reduced, but some metadata still remains (as, for example, in C++).

Consider a binary file

If you want to go deeper, install the .NET 8 SDK (I’m using version RC1, the latest available at the time of this writing). You can skip the installation and simply download the ZIP archive and specify the location of the extracted files in your PATH.

Let’s start with the Hello World application with native AOT:

$ dotnet new console --aot -o TestApp

Create a new TestApp directory and place the Hello World console application project there, configured for ahead-of-time compilation.

$ cd TestApp
$ dotnet publish

Once the publishing process is complete, you should see the binary in the bin\Release\net8.0\win-x64\publish folder (I did this on Windows, but it will work on Linux/Mac). The binary file is about 1.2 MB in size, and next to it is a file with native debugging information (PDB on Windows, DBG on Linux, and whatever on Mac). Let’s take a look at what we got.

$ dumpbin bin\Release\net8.0\win-x64\publish\TestApp.exe
Microsoft (R) COFF/PE Dumper Version 14.37.32824.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file bin\Release\net8.0\win-x64\publish\TestApp.exe

File Type: EXECUTABLE IMAGE

  Summary

        D000 .data
       5E000 .managed
        B000 .pdata
       60000 .rdata
        1000 .reloc
        1000 .rsrc
       64000 .text
        1000 _RDATA
       31000 hydrated

Nothing unusual in appearance. The section .managedcontains managed code (in this case, “native code whose memory is managed by the garbage collector”). The section hydrated is not initialized, but it is populated early in startup with runtime data structures.

The remaining sections also look pretty standard: .text they contain unmanaged code, in particular, the garbage collector itself, or other native code that the user himself associated with the executable file.

Running the command strings in the executable brings us to some interesting things, in particular:

8.0.23.41904v8.0.0-rc.1.23419.4+92959931a32a37a19d8e1b1684edc6db0857d7de

(The version of the commit hash from the dotnet/runtime repository, the same one from which the executable file was obtained, may be useful to us later.)

Pay attention to lines such as DivideByZeroException or get_CanWrite, if you’re lucky, based on them we can recover useful information about types and methods.

Debug memory allocation and virtual call

An interesting experiment to understand how everything works is to run a small piece of code. Let’s replace Program.cs with the following listing:

using System.Runtime.CompilerServices;

class Program
{
    [MethodImpl(MethodImplOptions.NoOptimization | MethodImplOptions.NoInlining)]
    static void Main() => Console.WriteLine(new Program().ToString());

    public override string ToString() => "Hello World!";
}

We again execute dotnet publish and run the program under the debugger. This is where we really need the luxury of debugging symbols in the application. When researching malware, the chances of getting a PDB/DBG are very low. Let’s set a breakpoint in the Main line and see what disassembling gives us:

00007FF730B8FD50  push        rbp  
00007FF730B8FD51  sub         rsp,30h  
00007FF730B8FD55  lea         rbp,[rsp+30h]  
00007FF730B8FD5A  xor         eax,eax  
00007FF730B8FD5C  mov         qword ptr [rbp-8],rax  
00007FF730B8FD60  lea         rcx,[TestApp_Program::`vftable' (07FF730BCC688h)]  
00007FF730B8FD67  call        RhpNewFast (07FF730AF1DE0h)  
00007FF730B8FD6C  mov         qword ptr [rbp-8],rax  
00007FF730B8FD70  mov         rcx,qword ptr [rbp-8]  
00007FF730B8FD74  call        TestApp_Program___ctor (07FF730B8FDB0h)  
00007FF730B8FD79  mov         rcx,qword ptr [rbp-8]  
00007FF730B8FD7D  mov         rax,qword ptr [rbp-8]  
00007FF730B8FD81  mov         rax,qword ptr [rax]  
00007FF730B8FD84  call        qword ptr [rax+18h]  
00007FF730B8FD87  mov         rcx,rax  
00007FF730B8FD8A  call        System_Console_System_Console__WriteLine_12 (07FF730B56190h)  
00007FF730B8FD8F  nop  
00007FF730B8FD90  add         rsp,30h  
00007FF730B8FD94  pop         rbp  
00007FF730B8FD95  ret

The code looks pretty standard. There have been additional register/stack shuffles since we turned off optimizations for clarity. Symbolic names are only visible because we had debugging information. If we didn’t have it, TestApp_Program::vftable' it could have a single value of 07FF730BCC688h.

Let’s take a closer look:

00007FF730B8FD60  lea         rcx,[TestApp_Program::`vftable' (07FF730BCC688h)]  
00007FF730B8FD67  call        RhpNewFast (07FF730AF1DE0h)

Here you can see how memory allocation is organized: we load the address of the structure vftable that describes the class Program, and call the helper RhpNewFast to allocate an instance of this structure from the garbage collector heap. Since .NET is open source, we can look at the details, but essentially we are reading a field from the structure vftable to determine the size of the allocated memory (the size of the Program class instance). We cut out a piece of memory that is reset when allocated (“bump allocation”) and write the address vftable into the first field of the newly allocated instance, thus “identifying” the memory fragment. If the buffer allocator runs out of memory, then everything can be done a little slower, but this option is not so interesting.

The code RhpNewFast is written in assembly language and rarely changes, so it’s likely that you can find it yourself without even resorting to debugging symbols.

After a new object instance is allocated, the instance constructor is called:

00007FF730B8FD70  mov         rcx,qword ptr [rbp-8]  
00007FF730B8FD74  call        TestApp_Program___ctor (07FF730B8FDB0h)

Since we have debug symbols, we see the symbol name (TestApp_Program___ctor). If we didn’t have characters, this would be a call 07FF730B8FDB0h.

After the constructor completes, we make a virtual call ToString. This is another interesting detail:

00007FF730B8FD81  mov         rax,qword ptr [rax]
00007FF730B8FD84  call        qword ptr [rax+18h]

First we dereference the object reference. As we saw during the allocation, we still have the address of the structure vftable in rax. We then call the address 0x18 bytes into the structure vftable. Presumably, this is where the method address is stored (Program.ToString).

The structure vftable is a table of virtual methods, familiar to us from C++. It lists all addresses of virtual methods implemented by the type. It also contains additional metadata, such as the size of the object instance, whether it is a struct or a class, etc. In the .NET world, the first three slots of vftable are almost always implementations of object.ToString, object.GetHashCode, and object.Equals (however, the order of these three entities depends on the optimization of the overall program).

Native AOT causes a structure vtable MethodTable or EEType, moreover, one can replace another. You can learn more about it by viewing the write or read implementation. (Keep in mind that the CoreCLR VM also has a MethodTable function, but its structure is different.)

Although the data structure MethodTable is informative, the most useful information, particularly type names, is not always available. Other things we also don’t have:

List all methods (we can at least list virtual method addresses, like in C++ reverse engineering)
List of all fields (however, the garbage collector information given before MethodTable allows you to judge at what offsets within the object instance the garbage collector pointers are located, which is still better than nothing)
Contents of assembly type
Etc.

Dehydrogenated data

An additional complication is that the MethodTable data structures are placed in the hydrated segment of the executable, which is zeroed upon initialization (zero init). At the beginning of the startup sequence there is a small piece of code that fills this segment with actual data. This makes it even more difficult for static analytics tools to interpret the contents of MethodTables unless they are paged out of memory.

Data dehydrogenation was discussed here, and this pull request describes what happens even better than I could describe in this article. Essentially, this data is stored in a more compact form in a file format and is expanded at runtime. Perhaps this phenomenon could be simulated in static analytics tools by determining which blob contains the relevant data, starting with the RTR header. However, this file format does not conform to any ABI, it is subject to change, and may have to be updated every year for new versions of .NET.

Reflection of data structures

Although naming information is not so easily obtained, it is still present in the row dump, as we can see. Reflection keeps track of all type names because in .NET you can simply call object.GetType on any object and find out its name.

The data blob that maps MethodTable data structures to metadata descriptors is associated with an RTR header, as is the metadata blob itself. In theory, you could use the metadata reading APIs to recover the symbolic names for all MethodTables in a program. However, none of these formats or APIs are intended to be freely usable and will likely change with each major .NET release.

For example, an experienced malware author could also publish his application with the property IlcDisableReflection set to true, which would allow him to disable reflection and not generate any metadata about reflection. This mode is not supported or documented outside the dotnet/runtime repository.

Stacktrace data structures

Similarly, as with dump strings, we need information about method names. The only reason it is available is because this information is needed to generate a stack backtrace. When an exception is thrown, the developer can check for it using ToString or access the property StackTrace to print a text stack trace. To do this, the addresses of native methods and metadata are constantly compared, so names and signatures can be created. Reflection data is generated in much the same way, and the file formats are the same (they are also referenced in the RTR header).

using System.Runtime.CompilerServices;

class Program
{
    [MethodImpl(MethodImplOptions.NoOptimization | MethodImplOptions.NoInlining)]
    static void Main() => Console.WriteLine(new Program().ToString());

    public override string ToString() => throw new Exception();
}

(We’ve updated the previous program to allow ToStringan exception to be thrown and still be left unhandled.)

Unhandled Exception: System.Exception: Exception of type 'System.Exception' was thrown.
   at Program.ToString() + 0x24
   at Program.Main() + 0x37

Note that the application was able to output the names and signatures of the methods involved. This approach will work even if we get rid of the debugging information by deleting the PDB/DBG file.

However, the user can set the StackTraceSupport property to false at the time of publishing their application to disable the generation of this data (stack trace data generation is enabled by default). Then the program will instead output:

Unhandled Exception: System.Exception: Exception of type 'System.Exception' was thrown.

   at TestApp!<BaseAddress>+0x9dab4
   at TestApp!<BaseAddress>+0x9da77

If the application was built this way, our chances of recovering method names or signatures are almost zero. Some method names may still be available in the reflection metadata, but there are usually very few methods visible through reflection - the compiler aggressively removes them unless an analysis of trimming capabilities recommends otherwise.

Summary

To summarize, parsing .NET binaries compiled natively using AOT requires the same skills as parsing C++, for example. Some information is easy to find (like stack unwinding, some type information, etc.), but we can forget about the luxury of being able to split types into separate fields and control access to them. The fields basically dissolve into access instructions (we can guess that something might be init if the field reads as 4 bytes). Method names will disappear if stack trace data is disabled. Type names may also disappear if you turn off reflection.