D-Generating EDR Internals, Part 1

Recently, @JonasLyk released D-Generate, a polyglot that utilises DTrace to provide users with introspection into the system calls made by: the entire system, a specific application specified by the PID or image file name. It would help if the reader got set up with the instructions and downloaded the associated symbols necessary. Furthermore, if possible, the reader should set up a virtual machine without any security solutions to observe unbiased behaviour (a critical step to determine whether the observed behaviour is that of the security product or the windows kernel).

Another possible step alternatively is to modify a clone of our current boot configuration entry.

bcdedit /copy {current} /d "Local Debug"

bcdedit /set {bd8e4076-eec8-11ec-9dec-uniqueid} dtrace on

(Ensure "Local Debug" is selected on reboot)

A few reasons for the express approval of such a tool include but are not limited to:

• It is trivial to enable DTrace with a simple bcdedit /set dtrace on and then interact with it from an elevated session
• The lack of test signing requirements allows it to seamlessly interoperate with EDRs and other security products.
• The script by Jonas automatically performs handle-tracking, allowing an easier time understanding application flows.
• It can ascertain the KPROCESSOR_MODE, which specifies whether it was previously either UserMode/KernelMode for the current thread.

The current post avoids EDR-specific implementations; alternatively, it will look at various implementations of correlating information to understand the process, therefore not covering any novel techniques. @jackson_t has significantly influenced this work and has focused more on improving mental models and understanding. Thus, this post will look at reverse engineering (no specific) endpoint security solutions alternatively to typical anecdotal testing. There may be better-suited ways of approaching this issue; however, the author has found great entertainment in this approach and will continue as such

 

By the end of this blog post, the reader should understand one approach to analysing D-Generate (DTrace) logs, specifically with a focus on endpoint security product internals. Understanding the flow of an application, and how it differs, is critical to breaking it, and seeing the system calls made, and the ability to perform a quick analysis is critical to evaluating EDRs.

To proceed with the testing, as mentioned earlier, we need to have a userland program to correlate information from the traced output. 

// ModuleTracker - Dumps modules and corresponding sections

// @rad9800

#include <Windows.h>
#include <winternl.h>

#pragma comment(linker,"/ENTRY:entry")

template <typename Type>                                        
inline Type RVA2VA(LPVOID Base, LONG Rva) {
    return (Type)((ULONG_PTR)Base + Rva);
}

#define PRINT( STR, ... )\
    if (1) {\
        LPWSTR buf = (LPWSTR)HeapAlloc( GetProcessHeap(), HEAP_ZERO_MEMORY, 1024 );\
        if ( buf != NULL ) {\
            int len = wsprintfW( buf, STR, __VA_ARGS__ );\
            WriteConsoleW( GetStdHandle( STD_OUTPUT_HANDLE ), buf, len, NULL, NULL );\
            HeapFree( GetProcessHeap(), 0, buf );\
        }\
    } 

    

// Application entry point
int entry(PPEB peb)
{

    LIST_ENTRY* head = &peb->Ldr->InMemoryOrderModuleList;
    LIST_ENTRY* next = head->Flink;
    LPVOID ntdll = NULL;
    DWORD pid = GetCurrentProcessId();
    PRINT(L"PID\t%d\t0x%X\n", pid, pid);
    while( next != head )
    {
        LDR_DATA_TABLE_ENTRY* entry = (LDR_DATA_TABLE_ENTRY*)((PBYTE)next - offsetof( LDR_DATA_TABLE_ENTRY, InMemoryOrderLinks ));
        UNICODE_STRING* fullname = &entry->FullDllName;
        UNICODE_STRING* basename = (UNICODE_STRING*)((PBYTE)fullname + sizeof( UNICODE_STRING ));

        PRINT(L"%s\t\t0x%p\n", basename->Buffer, entry->DllBase);

        HMODULE module = (HMODULE)entry->DllBase;

        PIMAGE_DOS_HEADER dos = (PIMAGE_DOS_HEADER)entry->DllBase;
        PIMAGE_NT_HEADERS nt = RVA2VA<PIMAGE_NT_HEADERS>(entry->DllBase, dos->e_lfanew);


        for (int i = 0; i < nt->FileHeader.NumberOfSections; i++) {
            PIMAGE_SECTION_HEADER section =
                (PIMAGE_SECTION_HEADER)((DWORD_PTR)IMAGE_FIRST_SECTION(nt) +
                    ((DWORD_PTR)IMAGE_SIZEOF_SECTION_HEADER * i));
            
            PVOID base = RVA2VA<LPVOID>(module, section->VirtualAddress);
            ULONG size = section->Misc.VirtualSize;

            PRINT(L"\t%S\t0x%p\t0x%x\n", section->Name, base, size);
        }

        next = next->Flink;
    }

    return 0;
}

The code sample above does a few various things that complement our tracing as opposed to running an empty application:

• Prints our process ID (GetCurrentProcessID()) so we can confirm in our DTrace logs that these are the corresponding logs
• Walks the InMemoryOrderModuleList to print loaded modules and their corresponding address
• Prints the associated section names, base address, and size for each loaded modules

Thus we can addresses allow us to quickly search through the produced logs to find references to these memory addresses.

On the target system, launch D-Generate from an elevated prompt, passing the image name, and then after a bit, in another prompt, launch Application.exe.

dg ModuleTracker.exe > ModuleTracker_Trace.txt

The ModuleTracker.exe should produce a log similar to the truncated example below.

 

 

ModuleTracker.exe          0x00007FF763FC0000
        .text   0x00007FF763FC1000      0x204
        .rdata  0x00007FF763FC2000      0x316
        .pdata  0x00007FF763FC3000      0xc
        .rsrc   0x00007FF763FC4000      0x1e0
NotNtdll.dll               0x000002603A7D0000
        .text   0x000002603A7D1000      0x128850
        PAGE    0x000002603A8FA000      0x503
        RT      0x000002603A8FB000      0x1e3
        .rdata  0x000002603A8FC000      0x476bf
        .data   0x000002603A944000      0xb068
        .pdata  0x000002603A950000      0xebbc
        .mrdata 0x000002603A95F000      0x3520
        .00cfg  0x000002603A963000      0x20
        .rsrc   0x000002603A964000      0x73528
        .reloc  0x000002603A9D8000      0x53c
NotKernel32.dll            0x000002603A6D0000
        .text   0x000002603A6D1000      0x7cdc7
        .rdata  0x000002603A74E000      0x334ac
        .data   0x000002603A782000      0x1234
        .pdata  0x000002603A784000      0x5490
        .didat  0x000002603A78A000      0xa8
        .rsrc   0x000002603A78B000      0x520
        .reloc  0x000002603A78C000      0x334
ntdll.dll               0x00007FFFD9FE0000
        .text   0x00007FFFD9FE1000      0x128850
        PAGE    0x00007FFFDA10A000      0x503
        RT      0x00007FFFDA10B000      0x1e3
        .rdata  0x00007FFFDA10C000      0x476bf
        .data   0x00007FFFDA154000      0xb068
        .pdata  0x00007FFFDA160000      0xebbc
        .mrdata 0x00007FFFDA16F000      0x3520
        .00cfg  0x00007FFFDA173000      0x20
        .rsrc   0x00007FFFDA174000      0x73528
        .reloc  0x00007FFFDA1E8000      0x53c
KERNEL32.DLL            0x00007FFFD8DD0000
        .text   0x00007FFFD8DD1000      0x7cdc7
        .rdata  0x00007FFFD8E4E000      0x334ac
        .data   0x00007FFFD8E82000      0x1234
        .pdata  0x00007FFFD8E84000      0x5490
        .didat  0x00007FFFD8E8A000      0xa8
        .rsrc   0x00007FFFD8E8B000      0x520
        .reloc  0x00007FFFD8E8C000      0x334

[...TRUNCUATED...]

Following this, any EDR which utilises a userland DLL for introspection may show a module loaded into our process, often with a unique name. The injected DLL name will be helpful for us later, so take note. This DLL will often be responsible for many activities that look to grant it introspection, including but not limited to the installation of process instrumentation callbacks, native API hooks, and memory page guards. 

Let us think about what we are trying to achieve and thus define our independent and dependent variables at a high level and outline a rough hypothesis.

 

IV = The presence of an endpoint security product on the host.

DV = The syscalls made by/relative to a target application.

Hypothesis = Running our bespoke application in the presence of an endpoint security product will increase the number of security-related syscalls made.

We can achieve our required IV with an environment we can use clone/duplicate trivially. Microsoft provides us with a Windows 11 virtual environment on a 90-day license. Set up DTrace as required, ensure the virtual machine is up-to-date, and re-acquire symbols for ntoskrnl.exe if necessary. Other tools often utilised installed on both systems include ProcExp and Everything.

We can measure our IV's effect using D-Generate, manually looking through the traces, and observing patterns around activities starting with the addresses output by our helper application.

Syscall Frequency

Given 2 logs, one affected by our IV, we can calculate the frequency of the syscalls to obtain a picture of the behaviour of the endpoint security product and the default behaviour. 

Python will be used to perform this analysis due to its simplistic nature. The author by no means claims to be a Python specialist, so there probably are better "Pythonic" ways of approaching the same issue. We aggregate the syscalls and their frequency in the first script, an easily comparable statistic.

import io
import sys

def parse( filename: str ) -> int:
    with io.open( filename ) as f:
        log = f.read( )

    results = { }
    total = 0

    for _, token in enumerate( log.split( ) ):
        if token[-4:] == 'CALL' and token[:2] == 'Nt':
            if token not in results:
                results[token] = 1
            else:
                results[token] += 1
               
    sorteddict = { k: v for k, v in
    sorted( results.items( ), key=lambda item: item[1] ) }
    
    for function, count in reversed( sorteddict.items( ) ):
        print( "{} {}".format( function, count ) )
        total += count

    return total

def main( ):
    total = parse( sys.argv[1] )
    print("Total={}".format(total))
    
if __name__ == '__main__':
    main( )

 

 

 

Running this script against our affected (security solution present) and unaffected results will give us data we can begin to compare.

EDR Present

NtProtectVirtualMemoryCALL 226
NtCloseCALL 189
NtQueryVirtualMemoryCALL 181
NtDeviceIoControlFileCALL 153
NtAllocateLocallyUniqueIdCALL 152

[...TRUNCUATED...]

NtQuerySymbolicLinkObjectCALL 1
NtOpenSymbolicLinkObjectCALL 1
NtManageHotPatchCALL 1
NtQueryFullAttributesFileCALL 1
 

 

Default Windows

NtQueryVirtualMemoryCALL 459
NtCloseCALL 124
NtAllocateLocallyUniqueIdCALL 98
NtDeviceIoControlFileCALL 97
NtProtectVirtualMemoryCALL 91

[...TRUNCUATED...]

NtConnectPortCALL 1
NtQuerySymbolicLinkObjectCALL 1
NtOpenSymbolicLinkObjectCALL 1
NtManageHotPatchCALL 1

Our original hypothesis is confirmed (or so the author would like to believe)

Biased Result Total=1798

Unbiased Result Total=1397

The biased result (presence of the endpoint security product) shows just over 400 more syscalls made. To improve this, the reader could repeat the experiment three times. To further evaluate this, I am unsure why the default version of Windows (both 22000.978) called NtQueryVirtualMemory over 2 times the one with an EDR present!

 Now, we will look at some exciting functions and create hypotheses around their usage. We start at NtProtectVirtualMemory, called 115 times more in the presence of an EDR, as shown by the logs. Prior knowledge will allow us to build a hypothesis we can look to confirm through manual parsing of the logs or scripting. It is now worth redefining our DV for clarity (note that we do not need to measure anything else).

DV = The arguments passed to the syscalls made by a target application.

Hypothesis = Endpoint security products will call NtProtectVirtualMemory to set inline hooks and apply page guards.

Let us look to now verify this hypothesis. This part of the analysis requires the reader to understand the default pattern of memory protection, using the module and section addresses printed out by our helping application.

 

A script to pull out various arguments of NtProtectVirtualMemory captured by DTrace allows us to focus on the raw bits.

 

import io
import sys

def isNtToken( token: str ) -> bool:
    if token[-4:] == 'CALL' and token[:2] == 'Nt':
        return True
    else:
        return False
        
def select( filename: str ):
    with io.open( filename ) as f:
        log = f.read( )
    
    select = ( ["caller", "PreviousMode", "BaseAddress", "RegionSize", "NewProtect", "OldProtect", "result"] )
    results = [ ]
    index = 0
    
    split = log.split( )
    splitlen = len( split )
    while( index < splitlen ):
        token = split[index]
        if isNtToken( token ):
            if token == "NtProtectVirtualMemoryCALL":          
                meta = { }
                offset = 0
                while( offset < 128 and ( index + offset ) < splitlen ):
                    newToken = split[index + offset]
                    if  newToken in select:
                        meta[newToken] = split[index + offset + 2]
                        offset += 2
                    offset += 1
                results.append( meta )
        index += 1
        
    for entry in results:
        for k, v in entry.items( ):
            print( "{}={}".format( k, v ) )  

def main( ):
    select( sys.argv[1] )

    
if __name__ == '__main__':
    main( )

 

Example output (extracted information highlighted)

struct NtProtectVirtualMemoryCALL {
    BOOL syscallEntered = false
    PEPROCESS EPROCESS = 0xffff8581a33a60c0
    PETHREAD ETHREAD = 0xffff8581a2261080
    PVOID caller = 0
    struct execimage imageFile = {
        char [15] filename = [ "tracker.exe" ]
    }
    KPROCESSOR_MODE PreviousMode = UserMode
    PID pid = 0x1450
    TID tid = 0x1544
    BOOL impersonating = false
    struct _TOKEN *token = 0xffffc30f7e3ba930
    TOKEN_TYPE tokenType = TokenPrimary
    INTEGRITY_LEVEL integrity = MEDIUM
    struct NtProtectVirtualMemory syscall = {
        HANDLE ProcessHandle = {
            void *h = 0xffffffffffffffff
            STR name = {
                char [256] chars = [ "NtCurrentProcess" ]
            }
        }
        PVOID BaseAddress = 0x7ffc1c172510
        SIZE_T RegionSize = 0x8
        ULONG NewProtect = 0x4
        ULONG OldProtect = 0
        NTSTATUS result = STATUS_SUCCESS
    }
}

 

Extracted to:

 

caller=0
PreviousMode=UserMode
BaseAddress=0x7ffc1c172510
RegionSize=0x8
NewProtect=0x4
OldProtect=0
result=STATUS_SUCCESS

 

With a clean output, we will look for page guards. Various memory-protection constants are available here or in WinNT.h. Enough chat; let us start picking apart and understanding patterns. 

In the application output on an unbiased device, we get something akin to this:

Application.exe 0x00007FF7A9040000

        .text 0x00007FF7A9041000 0x204

ntdll.dll 0x00007FFC1BFE0000

        .text 0x00007FFC1BFE1000 0x128850

KERNEL32.DLL 0x00007FFC19E70000

        .text 0x00007FFC19E71000 0x7cdc7

We can manually search through the NtVirtualProtectMemory, looking for references to 7FF7A90, 7FFC1BFE/7FFC1C, and 7FFC19E/7FFC19F. While our process loads 11 DLLs, we must understand the protections around NTDLL and Kernel32 as they have a greater priorty (NTDLL allows us to make syscalls, and Kernel32 provides a great deal of functionality and wrapper functions for NTDLL).

7FF7A90, the "stem" virtual address for Application.exe, is referenced only twice by NtProtectVirtualMemory (unbiased). Both are setting the
protections of the .rdata section located at 0x00007FF7A904200, one
setting PAGE_READONLY and another setting PAGE_READWRITE.

caller=0
PreviousMode=UserMode
BaseAddress=0x7ff7a9042000 (.rdata)
RegionSize=0x48
NewProtect=0x4 (PAGE_READWRITE)
OldProtect=0
result=STATUS_SUCCESS

Searching for merely 7FFC1BFE is unsuitable for NTDLL.dll, and there are no references. The other sections begin with 7FFC1C (following .text); thus, we can look for references to this. There are 29 references to the .MRDATA sections where it oscillates its protections between 0x4 (PAGE_READWRITE) and 0x2 (PAGE_READONLY). There are two further calls to change parts of the protection of .00cfg to PAGE_WRITECOPY and PAGE_READWRITE (a control flow guard section, read more here).

caller=0
PreviousMode=UserMode
BaseAddress=0x7ffc1c16f000 (.mrdata)
RegionSize=0x3520
NewProtect=0x2 (PAGE_READONLY)
OldProtect=0x2
result=STATUS_SUCCESS

caller=0
PreviousMode=UserMode
BaseAddress=0x7ffc1c16f000 (.mrdata)
RegionSize=0x3520
NewProtect=0x4 (PAGE_READWRITE)
OldProtect=0x4
result=STATUS_SUCCESS

The NtProtectVirtualMemory calls on the biased machine show it making six calls (as opposed to two). Without going into the EDR-specific details, we can quickly identify a pattern (thanks to the isolated logs). It changes an address after the virtual base address of our application and before the text section to 0x8 (PAGE_WRITECOPY) and then sets it back to 0x2 (PAGE_READONLY). Our initial assumptions would be that they tamper with something internal to our process; the author would like to leave that as a challenge to the reader. It does this twice and then has the calls made in an unbiased environment.

This particular EDR has fake replica DLLs, as shown at the start of the blog post. On these DLLs, it sets a protection value of 0x120 (PAGE_GUARD | PAGE_EXECUTE_READ) on the .text section. Several EDRs use PAGE_GUARDs in order to fool shellcode/malware. Regarding the original NTDLL, there is still the same activity.MRDATA and .00cfg but no additional activity about it.

caller=0
PreviousMode=UserMode
BaseAddress=0x2603a7d1000 (NotNtdll.dll .text)

RegionSize=0x115cb1
NewProtect=0x120 (PAGE_GUARD | PAGE_EXECUTE_READ)

OldProtect=0x3a5f1c70
result=STATUS_SUCCESS

 

An interesting observation is that it set around 50 pages to 0x40 (PAGE_EXECUTE_READWRITE) for a region size of 0x8 and then restored it to 0x80 (PAGE_EXECUTE_WRITECOPY) or 0x20 (PAGE_EXECUTE_READ) when finished. 

Setting PAGE_WRITECOPY is NOT normal behaviour of process initialisation and can be categorised as an EDR behaviour, and it is likely to be that of the userland DLL has given to the caller.

These permission changes were not limited to NTDLL/Kernel32 but included various DLLs such as win32u or GDI32. An initial assumption would be that the endpoint was installing userland hooks due to the region size of 0x8 (unconfirmed).

win32u.dll              0x00007FFFD7DD0000

caller=0
PreviousMode=UserMode
BaseAddress=0x7fffd7dd1dc0 (win32u.dll .text)
RegionSize=0x8 (PAGE_WRITECOPY)
NewProtect=0x40 (PAGE_EXECUTE_READWRITE)
OldProtect=0
result=STATUS_SUCCESS

caller=0
PreviousMode=UserMode
BaseAddress=0x7fffd7dd1dc0 (win32u.dll .text)
RegionSize=0x8
NewProtect=0x80 (PAGE_EXECUTE_WRITECOPY)
OldProtect=0x40 (PAGE_EXECUTE_READWRITE)
result=STATUS_SUCCESS

Given that the request originates from user mode, this is likely the injected EDR DLL, a key focus of our later research.

The hypothesis we defined above is most likely true; however, without further analysis, we would be unable to determine the statement's validity. 

Another interesting target includes NtSetInformationProcess. Let us modify our script to print the corresponding caller, PreviousMode, ProcessInformationClass, ProcessInformation, ProcessInformationSize, and result.  

Changing the lines

select = ( ["caller", "PreviousMode", "ProcessInformationClass", "ProcessInformation", "ProcessInformationSize", "result"] )

and

if token == "NtSetInformationProcessCALL":          

The biased process calls ProcessFaultInformation from both Kernel and Userland; the author has not yet understood the reasoning behind this; therefore table is open to discussion. A more interesting class of process information class observed is InstrumentationCallback. 

caller=0xfffff80471ecb178

PreviousMode=KernelMode

ProcessInformationClass=ProcessInstrumentationCallback

ProcessInformation=0xffffdf8d98ef6c50

ProcessInformationSize=0x10

result=STATUS_SUCCESS

caller=0
PreviousMode=KernelMode
ProcessInformationClass=ProcessFaultInformation
ProcessInformation=0xfbe8f0def0
ProcessInformationSize=0x8
result=STATUS_SUCCESS

A good source for the reader to acquire the required knowledge regarding Instrumentation Callbacks is available, and it is trivial to bypass. Instrumentation callbacks are a post-syscall hook invoked before returning execution to the calling thread; these callbacks allow EDRs to determine if the syscall was direct. 

We cannot debug the captured syscall without a kernel debugger (kd.exe) setup; ProcessInformation is 0xffffdf8d98ef6c50, a kernel address (await the following blog post). However, as we know of its existence, evading it is trivial.

This short blog post covered the author's initial approach to using DTrace, avoiding manual analysis (though partially done); the next series will cover more automated analysis (building out our scripts) and utilising DTrace to aid our kernel debugging.

The author would like to clarify that none of this would be possible without @JonasLyk and his terrific work that left the author flabbergasted.