Address Space Layout Randomisation (ASLR) and Data Execution Protection (DEP) Part 1

I guess this topic, could be Security related but also Windows Internals related. Nonetheless, I hope you find this to be a interesting topic. I always believe you shouldn't just accept that something works the way it does, or what a concept is, I always find it's best to understand how it works and why it works that way.

 ASLR was mostly implemented for security purposes, it generally works, as the name implies, but randomizing the address space of a process, making it more difficult for attackers to predict where a function return address may be or a local variable to be exploited. It is usually implemented with Data Execution Protection (DEP) which I will also explain in this blog post.

There are a few different kinds of randomization: Stack, Heap and Image.The memory load addresses of these key structures are random and calculated upon a algorithm. Furthermore, the PEB and TEB may also be random if ASLR has been enabled. By default, ASLR is enabled for processes running on Windows Vista and onwards.

Viewing ASLR and DEP Processes/DLLs with Process Explorer

We can view processes which have DEP and ASLR enabled, by using Process Explorer. You will need to change the View Columns radio buttons to show DEP and ASLR.

You will then notice two additional columns in Process Explorer. In my example, I have chosen to show Firefox.

DEP Internals

Data Execution Protection causes any instructions executed when used within a page which is non-executable, then a ATTEMPTED_EXECUTE_OF_NOEXECUTE bugcheck will occur. Please be aware, when debugging, the above bugcheck does not necessarily mean malware, it could suggest poorly written code by third-party programs. In the User-Mode, a Access Violation exception will be raised instead.

If a page needs to be executable, then the PAGE_EXECUTE, PAGE_EXECUTE_READ, PAGE_EXECUTE_READWRITE and PAGE_EXECUTE_WRITECOPY.

PAGE_EXECUTE - Enables committed pages to be executable.

PAGE_EXECUTE_READ - Enables committed pages to be read and executable.

PAGE_EXECUTE_READWRITE - Enables committed pages to be read, written and executed.

PAGE_EXECUTE_WRITECOPY - Enables file-mapped files to be executed, written (private page being written to the process), read and write-on-copy.

Full Listing - Memory Protection Constants

You may wish to check the VirtualProtect, VirtualProtectEx, VirtualQuery and VirtualQueryEx.

There is two forms of Data Execution Protection: Software and Hardware. The Hardware option only applies to supported processors, and therefore I will continue with the Software aspect first. In regards, to the software implementation of Data Protection Execution, there is Stack Cookies and Pointer Encoding.

Let's take a look at the Stack Cookies method first. Stack Cookies are special code which is inserted at the beginning and the ending of a potentially exploitable function call. The important point to remember, and I should have mentioned this in the introduction paragraphs, is that ASLR and DEP were designed to protect from Buffer Overflows or Stack Smashes.

Stack Cookies are implemented by default to programs which meet the criteria. It will be shown with the /GS (Buffer Security Check) compiler switch. It can also be enabled by using the BufferSecurityCheck property. A Stack Cookie is s special value stored in the _KUSER_SHARED_DATA data structure, which is compared against before the function returns. If the Stack Cookie value has been modified, for example malware, then the program would crash.

There has been methods of bypassing the Stack Cookies protection. There's quite a few whitepapers, which I'll upload to my SkyDrive account. 

The second point is the the Pointer Encoding method, which uses Stack Cookie to encode and decode pointers, this can be seen with the EncodeSystemPointer and DecodeSystemPointer APIs. Any unencoded pointers will attempt to be decoded with the a cookie value, and this where the program will crash since the cookie value will become corrupt. 

If you attempt to call the function through the function pointer, while the function is still encoded, then a Access Violation will result and the program will crash.

Using OllyDbg, which can see where the Stack Cookie is used with the Pointer Encoding:

Our next point, is to discuss DEP in terms of Hardware. Hardware DEP only applies if the NX bit is supported, as well as, PAE or Physical Address Extension. You may wish to use my PAE Checker program from my blog post discussing PAE.




For x86 systems, the processor must support the PAE and have this bit set, if PTEs are to be marked as non-executable. You can check the CR4 register for the bit setting. For more details, please refer to my post here.

By default, the Boot Configuration Data (BCD) has been set to enable PAE, and thus enable DEP for all Windows processes. On x86 systems, the execution protection only applies to thread stacks and user-mode pages. Furthermore, you can enable Windows to enable DEP for certain programs only, by going to the Advanced System Properties and Performance settings.

For x64 systems, execution protection applies to all 64-bit processes and device drivers, thread stacks, kernel paged pool, user-mode pages and kernel session pool. The BCD nx option will need to be set to AlwaysOff, if you wish to disable Hardware DEP.

The NX bit stands for Non-Execute, and will cause the operating system to mark certain areas of memory as non-executable. This is strictly related to Hardware DEP. We can check if the processor supports the NX bit by using the CoreInfo program as shown below.

The second part of this blog post will take a look at ASLR, and changes in Windows 8/8.1.