I apologize for not updating my blog very regularly this month (it's been my birthday too), since I haven't been able to find any dump files which were interesting or useful to write about. I've wanted to post this particular example a few days ago, but couldn't find the thread again until now.
The example I wanted to write about was a Stop 0xE3, but since the post within the thread is already "blog material", I thought I might as well just post the link in this blog post.
BSOD dxgmms1.sys randomly
Tuesday, 30 July 2013
Wednesday, 24 July 2013
Exactly Why Are Page Faults Illegal At IRQL Level 2?
You may notice with Stop 0xA, Stop 0xD1 and Stop 0xC5, there always seems to be a illegal page fault or some other illegal instruction, although, page faults seem to be the most common.
To answer the question in the title, the answer is the the scheduler is turned off, therefore only non-paged pool can be accessed, since accessing memory which isn't available would lead to a page fault, which would then result in the thread being placed in a wait state with a context switch to a different thread being used, whilst the thread waited for the Memory Manager to process the page fault and make the memory address available. By changing to the contexts of different threads, then important data structures which are being synchronized at IRQL Level 2, would then need to be resynchronized.
Example - http://www.sevenforums.com/bsod-help-support/297605-bsod-when-playing-anything-0x000000d1-3.html#post2468537
To answer the question in the title, the answer is the the scheduler is turned off, therefore only non-paged pool can be accessed, since accessing memory which isn't available would lead to a page fault, which would then result in the thread being placed in a wait state with a context switch to a different thread being used, whilst the thread waited for the Memory Manager to process the page fault and make the memory address available. By changing to the contexts of different threads, then important data structures which are being synchronized at IRQL Level 2, would then need to be resynchronized.
Example - http://www.sevenforums.com/bsod-help-support/297605-bsod-when-playing-anything-0x000000d1-3.html#post2468537
Thursday, 18 July 2013
Debugging a Stop 0x9F Even Further - Using !drvobj
We're back with another Stop 0x9F to debug, in my opinion, these bugchecks are usually the easiest bugchecks to debug, since they usually point out the driver with the !irp extension, but sometimes the !irp extension may not reveal the true cause.
The two parameters we're interested in are, parameter 2 and parameter 4. The second parameter specifies the physical device which the associated problematic driver belongs to, and the fourth parameter specifies the IRP which seems to be blocking any further progress.
If we can examine the IRP, we can from the stack, that the driver which seems to be causing the problem is a Microsoft USB Hub driver, but from my experience, Microsoft driver are rarely the true cause for the crash.
So, let's debug deeper into the dump file (Minidump), and see which drivers are associated with the device. We can the !drvobj extension to view the associated drivers which control the device.
We can now see all the associated Dispatch routines for that device, and the drivers which issue these routines, we can at the bottom a third-party driver which is related to theVMware USB Monitor driver, which seems to be potentially outdated. The driver's associated dispatch routine is IRP_MJ_PNP, this major function code is used to request when PnP activity should occur.
So, let's debug deeper into the dump file (Minidump), and see which drivers are associated with the device. We can the !drvobj extension to view the associated drivers which control the device.
Monday, 15 July 2013
Debugging a Stop 0x9F - Power Transistion Time Out
I've previously explained how to debug a Stop 0x9F with a blocked IRP, although, there is a different situation when the !irp, !devobj and !devstack extensions are not viable option for debugging. This is because the cause of the crash is slightly different.
From the parameters provided by the bugcheck, we get a general idea of the problem, and what happened, basically a thread was holding a lock, which wasn't released, causing the synchronization to time out.
From my understanding, the lock was never released by the thread, since it's wait wasn't satisfied, it wasn't able to obtain the objects it was waiting for.
We can view which objects the locked thread was waiting for, with the !thread extension and parameter 3.
We can see that the thread is waiting for three Event objects, which are all Notification Events,
the thread is waiting for the completion of a certain event of procedure, once this has become complete, the Event objects will switch from Non-Signaled to Signaled, and will release the thread(s) from it's waiting state. Once a thread has left it's wait state, then it can resume it's normal operations.
From the call stack, there is further evidence to support that thread holding the lock is waiting for multiple Event objects: nt!KiCommitThreadWait and nt!KeWaitForMultipleObjects.
Events can be exposed to Device drivers, so let's check which drivers may be causing the thread to wait. I dumped the raw stack of the current thread we were viewing, and found a very outdated ATI graphics card driver.
Two useful data structures you could use in a situation like this are: nt!_DISPATCHER_HEADER and nt!_KOBJECTS. The type field of the Dispatcher data structure will correspond to the a value within the Objects data structure, which in turn will indicate the type of object the thread is waiting for.
References:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff553202%28v=vs.85%29.aspx
From my understanding, the lock was never released by the thread, since it's wait wasn't satisfied, it wasn't able to obtain the objects it was waiting for.
We can view which objects the locked thread was waiting for, with the !thread extension and parameter 3.
We can see that the thread is waiting for three Event objects, which are all Notification Events,
the thread is waiting for the completion of a certain event of procedure, once this has become complete, the Event objects will switch from Non-Signaled to Signaled, and will release the thread(s) from it's waiting state. Once a thread has left it's wait state, then it can resume it's normal operations.
From the call stack, there is further evidence to support that thread holding the lock is waiting for multiple Event objects: nt!KiCommitThreadWait and nt!KeWaitForMultipleObjects.
Events can be exposed to Device drivers, so let's check which drivers may be causing the thread to wait. I dumped the raw stack of the current thread we were viewing, and found a very outdated ATI graphics card driver.
References:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff553202%28v=vs.85%29.aspx
Debugging Counterfeit Installations - Why I Don't Bother With Them
Note: This article is my opinion, and isn't intended to offend anyone [Point 3]
Although this blog post, is not exactly a tutorial or a link to a tutorial, I would like to explain why I don't even consider debugging a counterfeit installation and why you shouldn't either.
Effectively, most counterfeit installations take advantage of some exploit or apply some hack to bypass the activation requirements for Windows, this may explain to some of you, how you might notice a OEM_SLP key on a retail motherboard.
I'm assuming you know how to tell the difference between a counterfeit and geniune installation, therefore I'm just going to describe some of my reasons for not considering counterfeit installs to be worth debugging.
Firstly, counterfeit installations have key Windows files altered, in order to make the installation work, although, this can lead to most people having Windows Update difficulties. If the installation is not within it's original state, then it could give very misleading BSODs or crashes. For example, Stop 0x124's are handled by WHEA, part of the operating system, a corrupt driver could be the cause, but instead the cause is given to some hardware component, such as a data read from the Level 2 cache.
Secondly, counterfeit installations have many vulnerabilities and stability issues, the installation could easily be at fault and cause of the all the problems, if it can't handle basic tasks or procedures required by device drivers. The installation is most likely to be infected with some malicious software or code, causing further instability, if you able to resolve instablity at this moment in time, the same problem is very likely to repeat itself in the future.
Thirdly, and this may be slightly discriminatory, but from my experience, the user with the counterfeit installation, is usually very rude, impatient and wants a "quick fix" for their problems. The users I have had, mostly feel they are "smarter" than you, and will discount the installation as being the problem, and will begin to become abusive or impatient.
Most forums will not even accept counterfeit installations. Some users are completely unaware of their counterfeit installation, and will often be thankful for pointing it out to them. If you are victim of a having been sold a counterfeit installation, and can prove you were completely unaware, then I believe Microsoft can help you with it.
Although this blog post, is not exactly a tutorial or a link to a tutorial, I would like to explain why I don't even consider debugging a counterfeit installation and why you shouldn't either.
Effectively, most counterfeit installations take advantage of some exploit or apply some hack to bypass the activation requirements for Windows, this may explain to some of you, how you might notice a OEM_SLP key on a retail motherboard.
I'm assuming you know how to tell the difference between a counterfeit and geniune installation, therefore I'm just going to describe some of my reasons for not considering counterfeit installs to be worth debugging.
Firstly, counterfeit installations have key Windows files altered, in order to make the installation work, although, this can lead to most people having Windows Update difficulties. If the installation is not within it's original state, then it could give very misleading BSODs or crashes. For example, Stop 0x124's are handled by WHEA, part of the operating system, a corrupt driver could be the cause, but instead the cause is given to some hardware component, such as a data read from the Level 2 cache.
Secondly, counterfeit installations have many vulnerabilities and stability issues, the installation could easily be at fault and cause of the all the problems, if it can't handle basic tasks or procedures required by device drivers. The installation is most likely to be infected with some malicious software or code, causing further instability, if you able to resolve instablity at this moment in time, the same problem is very likely to repeat itself in the future.
Thirdly, and this may be slightly discriminatory, but from my experience, the user with the counterfeit installation, is usually very rude, impatient and wants a "quick fix" for their problems. The users I have had, mostly feel they are "smarter" than you, and will discount the installation as being the problem, and will begin to become abusive or impatient.
Most forums will not even accept counterfeit installations. Some users are completely unaware of their counterfeit installation, and will often be thankful for pointing it out to them. If you are victim of a having been sold a counterfeit installation, and can prove you were completely unaware, then I believe Microsoft can help you with it.
Friday, 12 July 2013
Basics of the Windows Driver Model
According to the Windows Driver Model, there are three different kinds of drivers, which will operate within the driver stacks etc. These types of drivers are: Bus Drivers; Filter Drivers and Function Drivers.
The bus drivers are typically already provided by Microsoft, they are as the name suggests, used to communicate with the various different buses on the motherboard, for instance the pci.sys for PCI/PCIe devices.
The function drivers, provide the operational side of things for the device, and is used to manipulate the device and it's operations. The driver will handle tasks, such as reads/writes and power management states for the device(s).
The filter drivers are used to add greater functionality to a device, and are used to modify I/O requests and respond to other device driver responses, as an example from the Windows Internals book, they can be used to modify any mistakes made by the hardware regarding the amount of required resources.
The bus drivers are typically already provided by Microsoft, they are as the name suggests, used to communicate with the various different buses on the motherboard, for instance the pci.sys for PCI/PCIe devices.
The function drivers, provide the operational side of things for the device, and is used to manipulate the device and it's operations. The driver will handle tasks, such as reads/writes and power management states for the device(s).
The filter drivers are used to add greater functionality to a device, and are used to modify I/O requests and respond to other device driver responses, as an example from the Windows Internals book, they can be used to modify any mistakes made by the hardware regarding the amount of required resources.
Monday, 8 July 2013
Debugging Stop 0x1A - MEMORY_MANAGEMENT
Thanks muhahaa for pointing the PFN data structure out to me.
A Stop 0x1A is quite common in my opinion, and usually is caused by two things: device drivers and RAM. Although, other causes can include the Windows Kernel and memory located in other areas.
Firstly, looking at the parameters presented to us by WinDbg, we can see that the first parameter usually indicates the type of memory management violation which has occurred.
From this example, 41790 indicates that a page table page has become corrupt. The page table is used to store mappings between physical and virtual memory addresses.
We can see from the call stack for the thread, that some virtual memory related routines were being called by the Windows Kernel, for instance the nt!NtUnmapViewOfSection is used to view the unmap the view (mapping of a section object in virtual memory for a process) from the virtual memory address space for a process.
The interesting thing about this dump file is the third parameter, which is 0xffff, this is very similar value when a device driver has called MmUnlockPages too many times on a MDL, which causes the reference count for the number of PFN entries to drop below zero.
We can view the data structure for a PFN database and check the reference count:
We can see that the number of Used Page Table Entries reference count has dropped below zero, here my suggestion would be to use Driver Verifier along with the Special Pool option.
A Stop 0x1A is quite common in my opinion, and usually is caused by two things: device drivers and RAM. Although, other causes can include the Windows Kernel and memory located in other areas.
Firstly, looking at the parameters presented to us by WinDbg, we can see that the first parameter usually indicates the type of memory management violation which has occurred.
From this example, 41790 indicates that a page table page has become corrupt. The page table is used to store mappings between physical and virtual memory addresses.
We can see from the call stack for the thread, that some virtual memory related routines were being called by the Windows Kernel, for instance the nt!NtUnmapViewOfSection is used to view the unmap the view (mapping of a section object in virtual memory for a process) from the virtual memory address space for a process.
The interesting thing about this dump file is the third parameter, which is 0xffff, this is very similar value when a device driver has called MmUnlockPages too many times on a MDL, which causes the reference count for the number of PFN entries to drop below zero.
We can view the data structure for a PFN database and check the reference count:
We can see that the number of Used Page Table Entries reference count has dropped below zero, here my suggestion would be to use Driver Verifier along with the Special Pool option.
Subscribe to:
Posts (Atom)