How the system Finds and Loads Drivers

How the system Finds and Loads Drivers


1. PnP device

It has an electronic signature that bus driver can detect the new hardware

2. Legacy device

Initiate the detection by invoking Add New Hardware Wizard.


In the end, in both PnP and Legacy Device situation, system uses the same automatic registry and INF file process to load the right driver.



Function driver:

It understands the details about how to make the HW work.

It's responsible for initiating IO operations, for handling interrupts, and for provding a way for end user to exersise control over the device.



Bus driver:

It's responsible for managing the connection between the HW and the computer.



Filter Driver:

to Modify the behavior of an existing function driver in some way.Upper filter driver sees IRPs before function driver. Lower filter can modify the stream of bus operations that the function driver is trying to perform.



PDO stands for physical device object. The bus driver uses it to represent the conncetion between the device and the bus.

FDO stands for function devive object. The function driver uses it to manage the functionality of the device.



Installing a PnP Device:

1.Bus driver detects the insertion or removal of hardware.

2.Bus driver calls IoInvalidateDeviceRelations to notify PnP manager that the bus's population of child devices has changed.

3.PnP manager sends IRP_MN_QUERY_DEVICE_RELATIONS to bus driver, to obtain an updated list of the PDOs for the child devices.

4.PnP manager sends IRP_MN_QUERY_ID to bus driver, to obtain Device ID

5.PnP manager locates INF install section and initialize driver. Memory Manager then calls DriverEntry routine.

6.PnP manager calls AddDevice, to inform driver that a new instance of the device hs been discovered

7.PnP manager sends IRP_MN_QUERY_RESOURCE_REQUIREMENTS to bus driver, to ask bus driver report resource requirements.

8.PnP manager configuire the hardware with as set of resource arbitrators to assign resources to the device.

8.PnP manager sends IRP_MN_START_DEVICE to your driver(the driver stack). Function driver handles thei IRP by configuring and connecting various kernel resources.


Order of AddDevice Calls (Driver Loading):

5. Class upper filters

4. Device upper filters

3. function driver

2. Class lower filters

1. Device lower filters
 
 
IRP Routing:
System sends an IRP to the topmost filter dirver in the stack, that driver can decide to process the IRP, to pass the IRP down to the next level, or to do both. Each driver that sees the IRP makes the same decision.

MDL

http://www.osronline.com/article.cfm?id=423

An MDL is a structure that describes the fixed physical memory locations that comprise a contiguous data buffer in virtual memory.
Each MDL can only describe a single virtually contiguous data buffer.
The data buffer that the MDL describes can be in either a kernel virtual address space, user virtual address space, or both.
The MDL describes the data buffer at a fixed position in physical memory. In other words, the data buffer an MDL describes will always be paged in (resident), and its pages will be locked-down ("pinned"). This means the data buffer can neither be paged out nor moved. These pages will remain locked for the lifetime of the MDL.
The data buffer that the MDL describes does not need to be page aligned, nor does it need to be an integral number of pages in length.

//
// An MDL describes pages in a virtual buffer in terms
// of physical pages. The pages associated with the
// buffer are described in an array that is allocated
// just after the MDL header structure itself.
//
// One simply calculates the base of the array by
// adding one to the base MDL pointer:
//
// Pages = (PPFN_NUMBER) (Mdl + 1);
//
// Notice that while in the context of the subject
// thread, the base virtual address of a buffer mapped
// by an MDL may be referenced using the following:
//
//         Mdl->StartVa | Mdl->ByteOffset
//
typedef struct _MDL {
struct _MDL *Next;
CSHORT Size;
CSHORT MdlFlags;
struct _EPROCESS *Process;
PVOID MappedSystemVa;
PVOID StartVa;
ULONG ByteCount;
ULONG ByteOffset;
} MDL, *PMDL;

From the comment you know the MDL is actually a variable length structure with an array that contains the underlying physical pages of the address range at the tail. Also you know that StartVa is the page aligned virtual address of the start of the range and ByteOffset is the starting offset into the first page.



Building and using MDLs.

Step1: Allocating the Structure  -- use the IoAllocateMdl DDI
Step2:Probing and Locking  --- MmProbeAndLockPages DDI
Step3: Mapping 
your driver wants a new virtual address for accessing the underlying pages  ---MmMapLockedPagesSpecifyCache

you want a kernel virtual address for this MDL---MmGetSystemAddressForMdlSafe macro

Win7: Memory Dump

A kernel dump gets created every time a machine has a kernel fault. This dump gets stored in the %systemroot% folder as a memory.dmp file.
(Control Panel > System and Security > System > Advanced system settings > Advanced tab > Startup and Recovery Settings.)

1. No Full Memory Dumps option?
   The I/O Manager originally limited the size of a complete memory dump to 2GB, now the fact is Windows support crash dumps larger than 2GB. To workaround this, set \HKLM\System\CCS\Control\CrashControl\CrashDumpEnabled DWORD value to 1
2. No memory.dmp generated after crash?
   Algorithm to decide whether to store/delete memory.dmp file generated after a system crash:
   First report the kernel fault to the Online Crash Analysis Service.
   Then, if machine has a registry setting AlwaysKeepMemoryDump set to 1, store the dump file on disk.
   Else, if machine is a Windows Server SKU, store the dump file on disk.
   Else, if the machine is joined to a domain (i.e. this is a corporate machine), store the dump file on disk.
   Else, if machine is not on a domain (i.e. this is a home user’s machine),
   If free disk space is >= 25GB, store the dump file on disk.
   Else (free disk space is < 25 GB), delete the dump file.
   The exact location of the AlwaysKeepMemoryDump setting in the registry is:
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl ! AlwaysKeepMemoryDump Type: REG_DWORD.

Reference:
http://blogs.msdn.com/b/wer/archive/2009/02/09/kernel-dump-storage-and-clean-up-behavior-in-windows-7.aspx

Make a partition on a RAW USB flash disk.

Briefly, here are the steps:
1. Use CreateFile to open PhysicalDriveX, that is the USB Flash disk
2. IOCTL_DISK_GET_DRIVE_GEOMETRY_EX to get the physical disk's geometry ( we need some information in it to fill partition data)
3. IOCTL_DISK_CREATE_DISK to creates an empty partition.
4. IOCTL_DISK_SET_DRIVE_LAYOUT_EX to repartition a disk as specified.
Note: use IOCTL_DISK_UPDATE_PROPERTIES to synchronize system view after IOCTL_DISK_CREATE_DISK and IOCTL_DISK_SET_DRIVE_LAYOUT_EX.
e.g. a 1GB size flash disk,
pBuf->PartitionEntry[0].PartitionStyle = PARTITION_STYLE_MBR;
pBuf->PartitionEntry[0].StartingOffset.QuadPart = 0x10000;//64KB
pBuf->PartitionEntry[0].PartitionLength.QuadPart = 0x1e200000; //Donot exceed DiskGeo.DiskSize
pBuf->PartitionEntry[0].PartitionNumber = 1;
pBuf->PartitionEntry[0].RewritePartition = TRUE;
pBuf->PartitionEntry[0].Mbr.PartitionType = PARTITION_FAT32;
pBuf->PartitionEntry[0].Mbr.BootIndicator = TRUE;
pBuf->PartitionEntry[0].Mbr.RecognizedPartition = TRUE;
pBuf->PartitionEntry[0].Mbr.HiddenSectors = 0x80; //StartingOffset/(DiskGeo.BytesPerSector) = 0x10000/0x200

Removal Policy for a Device

http://download.microsoft.com/download/1/6/1/161ba512-40e2-4cc9-843a-923143f3456c/Designing%20Hardware%20for%20Surprise%20Removal_v8.doc

http://msdn.microsoft.com/en-us/library/ff551967(VS.85).aspx


Removal Scenarios for Devices
1.Orderly-Removal Scenarios
2.Surprise-Removal Scenarios

Removal-related device capabilities that are relevant to this include SurpriseRemovalOK and Removable.
1.Removable. This capability is set by the bus driver for the bus on which a given device is installed.
2.SurpriseRemovalOK. This capability is set by the driver for the device. It specifies whether the device can be safely removed without using the Safely Remove Hardware applet.
3.If both Removable and SurpriseRemovalOK are set, the device can be removed without using the applet. If Removable is TRUE, the device is displayed in the Safely Remove Hardware applet unless SurpriseRemovalOK is also set to TRUE.

Windows XP has the following removal policy options:
1.ExpectNoRemoval.
2.ExpectOrderlyRemoval
3.ExpectSurpriseRemoval


The operating system uses the following algorithm to determine removal policy defaults:
1.The bus driver sets the Removable device capability according to the operating system default for the device. ACPI BIOS can override the Removable capability by using the _RMV method.
2.If the device is determined to be removable, the operating system checks to see if there is an INF override for removal policy.
3.If no INF override is provided, Windows checks IEEE 1394 devices for any override value in the configuration ROM.
4.In the absence of either an INF or hardware override, Windows bases the removal policy on the bus of the removable device. USB,IEEE1394,PCMCIA---ExpectSurpriseRemoval. Others---ExpectOrderlyRemoval

Override Removal Policy in a Device INF File
[DDInstall.NT] <-- Replace DDInstall with the name of the install section for the device ; Contains CopyFiles and other directives

[DDInstall.NT.Services]

; Contains AddService directive to install driver

[DDInstall.NT.HW]

; Contains AddReg/DelReg sections to manage hardware-specific registry settings

AddReg=DDInstall_AddReg_HW_Removal_Policy

[DDInstall_AddReg_HW_Removal_Policy]

HKR,,"RemovalPolicy",0x00010001,Policy <--- See legend below for Policy values

The Policy value can be one of the following:
• Policy = 2 - ExpectOrderlyRemoval. The user will rarely remove the device, or will initiate removal only through software or hardware.
• Policy = 3 - ExpectSurpriseRemoval. The user will regularly remove the device with no warning to the operating system.

Override Removal Policy in a Device INF File (
SetupDiGetDeviceRegistryProperty
SetupDiSetDeviceRegistryProperty (SPDRP_REMOVAL_POLICY is reserved for use by the operating system and cannot be used in the Property parameter)

SPDRP_REMOVAL_POLICY
(Windows XP and later) The function retrieves the device's current removal policy as a DWORD that contains one of the CM_REMOVAL_POLICY_Xxx values that are defined in Cfgmgr32.h.

SPDRP_REMOVAL_POLICY_HW_DEFAULT
(Windows XP and later) The function retrieves the device's hardware-specified default removal policy as a DWORD that contains one of the CM_REMOVAL_POLICY_Xxx values that are defined in Cfgmgr32.h.

SPDRP_REMOVAL_POLICY_OVERRIDE
(Windows XP and later) The function retrieves the device's override removal policy (if it exists) from the registry, as a DWORD that contains one of the CM_REMOVAL_POLICY_Xxx values that are defined in Cfgmgr32.h.

How Can I Open a Text File as Unicode?

http://www.microsoft.com/technet/scriptcenter/resources/qanda/apr06/hey0419.mspx
How Can I Open a Text File as Unicode?
with that in mind here’s a script that will open the Unicode file and correctly echo back the contents:

Const ForReading = 1
Const TriStateTrue = -1

Set objFSO = CreateObject("Scripting.FileSystemObject")

Set objFile = objFSO.OpenTextFile("c:\scripts\test.txt", ForReading,False,TriStateTrue)

strText = objFile.ReadAll
objFile.Close

Wscript.Echo strText

Windows Auto-Sleep

1. Application Power Management Best Practices for Windows Vista
http://www.microsoft.com/whdc/system/pnppwr/powermgmt/PM_apps.mspx
Sleep Idle Timeout
If a system has been idle longer than the sleep idle timeout in the power policy, Windows automatically places the system in the sleep state. Windows uses both user input and processor usage to determine if a system is idle and tracks this information over time.
Generally, applications must prevent the system from entering the sleep state while applications are performing a background task or undertaking an operation that the user expects the system to continue until the operation is complete. Applications might need to prevent the sleep idle timeout in the following scenarios:
· A media application that displays full-screen content or visualizations
· Background applications such as hard disk defragmentation or antivirus utilities while a system scan is in progress
· A media application that records television content
· An application or service that streams content to other devices on the network
· An application such as a Web browser while downloading large files from the internet or network
· Applications that synchronize content to another device or computer

When preventing the sleep idle timeout in their applications, developers must exercise caution because the user expects the system to idle to sleep if it is enabled in power policy. Therefore, applications should prevent the sleep idle timeout only when absolutely necessary.


2. System Sleep Criteria: System Sleep Criteria:
http://msdn.microsoft.com/en-us/library/aa373233%20(VS.85).aspx
If a system has been idle longer than the sleep idle timeout in the power policy, Windows automatically places the system in the sleep state. Windows uses both user input and processor usage to determine if a system is idle and tracks this information over time.


3. Windows Auto-sleepAuto-sleep is supposed to happen during periods of inactivity. It works with system idle timer.As long as the system determines that there is user or application activity, it will not enter sleep. The system can detect certain activities, such as user input or network communications. Windows takes consider many things to determine system is idle(inactive), including
· There is no user input.
· CPU and disk utilization levels.
· The system is not running on battery power.
· No presentation program, such as a slide show or a movie player, is running.
· Things going on in the background
· If you are connected to a broadband connection, that can be picked up as possible network activity
· Audio channel not open
· Modem device not in use
· Others

Safely Remove Hardware

1. Invoke the Safely Remove Hardware Dialog:
RunDll32.exe shell32.dll,Control_RunDLL hotplug.dll
2. First issue CM_Query_And_Remove_SubTree on the device node, and then follow up with CM_Request_Device_Eject on the device node. Note: use the device node for the USB storage device enumerated by the USB hub, not the volume device enumerated via USBSTOR.

Approach overview1. Open the device via the drive letter (\\.\d:) and issue a IOCTL_MOUNTDEV_QUERY_UNIQUE_ID to obtain the unique ID for the device.2. Use the SetupDi calls to enumerate all mounted devices. Open each device (via the interfaces) and issue IOCTL_MOUNTDEV_QUERY_UNIQUE_ID calls to find the mounted device that has a matching unique ID.3. Use the SetupDi calls to obtain the device node ID for the matching device. This can then be used by the CM_ calls to initiate a safe removal operation.Details1. Use CreateFile to open the drive ("\\.\d:"). Use FILE_SHARE_READ FILE_SHARE_WRITE and GENERIC_READ.2. Issue IOCTL_MOUNTDEV_QUERY_UNIQUE_ID with an output buffer size of sizeof(MOUNTDEV_UNIQUE_ID). The request will fail, but will return a copy of the MOUNTDEV_UNIQUE_ID structure with the UniqueIdLength field filled in. Check the count of bytes returned (should be >= sizeof MOUNTDEV_UNIQUE_ID) to be sure that the required buffer size was returned.3. Allocate a new buffer of size sizeof(MOUNTDEV_UNIQUE_ID) plus the value of UniqueIdLength returned by the previous call.4. Issue IOCTL_MOUNTDEV_QUERY_UNIQUE_ID with the new output buffer and size. The call should succeed.5. Call SetupDiGetClassDevs( &MOUNTDEV_MOUNTED_DEVICE_GUID, NULL, NULL, DIGCF_DEVICEINTERFACE) to obtain a device info object representing mounted storage devices.6. Repeatedly call SetupDiEnumDeviceInterfaces( dev_info, NULL, &MOUNTDEV_MOUNTED_DEVICE_GUID, index, &interface_data) to examine each interface.a. Call SetupDiGetDeviceInterfaceDetail to obtain the interface detail, including a device path that can be passed to CreateFile.b. Use CreateFile to open the device (interface_detail->DevicePath).c. Issue IOCTL_MOUNTDEV_QUERY_UNIQUE_ID with an output buffer size of sizeof(MOUNTDEV_UNIQUE_ID). The request will fail, but will return a copy of the MOUNTDEV_UNIQUE_ID structure with the UniqueIdLength field filled in. Check the count of bytes returned (should be >= sizeof MOUNTDEV_UNIQUE_ID) to be sure that the required buffer size was returned.d. Allocate a new buffer of size sizeof(MOUNTDEV_UNIQUE_ID) plus the value of UniqueIdLength returned by the previous call.e. Issue IOCTL_MOUNTDEV_QUERY_UNIQUE_ID with the new output buffer and size. The call should succeed.f. Compare the unique ID of the device with the unique ID found in step 4. If they match, we found the SetupDi device instance equivalent of the drive letter.7. Call SetupDiGetDeviceInstanceId to obtain the device instance ID of the device, which can then be used in calls to the CM_ APIs.Additional items1. When initiating a safe removal for a USB storage device, the removal should be done for the root of the device, which generally seems to be of class "USB" (as opposed to USB store). The function CM_Get_DevNode_Registry_Property can be used to query the device class. If the device is not of class USB, then navigate to the parent using CM_Get_Parent. When a device instance of class "USB" is reached, that can be used as the target for a safe removal operation.2. Alternatively, rather than checking device classes, the app can check for known good hardware IDs or compatible IDs, and keep walking up the tree until it finds one it can work with.

Test an unsigned driver on Vista 64

The recommendation is test-sign your driver...
Installing an Unsigned Driver during Development and Test
http://msdn.microsoft.com/en-us/library/aa906338.aspx
By default, 64-bit versions of Windows Vista and later versions of Windows will load a kernel-mode driver only if the kernel can verify the driver signature. However, this default behavior can be disabled to facilitate early driver development and non-automated testing. Developers can use one of the following mechanisms to temporarily disable load-time enforcement of a valid driver signature. However, to fully automate testing of a driver that is installed by Plug and Play (PnP), the catalog file of the driver must be signed. Signing the driver is required because Windows Vista and later versions of Windows display a driver signing dialog box for unsigned drivers that require a system administrator to authorize the installation of the driver. This PnP driver installation behavior cannot be disabled on Windows Vista and later versions of Windows.

How to work-around to install and load unsigned driver on Vista64?
1. To install unsigned dirver.
gpedit.msc -> User Configuration -> Administrative Templates -> System -> Driver Installation
->Code signing for device drivers.
2. To disable load-time signature enforcement for a kernel mode driver.

• Use F8 Advanced Boot Option. This setting does not persist across system restarts

• Attach a Kernel Debugger to Disable Signature Verification

Test Unit Ready (TUR)

The SCSI Test Unit Ready command is used to determine if a device is ready to transfer data (read/write). The device will then return either good status or a check condition

SCSI communication takes place between an initiator and a target. The initiator sends a command to the target which then responds. SCSI commands are sent in a Command Descriptor Block (CDB). At the end of the command the target returns a Status Code byte which is usually 00h for success, 02h for a Check Condition (error), or 08h for busy.

When the target returns a Check Condition in response to a command, the initiator usually then issues a SCSI Request Sense command in order to obtain more information.

http://support.microsoft.com/kb/842411

To check the status of a backup device, Plug and Play sends a Test Unit Ready request to the device every second

How to remove phantom/ghost devices

• What is phantom devices

Hidden, inactive deives, ghost devices. When a device is physically removed from a machine, the driver becomes a phantom and is no longer visible in Device Manager. Normally this is desirable, but can be a problem if you wish to remove the device driver.

• How to identify and remove phantom devices in device manager (manually)?

do the following:
1. From the command prompt on the problem media server, run:C:\>set devmgr_show_nonpresent_devices=1C:\>start devmgmt.msc
2. Then, select View from the drop down and select to Show Hidden Devices.At this point, any ghost tape devices will be seen with lighter, transparent icon and can be removed. This is done by right-clicking the ghost tape device and selecting "Uninstall".

• How to do this programming...

devcon command works!

Find * FindAll * Remove "@hwid"

Here is a script works...

Get installed hotfixes

1. Here is a command: wmic qfe list full
2. On XP, go to the registry directly: 
   Instances of this class represent updates found in two places in the registry:

   • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\    CurrentVersion\Hotfix

   • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Updates

3. On Vista, the registry doesn't exist any more. Try the WMI Class: Win32_QuickFixEngineering Class http://msdn.microsoft.com/en-us/library/aa394391(VS.85).aspx
   

PAE, 4GT and AWE

PAE: Physical Address Extension
4GT: 4-gigabyte tuning
AWE: Address Windowing Extensions

• PAE allows the operating system to access and use more than 4 GB of physical memory. PAE also enables several advanced system and processor features, such as hardware-enabled Data Execution Prevention (DEP), Non-Uniform Memory Architecture (NUMA) and the ability to add memory to a system while it is running (hot-add memory), so it can also be used on computers that have less than 4 GB of memory. PAE is supported only on 32-bit versions of Windows; 64-bit versions of Windows do not support PAE.
• 4GT extends the 32-bit user virtual address space from 2 GB to up to 3 GB.
• AWE is a set of APIs that allows a process to allocate non-paged physical memory and then dynamically map portions of this memory into the virtual address space of the process.
• When neither 4GT nor AWE are being used, the amount of physical memory that a single 32-bit process can use is limited by the size of its address space (2 GB). In this case, a PAE-enabled system can still make use of more than 4 GB of RAM to run multiple processes at the same time or to cache file data in memory.
• 4GT can be used with or without PAE. However, some versions of Windows limit the maximum amount of physical memory that can be supported when 4GT is used. On such systems, booting with 4GT enabled causes the operating system to ignore any memory in excess of the limit. For details, see Memory Limits for Windows Releases.
• AWE does not require PAE or 4GT but is often used together with PAE to allocate more than 4 GB of physical memory from a single 32-bit process.