SequentialRead: Tool for investigating Windows Cache Manager Features

Version 1.2 Oct 2013

This tool was written to investigate how the Windows Cache Manager and Disk I/O subsystems precache file data leading to improved performance when file I/O patterns are sequential. It is also useful as a test tool to produce reliable, known, file read patterns.

As explained my the book “Windows System Performance Through Caching”, the System Cache Manager of the Microsoft Windows operating systems detect read patterns of files, looking for File I/O that looks to be sequential. The sequential patterns may be in for forward direction (typical when reading a file from start to finish), or reverse direction (typical when reading a log file). When last documented, it should kick in when the third consecutive read occurs.

When detected, the System Cache Manager inside the kernel of the OS will request a pre-read of an extra buffer in anticipation of the application requesting it.

While additional pre-reading or caching may occur in the Disk subsystem, such help may be spotty, depending on either unfragmented files or recent file use.

This tool reads data in either sequential or random order and reports timing and other information obtained from the OS about the activity.

This information may be used to better understand how the System Cache Manager operates.

Usage

SequentialRead is a GUI tool. It consists of a single exe file and has no installer.

When you lauch the tool you select a file to read and the type of I/O patterned read you would like to use. A large file is recommended to be able to see the results. You should also disable the SuperFetch service and clear out both the standby lists and the system file cache (The TMurgent ATM tool can clear these lists and the cache). Then click the load button.

The "Diff" field in the display output represents the number of times that the System Cache Manager performed read ahead operations. Keep in mind that other file activity is included in the single system-wide counter. The images below show the results of three different read paterns.

Even Random I/O sometimes manages to hit a pattern.

Notice the number of ReadAhead I/Os and the difference in time. Some of that performance boost is due to the readaheads.

Even with the ReadAheads, a reverse read still takes significantly longer. The reason for this is that with a rotating media disk, the disk controller will often cache the "next" sector in it's RAM buffer as the disk read head passes over that sector while waiting for the next command. With an unfragmented file, this would be the next sector in a "forward" direction. Even with "random" access, the disk controller RAM buffer might hold onto that sector long enough to be used. With an unfragmented file and reverse reading, that would never happen since the following sector would have already been read.

So, at least with rotating media that is locally attached, a good disk controller beats this optimization every day. The advantages of the System cache pre-read is more obvious when there is increased latency between the CPU and the disk controller, such as in remote disk I/O, NAS, and SAN scenarios. Flash based drives, which have no rotating head, are another scenario where the System cache read ahead is more significant. Unfortunately, Microsoft only makes the effect of the feature available through the readahead counter; if they allowed us to disable the feature we could then demonstrate the advantages of it.

What does SequentialRead cost?

TMurgent believes in giving back to the community.   SequentialRead is free for for use by all.

What OSs will it run on?

The tool will run on Windows XP and above. There is only a 32-bit version of the tool, but it works equally well on x64 systems.

How do I install these tools?

No installer. Just download, unzip, read the readme, and copy the exe somewhere convenient.

Are there more free tools?

TMurgent has a bunch of tools on this page. Usually they are free, unless you are a Software Vendor. Tools for performance have a segregated list here.

Where Can I download the tools?

Here is a link to the ZIP package:    SequentialRead.zip Version 1.2 (size 1MB )

Where Can I buy the book to learn more?

The book, “Windows System Performance Through Caching”, will be published through Lulu, and you can find it from this page when published.