ATM: The Cache Machine!

ATM is a new tool designed to allow you to better explore the System File Cache use on your system. In addition to some monitoring functions, on newer OSs you may also modify your system cache behavior.

More importantly, on Vista, and the "Code named Longhorn" server, this tool allows you to investigate changes made to the system cache, especially with what Microsoft calls "SuperFetch" or "SuperCache".

Download links are at the bottom of this page.

What is the System File Cache?

The System File Cache is a component of the operating system kernel. It works in conjunction with the Memory Management sub-system and the IO sub-system to cache files, or more acuratly portions of file, in RAM. This important function significantly affects system performance by supporting sharing of commonly used files (such as system dlls) in memory, and avoids unnecessary disk IO.

What am I looking at?

ATM was designed for IT professionals, not your average user, so there is a lot stuff on the screen. But just start with the two graphs.

The upper graph provides a view of RAM (memory) usage. It is a historical graph showing the last 60 seconds with the most recent interval on the right. The two "green" categories reflect what is considered "Used Memory" and will match up with what the Task Manager reports. The dark green shows the ammount of that memory which is in the System File Cache Working Set. The other colors (Blue shades and possibly yellow) are the "available" memory. Not all available memory is empty, however. This includes things being written out to a disk file, or written to the page file, or sometimes parts of files not in use in the System File cache at present (no longer in the Cache Working Set).

The image above was taken from a Vista OS.  If there is yellow on your graph, you are on Vista or Longhorn.  On earlier operations systems, there are fewer colors (categories) shown.  On these older operating systems, a dark blue represents the portion of available memory that contains parts of files which were recently in the System File Cache but are no longer in use.  If requested again (soon), the OS can recover these file pages without reading them in from the disk again.

On Vista/Longhorn some things have changed and we have more categories of "available" memory to display. 

• Standby-core list.  This is pageable kernel memory that has been paged out recently.   If a page fault occurs for this memory, it is recovered via a soft page fault as long as it remains in this list, otherwise a hard page fault is required to recover from the page file.
• Standby-Normal Priority List.  This is primarily user process memory that has been paged out recently.   If a page fault occurs for this memory, it is recovered via a soft page fault as long as it remains in this list, otherwise a hard page fault is required to recover from the page file.
• Standby-Reserve List.  This is what Microsoft calls SuperFetch.  It is files read in by the System File Cache in anticipation of needing them soon.
• Free & Zero list.  These are memory blocks that do not have recoverable memory and will be used for any application or  kernel component requesting memory.

The bottom graph shows a historical view of cache activity (file system requests for files that  might be  cached),  and the  effectiveness of the cache in having what is needed on-hand.   A miss occurs only if the requested file portion  is not in the  working set or may be recovered from one of the standby pools.  These values are as reported by the OS.

The text on the left of the display provides greater detail and additional interesting things to look at.

What does ATM cost?

TMurgent believes in giving back to the community.   ATM is free for for use by consultants and enterprises. The exception to this is that we ask ISVs that want to use this tool in their labs to contact us for a license.

How does this tool work?

The tool makes use of published APIs, when applicable, but also uses some unpublished kernel interfaces to obtain information or request system modifications. The tool understands the differences between versions of the operating system and adjusts to those differences.

What OSs will it run on?

The tool will run on Windows 2000 through Vista and code-named Longhorn. At the time of this writing, the April Beta (Beta 3) is the latest Longhorn version tested. Only the 32-bit version will work on 32-bit operating systems, but both the 32-bit and 64-bit versions will work on an x64 OS.

What is SuperFetch on Vista/Longhorn?

SuperFetch (or Supercache) is new in these OSs. Normally the File System Cache works with opened files only. Once the last user of a file closes its handle, the file is flushed from cache (and goes to the standby list).

In these new OSs, there is a Windows Service that watches file usage patterns and recognizing these, will pre-cache files (or portions of files) into the Standby Cache. Or more precisely, the standby cache has been broken into three parts and these pre-cached files are added to the Standby-Reserve cache.

One example of this is when I boot my PC and log-in. SuperFetch will recognize from past occurrences that I am likely to open Outlook and, when it sees idle time, start reading in Outlook and my prf. Now when I do start Outlook, Outlook starts faster than it would have because all that time-consuming IO is avoided by being pre-cached.

But doesn't SuperFetch use up all my available memory?

What is important is the zero-page and free memory, and the zero page threads of  the system idle process make sure there is a sufficient supply of that. In fact, when needed, memory will be taken from the Standby Reserve cache before taking from the other two standy lists.

Doesn't the prefetching itself slow down my system?

Microsoft says no. I say "not as much as you would think". The requests to pre-cache happen in low CPU priority threads. In addition, these new OSs support prioritized IO, and this pre-fetch is using low priority IO requests. You can also say that the memory used is low priority since it uses the standby reserve, which is a lower priority than standby-normal priority or standby--core from the viewpoint of freeing up memory for the zero page list.  Still, only if your system is truely idle for a while would it be transparent.  My Vista PC with 2GB of ram will typically have a standby reserve of 500MB to 1.3tmtnGB of stuff - and that is a lot of I/O when it first comes in.  Unless I went for coffee (which I do a lot) I will notice it somewhat.  But overall, I believe my system operates faster  because of it.   And I can always test that by simply turning off the service (but I don't because it is helping).

How do I install it?

No installer. Just download, unzip, read the readme, and copy the exe somewhere convenient.

Where Can I download it?

Here are the ZIP packages:    ATM 32-bit (175KB )   or   ATM 64-bit (224KB )