/O1 (minimize size)

By default, a project created by the Microsoft Foundation Class (MFC) library AppWizard uses /O2 (maximize speed) for its release-build configurations. However, Microsoft builds all its commercial applications with /O1, and that's what you should be using. What Microsoft has found is that after picking the best algorithm and writing tight code, avoiding page faults can help speed up your application considerably. As I've heard it said, "Page faults can ruin your day!"

Page faults occur when your executing code moves from one page of memory (4 KB for x86 Intel) to the next. To resolve a page fault, the operating system must stop executing your program and place the new page on the CPU. If the page fault is soft, meaning that the page is already in memory, the overhead isn't too terrible—but it's extra overhead nonetheless. If the page fault is hard, however, the operating system must go out to disk and bring the page into memory. As you can imagine, this little trip will cause hundreds of thousands of instructions to execute, slowing down your application. By minimizing the size of your binary, you decrease the total number of pages your application uses, thereby reducing the number of page faults. Granted, the operating system's loaders and cache management are quite good, but why take more page faults than you have to?

In addition to using /O1, you should look at using the Working Set Tuner (WST) utility from the Platform SDK. WST will help you order your most commonly called functions to the front of your binary so that you minimize your working set, the number of pages kept in memory. With your common functions up front, the operating system can swap out the unneeded pages. Thus, your application runs faster. For more on using WST, see my February 1999 "Bugslayer" column in Microsoft Systems Journal on MSDN.