Release mode debugging with VC++

The debug configuration of VC++ projects is the configuration that most debugging and development is done on. This is entirely appropriate because the debug information, lack of optimizations, and the extra debug checks in the Microsoft libraries, make stepping through your code and finding bugs much easier than in release builds.

However, sometimes you have to try and debug your release builds. Possible reasons include:

You may have a bug that only occurs in your release builds (for information on why this happens see this article) that you need to debug.
You may be trying to optimize your code, in which case stepping through the code and seeing what the compiler has created is essential.
Your shipped product may be crashing on a customer's machine.

You may be able to avoid release only crashes, and you may not need to optimize your release builds, but you never know when your program is going to start crashing on a distant customer's machine - and if you wait until it happens, it's too late.

If you want to be prepared and professional when your program crashes after you've released it then you need to create debug information for the version that you shipped, you need to archive the debug information, and you need to know how to use it.

At this point somebody inevitably says that it is obvious that enabling debug information is a nonsensical thing to do on a release build. They then claim that enabling debug information will make your program too big or too slow, and you certainly don't want to ship your precious program with symbols.

To this I say, the debug information in VC++ will not make your program bigger, it will not make it slower, and of course you don't ship your symbols. Done properly, enabling debug information lets you ship an executable that is less than a hundred bytes larger. The cost is essentially zero, and the benefits are huge. In fact, I claim that all executables and DLLs created with VC++ should be built with symbols - no exceptions. I further claim that failing to do this is irresponsible.

Enabling debug information in VC++ 6.0 is relatively easy - but terribly non-obvious. There are three separate settings that you have to change.

First go to Project->Settings, select the release build, then:

On the C/C++ tab for all librarys, DLLs and executables, in the general category, set Debug Info to Program Database. This tells the compiler to generate debug information.
On the link tab for all DLLs and executables, in the general category, put a check mark beside Generate Debug Info. This tells the linker to collate debug information into .pdb files. The linker also puts the name of the .pdb file in the executable, so the debugger can find it. That's why your executable gets a bit bigger. Dumpbin /headers will show you that information.
On the link tab, in the Project Options field, type in "/OPT:REF". This tells the linker to continue optimizing away unreferenced code. This is the default when debug information is turned off, but it defaults to off when you tell the linker to generate debug information. Failing to specify this will cause your executables and DLLs to get 10-20% larger. You should also specify /ignore:4089 to tell the linker to not generate warnings if it optimizes away all references to a DLL (this tip works with VC++ .Net also).

This MSDN article - http://msdn.microsoft.com/library/default.asp?URL=/library/techart/gendepdebug.htm - talks about the options for generating debug symbols, although it fails to differentiate between symbols that you ship with your product and symbols that you archive on site.

Creating debug information is nice, but it only helps if the debug information is available when you need it. In order to get maximum use out of the debug information, your build script (you do have an automated build script don't you?) should archive the .pdb file along with the executable (you do archive the executables you release don't you?), and you should put a label in your version control (you do use version control don't you?) so that you can recover the source code. Without these steps the debug information can easily be useless.

If you are using VC++ .Net or WinDbg and you are creating minidumps when your program crashes then the following steps are not necessary - just load the minidump. See this article for information on creating and using minidumps. However if you are using VC++ 6.0 or you aren't yet creating minidumps, follow these steps:

To analyze a crash address, load the .exe into DevStudio (yes, you can do that). Then instead of select Debug->Go, single step to load it into the debugger, and make sure the debug information loads - check the output window to see what it says. Then open the registers window, put an insertion point on EIP and type in the crash address.

When you hit enter, the debugger will take you to that address. Because you are in the debugger, the debugger will load the source code associated with that instruction, exactly as if you had crashed in the debugger.

Well, not exactly. The other registers aren't loaded, the stack isn't loaded, and a lot of information has been lost, but if it's an important bug and you're willing to pore over the stack dump for a while, you can frequently figure out what happened.

If you want to improve your odds, you should collect more information than just the instruction pointer. Check out John Robbins book "Debugging Applications" or download one of many exception handling articles - my favourite is here.

With all this information and a .pdb or .map file you can write tools to create call stacks - either starting from the EIP and working up the stack, or starting from a return address in WinMain and working down the stack. You can also programmatically load the stack and registers into the executable while it's in the debugger, to let the debugger give you more assistance in diagnosing the crash.

Let me know if you find this paper useful.

Please contact me if you have any questions.