Testing and Debugging in Visual C++ 6.0

Error Handling

Natural part of coding process - essential to robust applications.

Anticipating Coding Errors

Code frequently has logical errors, even those that do not might not cope well in unexpected environments. Never assume anything within program - include code to gracefully cope with error conditions - e.g. new statements failing to allocate memory. Anticipatory approach (dealing with error immediately in code) = inline error handling.

Continually checking return values can make code unreadable. Consider using exception handling where function separated into two logical sections - one for normal execution and other for trapping errors.

Exceptions

Any situation OS considers erroneous. When app raises exception the OS notifies the app it has caused an error by calling applications exception handler (if one exists). If the app fails to handle the error the OS resolves problem by terminating application.

Structured Exception Handling (SEH)

Available in both C and C++. Incorporated in application via __try, __except and __finally. __try blocks must be matched by either __except or __finally (not both).

__except block executes if code within __try causes an exception. If __try finishes successfully, execution resumes at instruction following __except. Through filter parameter the __except block indicates if it can deal with exception. The filter must evaluate to:

Filter parameter usually determined by helper function that analyses current conditions and attempts fix - if it succeeds then EXCEPTION_CONTINUE_EXECUTION is issued. Must use with care - can end up in infinite loop.

__finally has little to do with exception system. It defines block of instructions that compiler guarantees will execute when __try block finishes.

C++ Exception Handling

SEH is system service, C++ Exception Handling is application based. More sophisticated than structured exception handling. Use of SEH discouraged when writing C++ apps. Acts in a similar manner to SEH, uses keywords try, catch and throw.

The catch block specifies a class for which it handles errors, for example

catch(CMemoryException *e)

{

}

will handle memory related errors. C++ programs not using MFC can design own classes to provide to catch block. If the parameter list of the catch block is (...) it indicates that the code within will handle all types of exceptions.

If catch can fix problem, it retries failing instruction by executing throw command. If catch block completes without throw then execution continues next statement following catch block.

MFC used to use macros (TRY, CATCH and THROW) to implement exception handling. These now map to the standard C++ try, catch and throw statements.

Benign exception can occur. These do not show up in the application as the OS traps and handles them, for example when program accesses uncommitted memory page the system recognises this and commits another page of memory - the app is unaware of this happening.

Logging Errors

Not all error can be anticipated and caught - require ability to keep record of problems (error log) as they occur for later analysis. MFC supplies TRACE (+ variations) to provide this functionality.

TRACE macros display messages at location specified by AfxDump - by default IDE Output Window. For release builds macro not expanded. Macro accepts same formatting as printf, up to maximum of 512 characters.

COM Errors

COM components must support COMs standard error handling mechanism - HRESULT error codes.

HRESULT

A 32-bit value indicating success or failure condition. Many values predefined, such as S_OK for success, S_FALSE for failure, E_INVALIDARG for invalid argument, etc. E prefix indicates error, S a status code.

Can define own HRESULTS. Should use COM supplied ones where possible - prevents confusion. HRESULT consist of four fields:

Facility codes defined by COM, FACILITY_ITF available for developer specific codes.

COM assumes HRESULT of zero - success, +ve values also success (qualified).

COM provides macros SUCCEEDED and FAILED to determine status of HRESULT.

_com_error class

Encapsulate HRESULT, providing access functions to gain details on encapsulated error.

• Error - HRESULT of error
• ErrorInfo - Pointer to associated IErrorInfo or NULL (if no interface available)
• Wcode - HRESULT - 0x80040200 if FACILITY_ITF, otherwise 0
• ErrorMessage - TCAHR pointer to message describing error. If unavailable contains "Unknown error"

IErrorInfo provides contextual error information, such as entry within a help file.

Often used as object passed to catch block.

Client Access to HRESULT

HRESULT often unavailable to client due to limitation of IDispatch interface. Clients are aware an error occurred, but can not determine precise reason why.

Error Event

When COM components returning HRESULT to clients, the notification is synchronous - part of normal flow of client/server communication. Client learns of error only afer component ceases task and returns.

Sometimes component encounters error which client should be informed of immediately (asynchronous). Achieve this by firing an error event. MFC provides predefined dispatch identifier of DISPID_ERROREVENT. Invoke by issuing FireEvent().

Determine Appropriate Debugging Technique

VC++ debugger integrated into IDE with own menu and toolbars. Can not use unless code compiled in appropriate manner.

Debug version of program used during development and test while attempting to make it error free. Contains symbol information for use by debugger - debugger can associate lines of source code with corresponding portion of executable image. Note, optimised code can cause strange behaviours in debugger. Can execute image outside of debugger.

Release version more tightly compiled. No symbol information contained within image. Can execute release version in debugger, but get no source code association.

If problem encountered during execution outside IDE then debugger launched. If debug version of program executing then source code visible, otherwise only get assembly.

Library Support

TRACE already discussed.

ASSERT tests Boolean assumption during development, compiled out for release. Example - check pointer non-zero before using it. Traditionally use if...then blocks, in many cases conditions won't vary between debug and release - no need for check -> improved performance. If ASSERT fails program halts with system generated message, permitting program to be aborted, debugged or continued.

ASSERT_VALID checks to see if pointer provided points to object derived from CObject.

ASSERT_KINDOF checks to see if pointer provided points to object of specified type (must be derived from CObject).

VERIFY* series of macros similar to ASSERT, but are left in during release builds.

DEBUG_NEW used to help find memory leaks. By redefining new to be DEBUG_NEW (as in #define new DEBUG_NEW) MFC will track the allocation and release of the memory block. By calling appropriate helper functions can determine any objects that have not been de-allocated at any point within program.

IDE

Breakpoints

Marker set in source code that cause program to interrupt itself when reached. During program execution debugger sleeps, awakes when breakpoint hit.

Two forms of breakpoint, one based on location and other on data. Data breakpoints cause execution to be suspended when variable has certain value or changes.

Location breakpoints can be toggled by F9. Both can be set via breakpoints dialog - lists all currently set breakpoints and allows new ones to be inserted.

Dialog Box

Set location by function name (if C++ include class name) or label (assuming one set up within editor). Conditional breakpoints only triggered if specified condition is TRUE when execution reaches marked instruction.

Data breakpoints set by typing in name of variable (if array must specify number of elements) or expression to evaluate. Considerable performance degradation if more than four data breakpoints specified, or reference made to stack based variable.

Message breakpoints are attached to specific windows procedure. Invoked when specified message (such as WM_SIZE) is received. Not always useful in MFC as windows procedures handled deep within framework - easier set location breakpoint on function handling message.

Edit and Continue

Permits fixing of many problems in debugger source window - no need to exit compiler and recompile. Does not recognise source changes that are impossible, impractical or unsafe to compile while debugging:

• Alterations to exception handlers
• Deletion of functions
• Changes to class and function definitions
• Changes to static functions
• Changes to resource data

COM Components

Easily handles in-process. Can begin debugging either in container or component, debugger can cross process boundaries. If starting from component, must specify name of container (possibly ActiveX Control Test container supplied with VC) in debug tab of project settings.

Out-of-process handled very similarly. To invoke methods on server, specify name of client in debug tabe of project settings.

Dependency Walker

Windows applications are not self-sufficient. Require presence of supporting DLLs (dependencies) listed in header of exe or dll. Normally dependencies hidden from user, unless one can't be found.

The dependency walker reads information from application header, reveals details on:

• Name and location of dependencies
• Base address dependency loaded into
• Version information on each file
• Does dependency module contain debug information?

Spy++

Displays information on current processes, their threads, open windows, etc. at given point in time (acts as snapshot). Four main views:

• Windows - list of open windows (hidden or visible). Presented as tree view, indicating parentage. Can be tricky to find particular window - use find tool, an icon that can be dragged over screen and dropped over window of interest to highlight entry in tree.
• Processes - list processes
• Threads - list threads
• Message log - all messages received by window. Very useful for debugging. Records in real time all messages sent / received by window (in order arrived / dispatched).

These views can be tiled within display area.

Testing

Ensures app performs as specified. When nearly complete + stable testing begins in earnest to detect and fix errors. Conduct under different scenarios to reflect real world conditions - test under different OSs, memory conditions, system loads, etc.

Glossary of terms:

• Unit testing - verifies discrete piece of code (loop, sub-routine, event, etc.). Smallest piece of code for which a practical test can be conducted.
• Integration testing - confirms problems don't arise when combining units of code. Pay special attention to threading issues.
• System testing - test full application. Emphasis less on bug hunting, more checking that application and environment interact correctly, e.g. performance, resource usage, etc.
• Stress testing - check hw application behaves under adverse conditions - low memory, disk space, high network traffic, etc.
• Regression testing - repetition of previous test after changes made to source code. Verifies changes to fix bugs are successful and have no adverse affects elsewhere.
• Beta testing - distribute pre-release version of application to selected customers. Tests application under wide variety of real world situations.
• User acceptance testing - tested version of application provided to trained users who will expose system to the kind of usage it will receive in real life.

Plan

Written version of test-suite for application. Describes all testing to be performed identifying what constitutes success or failure in each case. Written to provide direction to people other than the author.

Provides formal basis on which to develop regression test.

Plan includes:

• Description of application and functionality to be tested
• Test objectives
• Description of how tests performed, indicating reliance on testing components such as scripts, checklists and user involvement.
• Description of test environment such as version of OS to use
• Listing of test data
• Description of restrictions placed on test team, e.g. small team trying to test system designed for hundreds of concurrent users.
• Relative priority of the tests - e.g. robustness over performance.
• Features of application to test - together with reasons why
• Test schedule identifying milestones. Should tie into overall project plan.

After designing plan should list each test scenario. Follows similar process to test plan, include:

• Reference to item being tested
• Expected results
• Describe how expected results confirm item being tested performs correctly.

Downloads

• testinganddebugging.rtf