Clicking a button control provides a good messaging example. This produces not only the message for the mouse button click, but also a wide array of other messages. These include messages to repaint the button in its depressed state, notification messages to inform other objects of the button's change in state, messages to determine the state of the mouse cursor, as well as others. Even a simple act such as moving the mouse or pressing a key on the keyboard can produce an astonishing number of messages.
In addition to communicating user actions, Windows also uses messages internally to do housekeeping. Messages need to be sent to update the time and date, to notify other objects of a change in state, and even to notify applications when system resources are exhausted.
The Windows messaging system is the heart of the operating system. As a result, the messaging system is very complex.
Subclassing and the Windows hooking mechanism operate on messages within the messaging system. This makes subclassing and hooking two very powerful techniques. With them, we can manipulate, modify, or even discard messages bound for other objects within the operating system and, in the process, change the way in which the system behaves. As you might already have guessed, a thorough understanding of the messaging system is critical to mastering the techniques of subclassing and hooking.
Along with this power comes responsibility. It is up to the developer to make sure that he or she is using these techniques correctly. Windows is very unforgiving if these techniques are used incorrectly.
Subclassing
Subclassing techniques deal with intercepting messages bound for one or more windows or controls. These messages are intercepted before they can reach their destination window. The intercepted message can be left in its original state or modified. Afterwards, the message can be sent to its original destination or discarded.
By intercepting messages in this manner, we can have a powerful influence on how the window or control will react to the messages it receives. Consider, for example, right-clicking the Visual Basic (VB) text box control. This action causes a default pop-up menu to be displayed containing the following menu items: Undo, Cut, Copy, Paste, Delete, and Select All. Replacing this menu with one of our own is a fairly simple task using subclassing. Subclassing has many other uses as well, such as:
- Determining when a window is being activated or deactivated and responding to this change
- Responding to new menu items that are manually added to the system menu of a window
- Displaying descriptions of menu items as the mouse moves across them
- Disallowing a user to move or resize a window
- Allowing a user to move or resize a window within specified boundaries
- Determining where the mouse cursor is and responding accordingly
- Modifying the look of a window or control
- Changing the way a combo box operates
- Determining when the display resolution has been changed
- Monitoring the system for a low system-resource condition
- Modifying or disallowing keystrokes sent to a window or control
- Modifying how a window or control is painted on the screen
There are three types of subclassing, all of which I will discuss. The first is instance subclassing, which makes it possible to intercept messages for a single instance of a window or control. This type of subclassing is the most commonly used. It is used to control, for example, the user's ability to size a single instance of a window. The second is global subclassing, which makes it possible to intercept messages for one or more windows or controls that are all created from the same window class. All windows derive from some type of class; these classes describe the fundamental look and behavior of windows created from them. Take, for example, a standard button control; each instance of this control derives from a BUTTON class. Using global subclassing, we can change the behavior of the class. This in turn allows us to intercept messages from all window or control instances created from this class. Using global subclassing we can control the user's ability to size any window created from a particular class. The third type of subclassing, superclassing, is a close relative of global subclassing. Superclassing also has the ability to intercept messages for one or more windows or controls. The difference is that a brand-new window class is created to facilitate this type of subclassing. Similar to global subclassing, superclassing allows users to size a window to be controlled.
Hooking
The window hooking mechanism, or hooks, also deals with intercepting messages, but at a much broader scope than subclassing. Hooking allows us to intercept messages at various set points within the operating system. For example, we can intercept a message before and after a window has processed it.
There are several different kinds of hooks, each with their own special purpose and location within the operating system. They are:
- WH_CALLWNDPROC
- WH_CALLWNDPROCRET
- WH_CBT
- WH_DEBUG
- WH_FOREGROUNDIDLE
- WH_GETMESSAGE
- WH_JOURNALPLAYBACK
- WH_JOURNALRECORD
- WH_KEYBOARD
- WH_KEYBOARD_LL
- WH_MOUSE
- WH_MOUSE_LL
- WH_MSGFILTER
- WH_SYSMSGFILTER
- WH_SHELL
- Modifying messages sent to dialog boxes, scroll bars, menus, or message boxes
- Subclassing a window that resides in a separate process
- Creating a macro recorder that can play back the recorded macro as well
- Developing computer-based training (CBT) applications
- Capturing and modifying mouse or keyboard messages at a system level
- Providing a help function key for menu items and message boxes
- Creating a utility similar to Spy++
- Creating an automated testing application
- Determining when an application is idle
- Modifying mouse buttons and keystrokes for a particular application, or for all applications
- Modifying ALT+TAB and ALT+ESC key functionality
A good VB6 example can be found here.
