menu
  Home  ==>  papers  ==>  colibri_utilities  ==>  events_record_and_playback   

Record and Playback Mouse and Keyboard Events - Felix John COLIBRI.

  • abstract : record and playback all mouse and keyboard messages : all windows messages are saved in a list (or a file) and can be replayed on the spot or later. A must for debugging VCL components, event logging, keyboard macro, computer based training. Includes a readable format which can be used for SendKey / SendClicks replays, simulations or unit test.
  • key words : record - playback - Windows Messages - windows hooks - wh_JournalRecord - wh_JournalPlayback - SetWindowsHookEx - tEventMsg - CallNextHookEx - UnhookWindowsHookEx - ClientToScreen - virtual key codes
  • software used : Windows XP Home, Delphi 6
  • hardware used : Pentium 2.800Mhz, 512 M memory, 140 G hard disc
  • scope : Delphi 1 to 2006, Turbo Delphi for Windows, Kylix
    Delphi 5, 6, 7, 8 Delphi 2005, 2006, Turbo Delphi, Turbo 2007, Rad Studio 2007 to 2009, Delphi XE
  • level : Delphi developer
  • plan :


1 - Record and Playback for Debugging

While developing a custom StringGrid based on a tPaintBox, we were using our standard textual log. However to replay the script when some inconsitencies arose, we faced the task of simulating keyboard and mouse events. Certainly SendMessage could have been used, but figuring out how to build messages for combinations of control keys (Alt, Shift, Control, Home etc) and mouse actions (right clic) was not that easy.

So we decided to bite the bullet and build a reasonable record / playback system which could be used in a non invasive way during our test.

The specification was

  • record and replay keyboard events
  • record and replay mouse events. Since our grid does not use mouse movement (we do not implement "onMouseOver" type of feedback, and resizing the columns was not under scrutiny), recording mouse moves was not included. But this can be easily reintroduced (commenting out an If)
  • be able to visualize the recorded messages (mainly for understanding the recording mechanism)
  • to be able to replay as many time as desired the recorded messages
  • be able to save, and reload the messages later, with the same application, but not necessarily at the same screen position
  • be able to add messages manually, using a textual representation



2 - wh_JournalRecord and wh_JournalPlayback Windows Hooks

2.1 - Windows Hooks

Our system basically uses Windows Journal Hooks. They were specifically created for keyboard and mouse events record and playback.

Hooks are a basic Windows functionality which enables us to ask Windows to call one of our procedure. The same old technique already used in Interrupt redirection

  • we make a call to tell Windows where our callback is
  • the callback is inserted in a (possibly empty) callback queue
  • when the target event happens, Windows calls our procedure. We then
    • perform whatever tasks we desire
    • we usually call the next callback in the queue
There are several hook types. However, we are only interested here by the keyboard and mouse journaling hooks.



2.2 - wh_JournalRecord Windows Journaling hook

We install the hook by calling

my_hook_handle:= SetWindowsHookEx(wh_JournalRecord
    my_journal_record_callbackhInstance, 0);

Where

  • my_journal_record_callback the callback we have to write
  • hInstance in the .EXE instance
  • my_hook_handle will be used to later remove the hook from the chain


Once this hook is installed, Windows will call our callback whenever a message is removed from the message queue:
  • every Window application has a "message loop" :

    While GetMessage Do
    Begin
      TranslateMessage;
      DispatchMessage;
    End;

    For Delphi, this is nested deep inside Application.Run (in the .DPR)

  • the user types on the keyboard and moves the mouse around
  • the messages are stored in a message queue
  • meanwhile, the While loop runs until we quit the program, and whenever there is some message in the queue, it is removed from the queue and the corresponding message handler, if created, is called

    windows_message_queue



When a record journal hook is installed, each time a message is removed from the queue, our callback is called, with the parameters of the message. The tasks of our callback is to store those messages somewhere (in memory, in a file, in a stream...)

wh_journalrecord_callback



Our callback will have the following header:

Function journal_record_callback(p_hook_codeinteger
    m_w_paramm_l_paramLongint): LongintStdcall;

where

  • p_hook_code is a "hook code", with the following values
    • HC_SYSMODALOFF : a system-modal dialog box has been destroyed. The hook procedure must resume recording.
    • HC_SYSMODALON : a system-modal dialog box is being displayed. Until the dialog box is destroyed, the hook procedure must stop recording.
    • HC_ACTION : the other keyboard and mouse events.
  • p_w_param is not used.
  • p_l_param is a pointer to the message information
  • the return value is not used
We are only interested in the hc_Action code. And in this case the p_l_param parameter is a pointer to an tEventMsg structure containing information about a message removed from the system queue.

The tEventMsg, defined in WINDOWS.PAS has the following definition:

Type PEventMsg = ^TEventMsg;
     tEVENTMSG = 
         Packed Record
           messageUINT;
           paramLUINT;
           paramHUINT;
           timeDWORD;
           hwndHWND;
         End;

where

  • message is the message code (wm_MouseDown, wm_KeyUp etc)
  • paramL and paramH are parameters depending on the type of event
  • time is the tick count of the event (allowing to detect double clicks)
  • hwnd is the handle of the window to which the message was posted. It allows Windows to set the focus
In the case of our events
  • for the mouse events
    • the low word of paramL contains the SCREEN x position
    • the low word of paramH contains the SCREEN y position
  • for keyboard events
    • only wm_KeyDown and wm_KeyUp are captured (the wm_Char is generated by the TranslateMessage in the message loop)
    • the virtual key code is in the lo byte of paramL


The tasks of our callback are :
  • to store the tEventMsg somewhere, to be in a position to fetch it back for replay
  • to call the next possible hook callback. This is done inside our callback by calling:

    Result:= CallNextHookEx(my_hook_handlep_hook_code
        p_w_paramp_l_param);

  • we are supposed to handle the vk_Cancel virtual key code, which is triggered by typing Ctrl+ Break, and we should terminate the journaling when we receive this key combination.

    We can also specify other key combinations in order to stop recording. In our application, we trigger the end by clicking on an tButton. We also noticed that within the IDE, Ctrl Alt Del also stops the journaling



To stop the recording, we remove the journal hook :

UnhookWindowsHookEx(my_hook_handle);



Finally, note that

  • the hook is system wide. It cannot be used as thread specific
  • the callback is always handled in the context of our thread


More importantly
  • the tEventMsg message parameters are the "raw unprocessed" windows message informations.
  • the normal driver processing will then take into account the national keyboard, the currently typed keys like Shift, Alt etc to qualify the raw parameters. vk_A might become "a", "A", "â" etc. Similarily the mouse events do not contain any p_button parameter or shift states or other values we usually find in OnKeyDown or OnMouseDown Delphi events.
  • so the tMessageEvent are the raw data before those processing. We store those, and during playback, the raw parameters will be reinjected just before the driver processing, thus producing the very nice event parameters we expect in Delphi


2.3 - wh_JournalPlayback Windows Journaling hook

Replaying the recorded events is symmetric to recording.
  • first, we install the replay hook by calling

    my_hook_handle:= SetWindowsHookEx(wh_JournalPlayback
        my_journal_playback_callbackhInstance, 0);

    Once the replay hook is installed, the GetMessage will call our callback to get the messages from our message store, instead of waiting for the user events:

    wh_journalreplay_callback

  • the replaying callback has the following header:

    Function journal_playback_callback(p_hook_codeinteger
        p_w_paramp_l_paramLongint): LongintStdcall;

    and:

    • p_hook_code is used to specify the type of action. Possible values are:
      • HC_NOREMOVE : the recording application had called the PeekMessage function with wRemoveMsg set to PM_NOREMOVE, indicating that the message is not removed from the message queue after PeekMessage processing.

      • HC_SYSMODALON : A system-modal dialog box is being displayed. Until the dialog box is destroyed, the hook procedure must stop playing back messages.
      • HC_SYSMODALOFF : A system-modal dialog box has been destroyed. The hook procedure must resume playing back the messages.

      • HC_SKIP : The hook procedure must prepare to copy the next mouse or keyboard message to the tEventMsg structure pointed to by lParam. Upon receiving the HC_GETNEXT code, the hook procedure must copy the message to the structure.

        If there are no more messages in our store, we call:

        UnhookWindowsHookEx(my_hook_handle);

      • HC_GETNEXT : the callback must copy the current mouse or keyboard message to the tEventMsg structure pointed to by p_l_param
    • p_w_param : not used
    • p_l_param : A pointer to an tEventMsg structure that represents a message being processed by the hook procedure. This parameter is valid only when the code parameter is HC_GETNEXT.

    • the result of the function specifies the amount of time, in clock ticks, that the system should wait before replaying the message:
      • This value can be computed by calculating the difference between the time attribute in the current and previous recorded tEventMsg. It can be 0.
      • To process the message immediately, the return value should be zero.
      • this value is used only if the hook code is HC_GETNEXT, otherwise, it is ignored.
    Our basic tasks are :
    • to handle HC_SKIP for initializing a tEventMsg variable
    • to handle HC_GETNEXT, where we initialize p_l_param to point to this variable. Eventually we tell how much tick we should wait
    • to call the possible next hook playback, with:

      Result:= CallNextHookEx(my_hook_handle
          p_hook_code,  p_w_paramp_l_param);



Please note that
  • to replay the same tEventMessage several time, we simply keep the same value in our tEventMessage (HC_SKIP does not grab the next message, but keeps the current one for some iteration).
  • to sleep between the replay of a message
    • in a first callback, in HC_GETNEXT we return the wait tick count.
    • after the pause, Windows calls back our procedure, and in HC_GetNext we now should return 0
    This could be used to slow the replay down
  • the callback is always handled in the context of our thread (the one which installed the wh_JournalReplay). It can be in our project, or in a Library (a .DLL)
  • If the user presses Ctrl+Esc or Ctrl+Alt+Del during journal playback, the system stops the playback, unhooks the journal playback procedure, and posts a wm_CancelJournal message to the journaling application.
  • if the hook procedure returns a message in the range wm_KeyFirst to wm_KeyLast, the following conditions apply:
    • tEventMsg.paramL specifies the virtual key code of the key that was pressed.
    • tEventMsg.paramH specifies the scan code (the keyboard touch number).
    • there's no way to specify a repeat count. The event is always taken to represent one key event.



3 - Delphi Mouse and Keyboard event Journal and Replay

3.1 - Overall architecture

  • the c_message_list is a tStringList container which stores the tEventMessages (in memory or disc)
  • to be able to replay the messages from a disc file, we must be able to set the window handles of the controls in the application which reloads the messages. Our c_win_control_list is in charge of mapping the control names to the Windows handles
  • our c_record_playback_message Class is in charge of launching the record and playback, and contains the two callbacks
  • finally a c_message_list_parser is able to parse a textual tEventMsg representation


3.2 - The UML class diagram

replay__uml_class_diagram



3.3 - The tWinControl list

This list is an auxiliary structure used to get the window handle of a control (say Button3) from its name ('Button3') and get the name and the control from is handle.

The definitions are:

Type c_win_control// one "win_control"
         Class(c_basic_object)
           m_c_wincontrol_reftWinControl;

           Function f_screen_xInteger;
           Function f_screen_yInteger;
         End// c_win_control

     c_win_control_list// "win_control" list
         Class(c_basic_object)
           m_c_win_control_listtStringList;

           Constructor create_win_control_list(p_nameString);

           Procedure add_win_control(p_win_control_nameString;
               p_c_win_controlc_win_control);
           Procedure build_wincontrol_list(p_c_controltWinControl);

           Function f_c_find_win_control_by_handle(p_handletHandle): c_win_control;
           Function f_handle_to_name(p_handletHandle): String;
           Function f_name_to_handle(p_control_nameString): tHandle;
         End// c_win_control_list

The structure is loaded by a recursive procedure using the Controls array:

Procedure c_win_control_list.build_wincontrol_list(p_c_controltWinControl);

  Procedure _build_wincontrol_list_recursive(p_c_controltWinControl);
    Var l_child_control_indexinteger;
    Begin
      With p_c_control Do
      Begin
        With f_c_add_win_control(NameDo
          m_c_wincontrol_ref:= p_c_control;

        For l_child_control_index:= 0 To ControlCount- 1 Do
          If Controls[l_child_control_indexIs tWinControl
            Then _build_wincontrol_list_recursive(p_level+ 1,
                tWinControl(Controls[l_child_control_index]));
      End// with p_c_control
    End// _build_wincontrol_list_recursive

  Begin // build_wincontrol_list
    _build_wincontrol_list_recursive(p_c_control);
  End// build_wincontrol_list

and the f_screen_x and f_scree_y are used to compute the current SCREEN position of the top left corner of any control on our form :

Function c_win_control.f_screen_xInteger;
  Begin
    With m_c_wincontrol_ref Do
      Result:= ClientToScreen(Point(LeftTop)).X;
  End// f_screen_x



3.4 - The message list

The message list is defined by:

Type t_message=
         Packed Record
           m_wm_codeCardinal;
           m_w_paramLongint;
           m_l_paramLongint;
           m_timeDWORD;
           m_window_handleHWND;
         End// t_message
     // -- to get the Windows messages while recording
     t_pt_message= ^t_message;

     // -- for restoring control handle from (file saved) control name
     t_message_with_control_name=
         Packed Record
           m_messaget_message;
           m_control_nameString[107];
         End// t_message_with_control_name

     c_message// one "message"
         Class(c_basic_object)
           m_messaget_message;

           Constructor create_message(p_nameString);
           Function f_display_messageString;
           Function f_is_mouse_messageBoolean;
         End// c_message

     c_message_list// "message" list
         Class(c_basic_object)
           m_c_message_listtStringList;
           m_c_win_control_listc_win_control_list;
           // -- for relative tick count computations
           m_start_tick_countInteger;

           Constructor create_message_list(p_nameString;
               p_c_win_controltWinControl);
           Function f_c_add_message(p_message_nameString): c_message;
           Function f_c_add_recorded_message(p_pt_messaget_pt_message): c_message;
           Function f_c_add_a_message(p_nameString;
               p_wm_codeCardinalp_w_paramp_l_paramLongInt;
               p_timeDWORDp_window_handleHWND): c_message;

           Procedure compute_relative_tick_count_and_mouse_xy;

           Procedure save_to_file_with_control_name(p_full_file_nameString);
           Procedure load_from_file_with_control_name(p_full_file_nameString);
         End// c_message_list



And:

  • the t_message has the same structure as the tEventMsg
  • the t_pt_message pointer is used to cast the p_l_param in the record and replay callback
  • t_message_with_control_name is a t_message along with the control name.

    Basically the t_message contains the window handle at the time of recording. If we replay during the same run this window handle is valid. However, if we save the messages on disc and reload them during another execution, the handle will no longer valid. We must somehow change the handles to match the handles of the actual controls.

    Therefore

    • when we save the messages, we naturally save the t_message parameters, and also the control name (say 'Edit3', or 'PaintBox5'.
      This name is looked up using the c_win_control_list. This list is built when we create the c_message_list, and used when we save the message list
    • whenever we reload the message from a file
      • we create a c_message_list, and rebuild the c_win_control_list with the current controls of the application
      • for each message read from the disc, we read the control name ('Panel18') and compute the window handle of the controls in this application. This is only required for mouse messages. Hence the c_message.f_is_mouse_message function
  • the c_message_list
    • has a Constructor with a tWinControl parameter. We call the Constructor with the main tForm object, and this object is used to recursively build the c_win_control control / handle map
    • we have several functions for appending messages to our list
    • the recorded t_message.m_time contains the tick count at the time of the recording.

      When we replay the message later, we might want do use the actual tick count.

      Therefore:

      • when we start the recording, we initialize the m_start_tick_count value (using a call to GetTickCount)
      • when we stop the recording, we compute a relative tick count, subtracting m_start_tick_count from all time values
      • when we replay, we initialize m_start_tick_count, and add this value back to the relative value saved in the message store

    • the same goes on for the mouse X, Y positions
      • at recording time, the values are relative to the SCREEN (the recording is SYSTEM WIDE)
      • when we reload the message list from the file, maybe a couple of days later, the window might not have the same screen position, or we might even have moved some controls away from one another (using splitters or tAlign and Anchor properties). So the screen position would be wrong, and all the mouse messages might not work properly (usually nothing happens, but we might as well click on some other button)
      • therefore, when we stop recording, we decrement the mouse values by the left top values of the win control. This is performed using the c_win_control_list. So the X, Y values are the control-relative mouse position
      • at replay time, the c_win_control_list has been initialized with the current control positions (the same as during the recording, or the values recomputed when we create the message list).
        The screen control offset are then added back to each message before replay
    • removing the absolute time and position values is performed in the compute_relative_tick_count_and_mouse_xy procedure
    To recap,
    • after the recording ends, the relative time and mouse positions are computed
    • the message list is saved after recording
    • therefore, after recording, the in-memory or the possibly saved message list always contain relative value
    • at replay time, the absolute values are recomputed, using the current tick count as well as the current control positions


Please note
  • the time value seems irrelevant. Computing the "nearly" actual time by adding back GetTickCount is not necessary. Running the list with relative times, or even with 0 time values (as we do for the manually crafted messages below) has the same result. However, we could use the relative values to inject some delays between the messages (return a non zero values from the callback)


3.5 - The record and playback Class

The definition is :

Type t_on_notifyProcedure();

     c_record_playback_message=
         Class(c_basic_object)
           m_c_message_listc_message_list;

           m_hook_handlehHook;

           m_is_recordingBoolean;

           m_replay_start_indexm_replay_end_indexword;
           m_is_playingBoolean;
           m_replay_indexInteger;
           m_c_next_messagec_message;

           m_on_notifyt_on_notify;

           Constructor create_record_playback_message(p_nameString;
               p_c_win_controltWinControl);
           // -- record / playback
           Procedure start_recording;
           Procedure stop_recording;

             Procedure get_next_message(p_replay_indexInteger);
           Function f_replay_messaget_replay_error_type;
           Procedure stop_playback;

           Destructor DestroyOverride;
         End// c_record_playback_message

and:

  • the recorded message list is managed by c_message_list
  • when we install the hook (a call to start_recording), the returned handle is saved in m_hook_handle. At the same time, m_is_recording is toggled to true to avoid recursive recording
  • before replaying, we initialize m_replay_start_index and m_replay_end_index:
    • by default the values are 0 and the message count- 1
    • however we might want to skip some initial messages (while tracking a bug), and usually we remove the last two messages which are the click on the "stop_recording" button
  • when we call start_replay
    • the current message index m_replay_index is initialized to m_replay_start_index, and m_is_playing is initialized to True
    • each time hc_Skip is reached
      • if m_replay_index is greater than m_replay_end_index, we call stop_playback
      • if not we get the the next message (make a copy of the current message and adjust the relative time and position values) and increment the current index
  • m_on_notify is a callback event that we used to display the current message being replayed


Our record playback procedure is:

Function journal_record_callback(p_hook_codeinteger;
    p_w_paramp_l_paramLongint): LongintStdcall;
  Var l_pt_messaget_pt_message;
  Begin
    Result:= 0;

    With f_c_record_playback_message(NilDo
    Begin
      // -- optionally stop recording when type Pause / Break
      If GetKeyState(vk_Pause)< 0
        Then Begin
            stop_recording;
            Result:= CallNextHookEx(m_hook_handle,
                p_hook_codep_w_paramp_l_param);
            Exit;
          End;

      Case p_hook_code Of
        HC_ACTION :
          Begin
            l_pt_message:= t_pt_message(p_l_param);
            With l_pt_messageDo
              If (m_wm_code<> wm_MouseMoveAnd (m_wm_code<> $FF)
                Then m_c_message_list.f_c_add_recorded_message(l_pt_message);
          End// HC_ACTION

        Else
          // -- call next hook in chain
          Result:= CallNextHookEx(m_hook_handlep_hook_codep_w_paramp_l_param);
      End// case p_hook_code
    End// with f_c_record_playback_message(Nil)
  End// journal_record_callback



And our replay callback is :

Function journal_playback_callback(p_hook_codeinteger;
    p_w_paramp_l_paramLongint): LongintStdcall;
  Begin
    With f_c_record_playback_message(NilDo
      Case p_hook_code Of
        HC_SKIP:
            Begin
              // -- increment message counter
              inc(m_replay_index);

              // -- check to see if all messages have been played
              If m_replay_index>= m_replay_end_index
                Then stop_playback
                Else
                  With m_c_message_list Do
                    get_next_message(m_replay_index);
              Result:= 0;
            End;
        HC_GETNEXT:
            Begin
              // -- move message in buffer to message queue
              t_pt_message(p_l_param)^:= m_c_next_message.m_message;
              // -- process immediately (no delay)
              Result:= 0;

              If Assigned(m_on_notify)
                Then m_on_notify;
            End

        Else
          // -- call next hook in chain
          Result:= CallNextHookEx(m_hook_handle,
              p_hook_codep_w_paramp_l_param);
      End// case p_hook_code
  End// journal_playback_callback



Finally, we compute the next message to replay with:

Procedure c_record_playback_message.get_next_message(p_replay_indexInteger);
    // -- dynamically recompute the absolute tick and, if mouse, xy
    // --  => allows to replay any number of time
  Begin
    // -- copy the message from the message store
    With m_c_message_list.f_c_message(p_replay_indexDo
      m_c_next_message.m_message:= m_message;

    // -- compute the current SCREEN position
    With m_c_next_messagem_message Do
    Begin
      If f_is_mouse_message
        Then
          With m_c_message_list.m_c_win_control_list
              .f_c_find_win_control_by_handle(m_window_handleDo
          Begin
            Inc(m_w_paramf_screen_x);
            Inc(m_l_paramf_screen_y);
          End;
      Inc(m_timem_c_message_list.m_start_tick_count);
    End;
  End// get_next_message

And

  • before installing the journal replay hook, we call get_next_message(0)
  • in each playback call, we increment the index and call this method again


Please note that
  • the message used for playback is contained in a separate variable. We copy the values of the current message in this structure and adjust this variable to get the absolute time and screen position
  • it was a design decision to recompute the values for each run in a separate variable instead of directly modifying the values in the message list.
    This former solution required using booleans to remember whether the values were absolute or relative, and quickly became a mess. And copying 20 bytes today is not such a big deal


Displaying the messages

Our message list contains all kind of display function that you can call at any time (while recording, after the recording has stopped, after reloading the saved messages, while replaying etc)



3.6 - Forging messages

We can manually build messages, by using any kind of message syntax. We must be able to build tEventMsg information :
  • for keyboard message, we simply convert some textual information (say "Return" or "A" into the binary wm_code, wParam and lParam
  • for mouse messages, we have to build the wm_code, then screen position and the window handle
We chose a readable, although somehow cumbersome, syntax, which is the following
  • for keyboard messages
    • the ascii character (say "a", or "b", or "c")
    • for non printable key, the virtual key code (say "vk_Return", "vk_F2", "vk_Shift" etc, with an additional incremental delay parameter measured in ticks. To avoid confusing letters "vk_Return" with "v", "k" "_", we enclose the coding in an "improbable" character. For now we chose the greek letter MU

      The coding of vk_Home with a 400 tick delay would look like

          µvk_home,400µ

    Our parsing unit contains a function

    Const k_virtual_key_code_separator'µ';

    Function f_relative_virtual_key_code_text(p_virtual_key_codeWord;
        p_delta_timeCardinal): String;
        // -- format "µvk_tab,R_tickµ"
      Begin
        Result:= k_virtual_key_code_separator
            + LowerCase(f_virtual_key_code_name(p_virtual_key_code))
            + ','
            + IntToStr(p_delta_time)
            + k_virtual_key_code_separator
            ;
      End// f_virtual_key_code_text

  • for mouse messages, we used special "vk_" texts and the X, Y position, as well as a

    Our encoding function is:

    Const k_left_mouse_down_name'vk_left_mouse_down';
          k_left_mouse_up_name'vk_left_mouse_up';
          k_right_mouse_down_name'vk_right_mouse_down';
          k_right_mouse_up_name'vk_right_mouse_up';

    Function f_relative_mouse_text(p_mouse_messageString;
        p_delta_xp_delta_yp_delta_timeIntegerp_control_nameString): String;
        // -- format "µvk_left_mouse_down,R_NNN,R_NNN,R_tick,NAMEµ"
        // -- like "µvk_left_mouse_downµ200,100,4000,Edit4"
      Begin
        Result:= Format(k_virtual_key_code_separator
                + '%s,%d,%d,%d,%s'k_virtual_key_code_separator,
            [p_mouse_messagep_delta_xp_delta_yp_delta_timep_control_name]);
      End// f_relative_mouse_text




There for, to create a message which
  • selects Edit3
  • types "a"
  • goes to the next control (assuming it is another edit), by typing "Tab"
  • goes at the start of the edit (avoiding to overwrite the AutoSelected Text) by typing "Home"
  • typing "b"
we would call:

With my_c_message_list_parser Do
  my_message_text:=  ''
    + f_relative_mouse_text(k_left_mouse_down_name, 10, 2, 0, 'Edit3')
    + f_relative_mouse_text(k_left_mouse_up_name, 10, 2, 0, 'Edit3')
    + 'a'
    + f_relative_virtual_key_code_text(vk_tab, 400)
    + f_relative_virtual_key_code_text(vk_home, 400)
    + 'b'
    ;

which will create the following text (line break added) :

    µvk_left_mouse_down,10,2,0,Edit3µµvk_left_mouse_up,10,2,0,Edit3µ
       aµvk_tab,400µµvk_home,400µb

This line is parsed by calling parse_relative_message_string, and the resulting c_message_list can be replayed, saved, reloaded etc, like any other recorded message.

The detail of the parsing in included in the downloadable .ZIP



3.7 - The Delphi Journaling Hook demo

Here is a quick demo :
   we started the app, clicked the "record_" toggle once, and
  • clicked on Edit3
  • types "a" Tab End "b"
   Here is the snapshot :

01_recording_mouse_and_keyboard

   to stop the recording, we clicked the "records_" toggle a second time
   to display the message, we clicked "display_message_list_":

   this is the message display :

02_stop_recording_windows_messages

   we clicked "playback_"
   here is the result:

04_replay_recorded_mouse_and_keyboard_events

notice that

  • the letters have been double ("aa" and "bb")
  • the caret is after the second "b"
   you can replay again. Make sure you click on "playback_" again, since the focus is now in Edit4. You can save, reload another day etc


In any case, remember, if your are stuck, Ctrl Alt Del is your best friend !



To demonstrate the textual message feature
   click "generate and parse"
   here is the display:

05_generate_textual_message_and_parse

   you can display, save, playback the message


Just for the record, writing this little app took us 2 days, plus one for the article. Enjoy !




4 - Improvements

Among the possible improvements:
  • provide for a multi-form journaling. This would require saving the form name along with the control name in the win control list
  • use a more concise textual representation for messages (like "~" for Ctrl, "@" for Alt etc)
  • implement repeat or delay features
  • add automatic unhook procedures, at least if we quit the project
  • our tWinControl structure could have been optimized (sorted, for instance), and maybe before replay we could include in each c_message a reference to the tWinControl to avoid lookups



5 - Download the Sources

Here are the source code files:
  • mouse_and_keyboard_record_and_playback.zip: the project with
    • the win control list
    • the message list
    • the record / playback manager
    • the textual message parser
      mouse_and_keyboard_record_and_playback.zip creating
    Size : (38 K)
The .ZIP file(s) contain:
  • the main program (.DPR, .DOF, .RES), the main form (.PAS, .DFM), and any other auxiliary form
  • any .TXT for parameters, samples, test data
  • all units (.PAS) for units
Those .ZIP
  • are self-contained: you will not need any other product (unless expressly mentioned).
  • for Delphi 6 projects, can be used from any folder (the pathes are RELATIVE)
  • will not modify your PC in any way beyond the path where you placed the .ZIP (no registry changes, no path creation etc).
To use the .ZIP:
  • create or select any folder of your choice
  • unzip the downloaded file
  • using Delphi, compile and execute
To remove the .ZIP simply delete the folder.

The Pascal code uses the Alsacian notation, which prefixes identifier by program area: K_onstant, T_ype, G_lobal, L_ocal, P_arametre, F_unction, C_lass etc. This notation is presented in the Alsacian Notation paper.
The .ZIP file(s) contain:

  • the main program (.DPROJ, .DPR, .RES), the main form (.PAS, .ASPX), and any other auxiliary form or files
  • any .TXT for parameters, samples, test data
  • all units (.PAS .ASPX and other) for units
Those .ZIP
  • are self-contained: you will not need any other product (unless expressly mentioned).
  • will not modify your PC in any way beyond the path where you placed the .ZIP (no registry changes, no path outside from the container path creation etc).
To use the .ZIP:
  • create or select any folder of your choice.
  • unzip the downloaded file
  • using Delphi, compile and execute
To remove the .ZIP simply delete the folder.

The Pascal code uses the Alsacian notation, which prefixes identifier by program area: K_onstant, T_ype, G_lobal, L_ocal, P_arametre, F_unction, C_lass etc. This notation is presented in the Alsacian Notation paper.



As usual:

  • please tell us at fcolibri@felix-colibri.com if you found some errors, mistakes, bugs, broken links or had some problem downloading the file. Resulting corrections will be helpful for other readers
  • we welcome any comment, criticism, enhancement, other sources or reference suggestion. Just send an e-mail to fcolibri@felix-colibri.com.
  • or more simply, enter your (anonymous or with your e-mail if you want an answer) comments below and clic the "send" button
    Name :
    E-mail :
    Comments * :
     

  • and if you liked this article, talk about this site to your fellow developpers, add a link to your links page ou mention our articles in your blog or newsgroup posts when relevant. That's the way we operate: the more traffic and Google references we get, the more articles we will write.



6 - References

Here are a couple of references


7 - The author

Felix John COLIBRI works at the Pascal Institute. Starting with Pascal in 1979, he then became involved with Object Oriented Programming, Delphi, Sql, Tcp/Ip, Html, UML. Currently, he is mainly active in the area of custom software development (new projects, maintenance, audits, BDE migration, Delphi Xe_n migrations, refactoring), Delphi Consulting and Delph training. His web site features tutorials, technical papers about programming with full downloadable source code, and the description and calendar of forthcoming Delphi, FireBird, Tcp/IP, Web Services, OOP  /  UML, Design Patterns, Unit Testing training sessions.
Created: oct-10. Last updated: mar-2017 - 94 articles, 229 .ZIP sources, 1183 figures
Copyright © Felix J. Colibri   http://www.felix-colibri.com 2004 - 2017. All rigths reserved
Back:    Home  Papers  Training  Delphi developments  Links  Download
the Pascal Institute

Felix J COLIBRI

+ Home
  + articles_with_sources
    + database
    + web_internet_sockets
    + oop_components
    + uml_design_patterns
    + debug_and_test
    + graphic
    + controls
    + colibri_utilities
      – delphi_net_bdsproj
      – dccil_bat_generator
      – coliget_search_engine
      – dfm_parser
      – dfm_binary_to_text
      – component_to_code
      – exe_dll_pe_explorer
      – dll_process_viewer
      – the_alsacian_notation
      – html_help_viewer
      – cooking_the_code
      – events_record_playback
      – colibri_hiragana_quiz
    + colibri_helpers
    + delphi
    + firemonkey
    + compilers
    + vcl
  + delphi_training
  + delphi_developments
  + sweet_home
  – download_zip_sources
  + links
Contacts
Site Map
– search :

RSS feed  
Blog