menu
  Home  ==>  papers  ==>  colibri_helpers  ==>  stdin_stdout   

StdIn and StdOut - Felix John COLIBRI.

  • abstract : sending and receiving strings to and from a CONSOLE project which uses Readln and Writeln
  • key words : Windows pipes, StdIn, StdOut, Readln, Writeln, CONSOLE, CreateProcess
  • software used : Windows XP, Delphi 6
  • hardware used : Pentium 1.400Mhz, 256 M memory, 140 G hard disc
  • scope : Delphi 1 to 8 for Windows
  • level : Delphi developer
  • plan :


1 - Introduction

For a simple CGI server, we had to communicate with a CGI.EXE CONSOLE application which uses Readln and Writeln. When our web server itself is a CONSOLE application, those input / output procedures work as expected. But when the CGI server is a GUI application, Delphi tells us that Readln and Writeln are no longer valid.

A couple of communications found using Google and the Borland Newsgroups did provide the solution. It was still a one day fight before I understood what was required. Here is our result, without all possible bells and whistles, but which somehow seem to work, at least for our simple CGI server.




2 - IO principle

2.1 - Console applications

Here is a simple my_echo.exe CONSOLE program, which reads a string, and sends it back in uppercase:

(*$APPTYPE CONSOLE *)

program p_console_read_write_one;
  var g_stringString;
  begin
    readln(g_string);
    writeln(UpperCase(g_string));
  end.



If we compile and execute
   we will see a black DOS windows, waiting for input:

my_echo.dpr

   we type some string on the keyboard and hit Enter, like

  Hello Enter

my_echo_readln.dpr

   the string will be echoed back

my_echo_readln.dpr

And since this is the last instruction, the program will terminate (and you won't see anything unless you block the program with another Readln before the END, and type Enter to quit the program).



Basically, the keyboard feeds the Readln, and Writeln sends the output to the screen. In C parlance, Readln grabs input from the StdIn handle, and Writeln sends characters to the StdOut handle.

my_echo_stdin_stdout.dpr



2.2 - GUI Applications

We now want to call this program from another Delphi GUI application, which will provide the string, and display the uppercase answer:

my_gui_echo.dpr

So basically we have to send our string to my_echo.StdIn, and fetch whatever my_echo.StdOut sends back. We CANNOT use my_gui.Writeln and my_gui.Readln, since Delphi tells us that my_gui.StdIn and my_gui.StdOut are not available in GUI application. We must use pipes which are some kind of files with two handles: a read handle and a write handle.

To send Hello from the gui to the echo:

  • my_gui writes the string at the write extremity of the pipe

    pipe_write_handle

  • my_echo reads this string from the read extremity of the same pipe:

    pipe_write_handle

In our case, the GUI application will create two pipes:
  • a std_in pipe where the my_gui application pushes the string, and my_echo will use for its Readln calls
  • a std_out pipe where my_echo will place its Writeln strings and that my_gui will use to get this string
pipe_write_handle

Therefore

  • my_gui creates both pipes
  • it launches my_echo by creating a Windows Process with special parameters to force my_echo to use our pipes for StdIn and StdOut
  • after my_eco is started
    • my_gui write a string
    • my_echo reads this string and writes the uppercase string and terminates
    • my_gui reads the uppercase string
    • everything is cleaned up



3 - The Delphi source code

3.1 - The echo program

The echo program Reads and Writes, and we also added logging to be able to monitor what's going on. Here is the program:

(*$APPTYPE CONSOLE *)

program p_console_read_write_one;
  var g_stringString;
  begin
    readln(g_string);
    writeln(UpperCase(g_string));
  end.



3.2 - The u_c_stdin_sdtout_exe unit

This unit creates the pipes, starts the console exe and can read and write strings to and from the console exe.



Since the pipe uses two handles, we placed them in a record:

 t_piperecord
           m_read_handlem_write_handletHandle;
         end;

And the handles are created with this function:

function f_create_pipe(Var pv_pipet_pipe): boolean;
  const k_pipe_buffer_size= 4096;
  begin
    with pv_pipe do
    begin
      // -- Create the pipe
      result:= CreatePipe(m_read_handlem_write_handlenilk_pipe_buffer_size);

      // -- recreate the handles, this time with the "inheritable" flag
      if result
        then result:= DuplicateHandle(GetCurrentProcessm_read_handle,
            GetCurrentProcess, @ m_read_handle, 0, True,
            DUPLICATE_CLOSE_SOURCE OR DUPLICATE_SAME_ACCESS);

      if result
        then result:= DuplicateHandle(GetCurrentProcessm_write_handle,
            GetCurrentProcess, @ m_write_handle, 0, True,
            DUPLICATE_CLOSE_SOURCE OR DUPLICATE_SAME_ACCESS);
    end;
  end// f_create_pipe

procedure close_pipe(p_pipet_pipe);
  begin
    with p_pipe do
    begin
      CloseHandle(m_read_handle);
      CloseHandle(m_write_handle);
    end;
  end// close_pipe

Note that:

  • the buffer size value included in the CreatePipe is irrelevant, since each ReadFile and WriteFile has its own size parameters
  • the strange DuplicateHandle call is mandatory to allow the handles to be inherited by the child process that will be created later.
    It seems that the same result could be obtained by using an NT security record during the CreatePipe call, but this record has to be created, initialized etc.
  • our creation function will be called to initialize both StdIn and StdOut pipes


The console .EXE is launched as a child process of the gui process, using the Windows CreateProcess function:

function CreateProcess(lpApplicationNamePChar
  lpCommandLinePChar;
  lpProcessAttributeslpThreadAttributesPSecurityAttributes;
  bInheritHandlesBOOL
  dwCreationFlagsDWORD
  lpEnvironmentPointer;
  lpCurrentDirectoryPChar
  const lpStartupInfoTStartupInfo;
  var lpProcessInformationTProcessInformation): BOOLstdcall;

where:

  • lpCommandLine is the console exe name and any command line parameters
  • bInheritHandles forces the new process to inherit the handles that can be specified in the lpStartupInfo parameter
  • lpStartupInfo is a record containing initial parameters for the new process, including the hStdIn, hStdOut and hStdErr handles (if bInheritHandles is True)
  • lpProcessInformation is the record filled by the CreateProcess call and containing the new hProcess handle. This handle will be used to check the process status, and eventually to kill it


The CreateProcess, sending and receiving have been encapsulated in a c_stdin_stdout_process CLASS. This is its definition (partial):

 c_stdin_stdout_processclass(c_basic_object)
                           m_exe_nameString;
                           m_stdin_pipem_stdout_pipet_pipe;

                           m_process_handleCardinal;
                           m_received_stringString;

                           constructor create_stdin_stdout_process(p_nameString);

                           function f_create_stdin_stdout_pipesBoolean;
                           procedure close_stdin_stdout_pipes;

                           function f_create_processBoolean;

                           procedure write_string(p_stringAnsiString);
                           procedure read_string;

                           procedure do_stop_process;

                           destructor Destroyoverride;
                         end// c_stdin_stdout_process

and:

  • m_exe_name: the path and name of the console exe which must be initialized before calling the main f_create_process method
  • m_stdin_pipe and m_stdout pipes are the two pipes which must be created before calling f_create_process
  • during f_create_process
    • the lpStartupInfo will be initialized:

      VAR l_startup_infoTStartupInfo;

      Zeromemory(@ l_startup_infoSizeOf(l_startup_info));
      with l_startup_info do
      begin
        cb:= sizeof(l_startup_info);
        dwFlags:= STARTF_USESHOWWINDOW or STARTF_USESTDHANDLES;
        hStdInput:= m_stdin_pipe.m_read_handle;
        hStdError:= m_stdout_pipe.m_write_handle;
        hStdOutput:= m_stdout_pipe.m_write_handle;
      end// with l_startup_info

    • CreateProcess is called
    • l_process_info.hProcess is saved
  • write_string can then be called to send a string to my_console.Readln
  • read_string will read a string and place it in m_received_string
  • kill_process can be used to kill the process


3.3 - The gui application

Here is a sample application:
  • clicking the "start_" button will
    • create a c_stdin_stdout_exe object
    • call the f_create_stdin_stdout pipes
    • fill the m_exe_name with p_console_read_write.exe
    • call f_create_process
  • clicking "do_write_" will sent Edit1.Text to p_console_read_write.exe
  • clicking "do_read_" will fetch the any string that p_console_read_write.Write has sent
  • kill can abort the p_console_read_write.exe


Here is a snapshot of the project:

stdin_stdout_pipe




4 - Improvements

This unit has been created to solve CGI server problem. In our case:
  • our simple server will send a "small" string (like 100 or 200 character long) to the CGI exe
  • the CGI.EXE only peforms one Readln, processes this string, which results in a single Writeln. And after this Writeln, the CGI.EXE reaches the end and the CGI.EXE terminates.
  • the CGI server will read back the "small" HTML page (also about 100 or 200 characters)
We did not spend much time to investigate other communication possibilities and on "industrial strength" issues:
  • are all the handles closed at the end, or if some exception is raised
  • what happens to programs which require many read / write sequences
  • can we transfer huge blocks (several megs). If the CONSOLE sends a big amount in one Write, is a single read enough ? What if the CGI.EXE calls several Writeln ?


Let us mention some possible solutions:
  • in the Borland Technical Information note, there is a read / write loop similar to:

    while WaitForSingleObject(m_process_handle, 0)<> WAIT_OBJECT_0 do
    begin
      // -- check if the process is still active
      GetExitCodeProcess(m_process_handlel_process_exit_code);
      if l_process_exit_code<> STILL_ACTIVE
         then break;

      // -- check to see if there is any data to read from stdout
      PeekNamedPipe(m_stdout_pipe.m_read_handle, @ m_reception_buffer, 1024,
          @ l_bytes_to_read_count, @ l_bytes_read_countnil);

      if l_bytes_read_count<> 0
        then begin
            FillChar(m_reception_buffersizeof(m_reception_buffer), 'z');

            // -- read by 1024 bytes chunks
            l_bytes_read_remaining_count:= l_bytes_read_count;
            while l_bytes_read_remaining_count>= 1023 do
            begin
              // -- read the stdout pipe
              ReadFile(m_stdout_pipe.m_read_handlem_reception_buffer, 1024, 
                  l_bytes_read_countnil);
              Dec(l_bytes_read_remaining_countl_bytes_read_count);
              handle_received(l_bytes_read_count);
            end;

            if l_bytes_read_remaining_count> 0
              then begin
                  ReadFile(m_stdout_pipe.m_read_handlem_reception_buffer,
                      l_bytes_read_remaining_countl_bytes_read_countnil);
                  display('read 'IntToStr(l_bytes_read_count));
                  handle_received(l_bytes_read_count);
                end;
          end// received some bytes

      // -- now write some
      WriteFile(m_stdin_pipe.m_write_handle'xxx'Length('xxx'), 
          l_bytes_write_countNil);
    end// while true

    This is the time-honored "busy waiting" loop, which blocks the execution alternately reading and writing.

  • the other suggestion is to use threads for reading and writing



5 - Download the Sources

Here are the source code files:

Those .ZIP files contain:
  • the main program (.DPR, .DOF, .RES), the main form (.PAS, .DFM), and any other auxiliary form
  • any .TXT for parameters
  • all units (.PAS) for units
Those .ZIP
  • are self-contained: you will not need any other product (unless expressly mentioned).
  • can be used from any folder (the pathes are RELATIVE)
  • will not modify your PC in any way beyond the path where you placed the .ZIP (no registry changes, no path creation etc).
To use the .ZIP:
  • create or select any folder of your choice
  • unzip the downloaded file
  • using Delphi, compile and execute
To remove the .ZIP simply delete the folder.



As usual:

  • please tell us at fcolibri@felix-colibri.com if you found some errors, mistakes, bugs, broken links or had some problem downloading the file. Resulting corrections will be helpful for other readers
  • we welcome any comment, criticism, enhancement, other sources or reference suggestion. Just send an e-mail to fcolibri@felix-colibri.com.
  • or more simply, enter your (anonymous or with your e-mail if you want an answer) comments below and clic the "send" button
    Name :
    E-mail :
    Comments * :
     

  • and if you liked this article, talk about this site to your fellow developpers, add a link to your links page ou mention our articles in your blog or newsgroup posts when relevant. That's the way we operate: the more traffic and Google references we get, the more articles we will write.



6 - References

Here are some pointers to other solutions found on the Web:
  • Using anonymous pipes to redirect standard input/output of a child process
      a C program written by Borland Staff - Code Central id= 10387 - October 08, 1999
  • http://www.elists.org/pipermail/delphi/2001-September/017041.html
      a forum message containing the Delphi version of the above C program
  • http://www.fulgan.com/delphi/DosPipes.zip
      a CLASS with StdIn StdOut redirection



7 - The author

Felix John COLIBRI works at the Pascal Institute. Starting with Pascal in 1979, he then became involved with Object Oriented Programming, Delphi, Sql, Tcp/Ip, Html, UML. Currently, he is mainly active in the area of custom software development (new projects, maintenance, audits, BDE migration, Delphi Xe_n migrations, refactoring), Delphi Consulting and Delph training. His web site features tutorials, technical papers about programming with full downloadable source code, and the description and calendar of forthcoming Delphi, FireBird, Tcp/IP, Web Services, OOP  /  UML, Design Patterns, Unit Testing training sessions.
Created: nov-04. Last updated: dec-2019 - 103 articles, 239 .ZIP sources, 1292 figures
Contact : Felix COLIBRI - Phone: (33)1.42.83.69.36 / 06.87.88.23.91 - email:fcolibri@felix-colibri.com
Copyright © Felix J. Colibri   http://www.felix-colibri.com 2004 - 2019. All rigths reserved
Back:    Home  Papers  Training  Delphi developments  Links  Download
the Pascal Institute

Felix J COLIBRI

+ Home
  + articles_with_sources
    + database
    + web_internet_sockets
    + rest_services
    + oop_components
    + uml_design_patterns
    + debug_and_test
    + graphic
    + controls
    + colibri_utilities
    + colibri_helpers
      – windows_environment
      – stdin_stdout
    + delphi
    + IDE
    + firemonkey
    + compilers
    + vcl
  + delphi_training
  + delphi_developments
  + sweet_home
  – download_zip_sources
  + links
Contacts
Site Map
– search :

RSS feed  
Blog