menu
  Home  ==>  papers  ==>  colibri_utilities  ==>  the_coliget_search_engine   

ColiGet: Full Text Expression Search - Felix John COLIBRI.

  • abstract : find the ASCII files satisfying an expression search condition
  • key words : file search, KMP, Knuth Morisson Pratt, expression evaluation, file search
  • software used : Windows XP, Delphi 6
  • hardware used : Pentium 1.400Mhz, 256 M memory, 140 G hard disc
  • scope : Delphi 1 to 2005 for Windows, Kylix
  • level : Delphi developer
  • plan :


1 - Introduction

Among all the files on my disk, I wanted to find the one presenting the Abstract Factory design pattern. The papers was supposed to talk about this pattern, but not about the other Gof design pattern.

All I was looking for was:

abstract factory BUT NOT the other patterns

In other words:

 
  - in the .HTML pages from my c:\download\ directory
  - tell me which files contain
       ("Gang Of Four" OR gof or GAMMA AND Helm ) And "abstract factory" AND NOT Strategy
 



Many full text search tools exist. We even have the fabulous "find in files" in the Delphi IDE, which I use many times every day. However we failed to find an engine that offered a simple expression syntax, and could be included in a Delphi application.

We did not want a simple "regular expression" engine only, which usually does not include logical operator evaluation. Regex tools often assume that their simple and nice ?*!+^#-/&~?|+$=%@ syntax is soo good that is does cover everybody's needs. Fine.

On the other hand, we truly dislike the kiddy syntax of all the Web search engines or the MSDN tool (+, -, no parenthesis nesting, implicit ANDing or ORing etc). No doubt they had rooms full of shrinks to tell them what THE best syntax was supposed to be. And everybody has his own best syntax. Anyway, I don't like those syntaxes all the same. And the dialogs with several edits and combo or checkboxes to select AND, OR, "all the words", "Uppercase", "whole word" etc are a pain in the neck. I am certainly biased concerning logical expressions, but for me the Pascal syntax has unrivaled power and simplicity. And I consider it outrageous to believe that non-programmers are not smart enough to grasp the rules.

In any case, we built our own engine. Our objective was to

  • find the text in a directory
  • allow a "natural" pascal like search syntax



2 - The searcher program

2.1 - The Basic algorithm

We will load all the files in the specified directory, and check whether the satisfy or not the condition. In more detail:
  • the user inputs the path, the extension filter, the search condition
  • for all the files satisfying the path / filter condition
    • load the file
    • check whether the literals are present or not
    • evaluate the boolean expression
    If the condition is satisfied, print out the path and the name of the file


2.2 - Finding the files

We use the FindFirst, FindNext DOS functions to get the files. Since we use this search extensively in all our utilities, we incorporated the search in a unit with a callback in the main program.

The search is performed by calling:

procedure handle_all_files_recursive(p_levelInteger;
    p_strict_pathp_extensionString;
    p_dir_handling_typest_dir_handling_types;
    p_pr_handle_filet_pr_handle_filep_pt_dataPointer);

where:

  • p_strict_path and p_extension are the search specification
  • p_dir_handling_types is a SET allowing different search options (file or directory, debugging display, recursion)
  • p_pr_handle_file is the procedural type
  • p_pt_data is a general purpose pointer
The call-back and the search options are defined as follows:

 t_pr_handle_fileprocedure(p_levelIntegerp_dirp_file_nameString;
    p_pt_dataPointer);

 t_dir_handling_type= (e_dir_recursive,
     e_dir_display_dire_dir_display_filee_dir_display_all_file,
     e_dir_display_debuge_dir_indent,
     e_dir_handle_dir_beforee_dir_handle_dir_after,
     e_dir_handle_dire_dir_handle_file);
 t_dir_handling_typesset of t_dir_handling_type;



An in the main Form, the call back will load each file and analyze it:

procedure check_file_call_back(p_levelIntegerp_pathp_file_nameString;
    p_pt_dataPointer);
  begin
    with tStringList.Create do
    begin
      LoadFromFile(p_path'\'p_file_name);

      // -- ... check the search condition

      Free;
    end// with tStringList
  end// check_file_call_back



In addition, the selection of the initial path, and eventually the individual file and extension filter, are performed with the tFileListbox, tDirectoryListbox and tFilterComboBox. I never understood why those have been classified "obsolete" or "deprecated". We use them all the time.



So when the user selects a file in the tFileListbox, we check the file:

procedure TForm1.FileListBox1Click(SenderTObject);
  var l_pathl_nameString;
  begin
    l_path:= DirectoryListBox1.Directory;
    with FileListbox1 do
      l_name:= Items[ItemIndex];

    check_file_call_back(0, l_path'\'l_nameNil);
  end// FileListBox1Click



When he changes the path in the tDirectoryListbox, we update a global variable:

procedure TForm1.DirectoryListBox1Change(SenderTObject);
  begin
    with DirectoryListBox1 do
      g_path:= GetItemPath(ItemIndex)+ '\';
    Caption:= g_path;
  end// DirectoryListBox1Change

And when he clicks the "directory", or "directory_recursive" buttons, we call the FindFirst routine:

procedure handle_the_files(p_set_of_dir_handling_typet_dir_handling_types);
  var l_extensionString;
      l_pathString;
  begin
    l_path:= Form1.DirectoryListBox1.Directory;
    l_extension:= UpperCase(Form1.extension_edit_.Text);
    Delete(l_extension, 1, 1);

    handle_all_files_recursive(1, l_pathl_extension,
        p_set_of_dir_handling_typecheck_file_call_backNil);
  end// handle_the_files

procedure TForm1.all_dir_Click(SenderTObject);
    // -- all the file of this folder (non rec)
  begin
    handle_the_files([e_dir_handle_file]);
  end// btn_all_dirClick

procedure TForm1.all_dir_recursive_Click(SenderTObject);
    // -- all the file of this folder (rec)
  begin
    handle_the_files([e_dir_recursivee_dir_handle_file]);
  end// btn_all_dir_recursClick



2.3 - The String Search

The search for a single word is performed with a Boyer Moore Horspool algorithm. The reference section presents many links to papers explaining how this algorithm works and URLS with many Pascal / Delphi implementations. Alternately, just jump to GOOGLE and ask for:

    +"Boyer Moore Horspool"+ "Delphi".

Maybe one day they will offer Pascal syntax ?



The search for one string in the text is performed in two steps:

  • we build a table which tells us how many character to skip when the current position in the text does not correspond to the searched string
  • we start the search from the first index in the text, and use this jump table whenever the current text position does not match the string
Notice that:
  • the table build up is a one shot construction, whereas the search could be performed several time
  • the search can be aborted as soon as we find an occurence, or we could count the occurences. Alternately we could start from any position in the text (to implement a "next" search)
  • if we want to look up another string, the table will have to be reconstructed from scratch
In our case, we are only interested to know whether the text contains a string or not.



The search class is defined as:

 c_text_searcherclass(c_basic_object)
                    m_text_to_searchString;
                    m_text_lengthInteger;

                    m_pattern_to_findString;
                    m_pattern_lengthInteger;

                    m_jump_tablearray[#0..#255] of Integer;
                    m_jump_valueinteger;

                    m_search_indexInteger;

                    Constructor create_text_searcher(p_nameString);

                    procedure initialize_text(p_text_to_searchString);
                    procedure initialize_pattern_to_search(p_pattern_to_searchString);
                    function f_index_ofInteger;

                    function f_found_string(p_pattern_to_searchString): Boolean;
                  end// c_text_searcher

And:

  • initialize_text set the text string
  • initialize_pattern_to_search builds the jump table. The .Zip contains the detail, which we do not include here, since the purpose of the paper is not to present the BMH algorithm
  • f_index_of behaves like Pos by returning the index of the string or 0 if none is found
  • f_found_string is a simple wrapper which initializes and returns the position of a match if any


2.4 - The Expression Evaluator

We know how to load the text. We know how to check the presence of a single string in the text. All that remains to be done is to check an expression.

To do so, we simply test the presence of each literal string, and combine the boolean results using a simple boolean evaluator.



As explained already, we chose a simple Pascal syntax:

  • boolean operators AND OR NOT (case insensitive)
  • Pascal priority (parentheses, NOT, AND, OR)
  • separators: spaces, parentheses
  • literal without quoting, unless they contain spaces, case sensitive, unless told to find all cases
So we will have to parse the expression, and evaluate it on each text.



When we analyze several texts (nearly 400 for some of the web sites we maintain), the repeated parsing of the expression could become costly.

So we chose to tokenize the expression, performing the query analysis only once. The request will then evaluate this tokenized query, avoiding the costly string analysis. The spirit is somehow similar to the Prepare / Open mechanism in Sql request processing. We are aware that the parsing of the string is in no way as complex as the NP-complete query optimization, but the idea is the same.



In the request, we have two kinds of symbols:

  • the operators (NOT AND OR and the parentheses)
  • the literals (interbase, "delphi asp.net")
We chose to use two arrays
  • one for the literal strings
  • the other for the tokens of the request. In this array
    • the positive values are the position of a literal in the literal tStringList
    • the negative values correspond to the operators (-1 for AND, -2 for OR and so on)
Here is an example:



The lexical analysis is quite standard stuff. The only special treatment is the first pass to count the literals and key words, in order to allocate the token array (another option would have been to use a tList, or redimension the array for each new token).

Here is the main loop of the tokenizer:

procedure fill_identifier_list_and_token_array;
  var l_request_indexInteger;
      l_token_countInteger;

  procedure analyze_identifier;
    var l_start_indexl_identifier_lengthInteger;
        l_identifierString;
        l_indexInteger;
        l_lowercase_identifierString;
    begin
      l_start_index:= l_request_index;
      while (l_request_index<= l_request_length)
          and (p_request[l_request_indexin k_lettersdo
        Inc(l_request_index);

      l_identifier_length:= l_request_indexl_start_index;
      SetLength(l_identifierl_identifier_length);
      Move(p_request[l_start_index], l_identifier[1], l_identifier_length);

      l_lowercase_identifier:= LowerCase(l_identifier);

      // -- check whether this is AND OR NOT
      if l_lowercase_identifier'and'
        then m_oa_tokens[l_token_count]:= k_token_AND
          else
            if l_lowercase_identifier'or'
              then m_oa_tokens[l_token_count]:= k_token_OR
                else
                  if l_lowercase_identifier'not'
                    then m_oa_tokens[l_token_count]:= k_token_NOT
                    else begin
                        // -- the identifier
                        l_index:= m_c_request_identifiers.Add(l_lowercase_identifier);
                        m_oa_tokens[l_token_count]:= l_index;
                      end;
      m_oa_token_positions[l_token_count]:= l_start_index;
      m_oa_token_lengths[l_token_count]:= l_identifier_length;

      Inc(l_token_count);
    end// analyze_identifier

  procedure analyze_litteral_string;
    begin
      // -- ... not shown
    end// analyze_litteral_string

  begin // fill_identifier_list_and_token_array
    l_request_index:= 1;
    l_token_count:= 0;

    while (l_request_index<= l_request_lengthand (m_error_index= 0) do
    begin
      case p_request[l_request_indexof
        'a'..'z''A'..'Z',
          '0'..'9' : analyze_identifier;
        '"' : analyze_litteral_string;
        ' ' : Inc(l_request_index);
        '(' : begin
                m_oa_tokens[l_token_count]:= k_token_opening_parenthesis;
                m_oa_token_positions[l_token_count]:= l_request_index;
                m_oa_token_lengths[l_token_count]:= 1;
                Inc(l_request_index);
                Inc(l_token_count);
              end;
        ')' : begin
                m_oa_tokens[l_token_count]:= k_token_closing_parenthesis;
                m_oa_token_positions[l_token_count]:= l_request_index;
                m_oa_token_lengths[l_token_count]:= 1;
                Inc(l_request_index);
                Inc(l_token_count);
              end;
        '.' : begin
                m_oa_tokens[l_token_count]:= k_token_end;
                m_oa_token_positions[l_token_count]:= l_request_index;
                m_oa_token_lengths[l_token_count]:= 1;
                Inc(l_request_index);
                Inc(l_token_count);
              end;
        else
          m_error_index:= l_request_index;
          m_error_length:= 1;
          Break;
      end// case
    end// while
  end// fill_identifier_list_and_token_array



Notice that special care has been paid to error handling. When the user forgets to close a parenthesis, we want to be able to display the error as accurately as possible.



The tokenized expression eases the computation, but we cannot just evaluate the result from the left to right because of operator precedence. In an arithmetic computation:

 
  2+ 3 * 4
 

we are supposed to multiply first, and then add. In Pascal, which is the syntax we chose to follow, the order is:

 
    parentheses, if any
    NOT  
    AND  
    OR  
 



When we look for a course with

  • either UML and Design Patterns
  • or any design material but without Use Cases
we would ask for:

 
    (uml AND pattern) OR NOT "use case" AND design
 

In this example

  • (uml AND pattern) is evaluated first
  • NOT "use case" comes next
  • then (NOT "use case" ) AND design
  • and finally the OR
The request could have been written in an equivalent way as:

 
    (uml AND pattern) OR ( design AND ( NOT "use case" ))  
 



In order to handle the operation in the correct order, we have to analyze the expression, to first find the innermost parentheses, then look if any NOT is present etc. This is called parsing.

Parsing is computed according to a grammar which embodies the operator precedence rules.

Our operator precedence rules are given by the following standard IEBNF grammar (Indented Extended Backus Naur):

expression= term { OR term } .
    term= factor { AND factor } .
        factor= STRING | NOT factor | '(' expression ')' .


The evaluation of the expression following our grammar can be handled in several ways:

  • implement a stack machine to evaluate the expression (byte-code interpreter is the politically correct term today)
  • build an expression tree, and walk up and down this tree to get the boolean result
  • use a top down parser which computes the boolean result during the travel along the expression
The possibilities are admirably presented in Niklaus WIRTH's book.

We chose the last possibility. And here is the parser / evaluator (error handling and tracing not shown):

function c_evaluate_request.f_evaluate_request(p_c_text_searcherc_text_searcher): boolean;
  type t_symbol_typeInteger;
  var // -- the position in the tokenized request
      l_token_indexInteger;
      // -- the current symbol
      l_symbol_typet_symbol_type;

  procedure read_next_symbol;
    begin
      l_symbol_type:= m_oa_tokens[l_token_index];
      Inc(l_token_index);
    end// read_next_symbol

  procedure display_evaluation_error(p_errorString);
    begin
      // ....
    end// display_evaluation_error

  function f_evaluate_expressionBoolean;

    function f_evaluate_termBoolean;

      function f_evaluate_factorBoolean;
        var l_identifierString;
        begin
          case l_symbol_type of
            k_token_opening_parenthesis :
              begin
                // -- skip (
                read_next_symbol;

                Result:= f_evaluate_expression;

                if l_symbol_typek_token_closing_parenthesis
                  then read_next_symbol
                  else display_evaluation_error('in "(expr)" manque )');
              end;

            k_token_NOT :
              begin
                read_next_symbol;

                Result:= NOT f_evaluate_factor;
                // -- do NOT read next symbol: was read by compile_expression
              end;

            else
              if l_symbol_type>= 0
                then begin
                    l_identifier:= m_c_request_identifiers[l_symbol_type];

                    // -- look if the symbol is in the text
                    Result:= p_c_text_searcher.f_found_string(l_identifier);
                    read_next_symbol;
                  end
                else begin
                    // -- avoid warnings
                    Result:= False;
                    display_evaluation_error('attend fact: k, var, (: 'IntToStr(l_symbol_type));
                  end;
          end// case
        end// f_evaluate_factor

      begin // f_evaluate_term
        Result:= f_evaluate_factor;

        while l_symbol_typek_token_AND do
        begin
          read_next_symbol;
          Result:= Result AND f_evaluate_factor;
        end// while l_symbol_type
      end// f_evaluate_term

    begin // f_evaluate_expression
      Result:= f_evaluate_term;

      while l_symbol_typek_token_OR do
      begin
        read_next_symbol;

        Result:= Result OR f_evaluate_term;
      end// while
    end// f_evaluate_expression

  begin // f_evaluate_request
    l_token_index:= 0;
    read_next_symbol;

    Result:= f_evaluate_expression;
  end// f_evaluate_request



So the evaluator is encapsulated in the following class:

 c_request_evaluatorclass(c_basic_object)
                        m_requestString;
                        m_c_request_identifierstStringList;
                        m_oa_tokensArray Of Integer;

                        Constructor create_request_evaluator(p_nameString);

                        procedure tokenize_request(p_requestString);
                        function f_display_token_type(p_token_typeInteger): String;
                        procedure redisplay_request;

                        function f_evaluate_request(p_c_text_searcherc_text_searcher): Boolean;
                        Destructor DestroyOverride;
                      end// Virtual;



2.5 - The main Project

The unit organzation is the following:

 
root\
  colibri_helpers\
    classes\
      u_c_basic_object.pas
      u_c_display.pas
      u_c_log.pas
      u_c_text_searcher.pas
    units\
      u_dir.pas
      u_handle_files_in_dirs.pas
      u_strings.pas
      u_types_constants.pas
  colibri_utilities\
    programs\
      coliget_text_search\
        data\
          sample_request.txt
        p_coliget_text_search.dpr
        u_coliget_text_search.pas
        u_coliget_text_search.dfm
        u_c_request_evaluator.pas

The main window contains:

  • the directory listbox, and file listbox for easy directory and file selection
  • an Edit for the request input (we also added a Listbox with predefined queries loaded from a "sample_query.txt" file)
  • a tMemo for debugging trace and result display


Here is a snapshot of the main form (with a request on the pages of this site):



2.6 - Mini HowTo

To use the program:
   enter the search request
   navigate in the DirectoryListbox to the desired directory
   set the file extension
   click "all_dir" or "all_dir_recursive"
   delphi will tell you which files contain the requested string combination



3 - Improvements

We have always chosen to present simple projects that you can easily understand and then improve to your heart's content.

So it is a small wonder that the improvements are numerous. Let me just mention a few of them:

  • file search
    • improve the unit: we could implement a CLASS to encapsulate FindFirst and FindNext. The descendent would then contain the method performing the computations on the files (instead of using the non-object call-back)
    • include .ZIP decompression
  • string search algorithm
    • use a multi string search algorithm instead of starting the search from the beginning of the text for each literal string
    • if the expression contains several times the same string, only perform the search once (doing the search before the tree evaluation)
    • add regular expressions, allowing some "fuzziness" in the search with some kind of wildcards (*? etc)
    • include the file path, name and extension in the search request. In our case, we have implemented this at the call-back level, avoiding for instance to look into the .HTML frame pages which do not contain any valuable text
    • allow qualified search (words only, allow user separator specification)
    • in addition to simple literals, add category specifications (image="interbase.png" and text="database"), and why not a full interpreter with loops, test etc. Alternately an SQL non procedural type language
  • if several requests are performed on the same file set, implement some cache mechanisms (for the files, for the jump tables ...)
  • for big files, perform the search on a text "abstract". We could easily compute the list of unique words, and search this much simpler text. This would miss the word combinations however
    Also add "semantic" abstracts, which could add words related to the content of the file but not present in the current text (similar to the HTML meta tags)
    This would lead to content analysis, with some kind of semantic processing
  • compute some success factor, at least a count of the string for simple AND or OR requests
  • adapt to the search of the pages from a Web Site (and peforms the special computations to avoid HTML syntax, like in "<B>D</B>elphi" for Delphi).
  • present the result in a treeview, with possibility to load the files. Quite naturally highlight the words from the request. And all in HTML or similar format. And allow "next" navigation, or even a synchronized list of all occurences
  • optimization has been our lowest priority. Before starting to twiggle the code here or there, I would do some measurements. They will depend on the kind of processing (many small files, of few big texts, simple or complex queries etc). As it is, the tool is quite satisfactory, and we will only try to spend time on optimization if the performance starts to slow down other developments.



4 - Download the Sources

Here are the source code files: The .ZIP file(s) contain:
  • the main program (.DPR, .DOF, .RES), the main form (.PAS, .DFM), and any other auxiliary form
  • any .TXT for parameters, samples, test data
  • all units (.PAS) for units
Those .ZIP
  • are self-contained: you will not need any other product (unless expressly mentioned).
  • for Delphi 6 projects, can be used from any folder (the pathes are RELATIVE)
  • will not modify your PC in any way beyond the path where you placed the .ZIP (no registry changes, no path creation etc).
To use the .ZIP:
  • create or select any folder of your choice
  • unzip the downloaded file
  • using Delphi, compile and execute
To remove the .ZIP simply delete the folder.

The Pascal code uses the Alsacian notation, which prefixes identifier by program area: K_onstant, T_ype, G_lobal, L_ocal, P_arametre, F_unction, C_lass etc. This notation is presented in the Alsacian Notation paper.



As usual:

  • please tell us at fcolibri@felix-colibri.com if you found some errors, mistakes, bugs, broken links or had some problem downloading the file. Resulting corrections will be helpful for other readers
  • we welcome any comment, criticism, enhancement, other sources or reference suggestion. Just send an e-mail to fcolibri@felix-colibri.com.
  • or more simply, enter your (anonymous or with your e-mail if you want an answer) comments below and clic the "send" button
    Name :
    E-mail :
    Comments * :
     

  • and if you liked this article, talk about this site to your fellow developpers, add a link to your links page ou mention our articles in your blog or newsgroup posts when relevant. That's the way we operate: the more traffic and Google references we get, the more articles we will write.



5 - Conclusion

We presented a simple Delphi project which combines a directory lookup with a quick string search routine and a simple boolean evaluator, in order to find all files containing a string expression.




6 - References

  • Niklaus Wirth
    Algorithms+ Data Structure= Programming
        Prentice Hall - 1976 - ISBN 0-13-022418-9

  • Nigel HORSPOOL
        Boyer Moore Horspool

  • G H GONNET, R BAEZA YATES
        Handbook of Algorithms and Data Structures
        Addison Wesley
        Boyer-Moore-Horspool text searching

  • Jody R. CAIRNS - Dept. of Fisheries and Oceans
         Boyer-Moore-Horspool pattern searching algorithm (from GONNET-YATES)
         Boyer More Horspool

  • Mike LIDELL
        String Searching Thesis - Aug 1997
        Thesis

  • Angus JOHNSON
        TSearch & TFileSearch components - May 2003
        Delphi implementation of the Boyer-Moore-Horspool search algorithm.
        Search components

  • Turbo POWER
        Tip 30: optimized search with Boyer Moore

  • Julian BUCKNALL
        Delphi Magazine - Delphi Magazine, Issue 36, August 1998
        Boyer-Moore algorithm compared to Delphi Pos
        The Delphi Magazine

  • Earl F GLYNN II - Efg lab
        Delphi Algorithms



7 - Other Papers with Source and Links

Database
database reverse engineering Extraction of the Database Schema by analyzing the content of the application's .DFMs
sql parser Parsing SQL requests in Delphi, starting from an EBNF grammar for SELECT, INSERT and UPDATE
ado net tutorial a complete Ado Net architectural presentation, and projects for creating the Database, creating Tables, adding, deleting and updating rows, displaying the data in controls and DataGrids, using in memory DataSets, handling Views, updating the Tables with a DataGrid
turbo delphi interbase tutorial develop database applications with Turbo Delphi and Interbase. Complete ADO Net architecture, and full projects to create the database, the Tables, fill the rows, display and update the values with DataGrids. Uses the BDP
bdp ado net blobs BDP and Blobs : reading and writing Blob fields using the BDP with Turbo Delphi
interbase stored procedure grammar Interbase Stored Procedure Grammar : The BNF Grammar of the Interbase Stored Procedure. This grammar can be used to build stored procedure utilities, like pretty printers, renaming tools, Sql Engine conversion or ports
using interbase system tables Using InterBase System Tables : The Interbase / FireBird System Tables: description of the main Tables, with their relationship and presents examples of how to extract information from the schema
eco tutorial Writing a simple ECO application: the UML model, the in memory objects and the GUI presentation. We also will show how to evaluate OCL expressions using the EcoHandles, and persist the data on disc
delphi dbx4 programming the new dbExpress 4 framework for RAD Studio 2007 : the configuration files, how to connect, read and write data, using tracing and pooling delegates and metadata handling
blackfishsql using the new BlackfishSql standalone database engine of RAD Studio 2007 (Win32 and .Net) : create the database, create / fill / read Tables, use Pascal User Defined Functions and Stored Procedures
rave pdf intraweb how to produce PDF reports using Rave, and have an Intraweb site generate and display .PDF pages, with multi-user access
embarcadero er studio Embarcadero ER Studio tutorial: how to use the Entity Relationship tool to create a new model, reverse engineer a database, create sub-models, generate reports, import metadata, switch to Dimensional Model
Web
sql to html converting SQL ascii request to HTML format
simple web server a simple HTTP web Server and the corresponding HTTP web Browser, using our Client Server Socket library
simple cgi web server a simple CGI Web Server which handles HTML <FORM> requests, mainly for debugging CGI Server extension purposes
cgi database browser a CGI extension in order to display and modify a Table using a Web Browser
whois a Whois Client who requests information about owners of IP adresses. Works in batch mode.
web downloader an HTTP tool enabling to save on a local folder an HTML page with its associated images (.GIF, .JPEG, .PNG or other) for archieving or later off-line reading
web spider a Web Spider allowing to download all pages from a site, with custom or GUI filtering and selection.
asp net log file a logging CLASS allowing to monitor the Asp.Net events, mainly used for undesrtanding, debugging and journaling Asp.Net Web applications
asp net viewstate viewer an ASP.NET utility displaying the content of the viewtate field which carries the request state between Internet Explorer and the IIS / CASSINI Servers
rss reader the RSS Reader lets you download and view the content of an .RSS feed (the entry point into somebody's blog) in a tMemo or a tTreeView. Comes complete with an .HTML downloader and an .XML parser
news message tree how to build a tree of the NNTP News Messages. The downloaded messages are displayed in tListBox by message thread (topic), and for each thread the messages are presented in a tTreeVi"ew
threaded indy news reader a NewsReader which presents the articles sorted by thread and in a logical hierarchical way. This is the basic Indy newsreader demo plus the tree organization of messages
delphi asp net portal programming presentation, architecture and programming of the Delphi Asp Net Portal. This is a Delphi version of the Microsoft ASP.NET Starter Kit Web Portal showcase. With detailed schemas and step by step presentation, the Sql scripts and binaries of the Database
delphi web designer a tiny Delphi "RAD Web Designer", which explains how the Delphi IDE can be used to generate .HTML pages using the Palette / Object Inspector / Form metaphor to layout the page content
intraweb architecture the architecture of the Intraweb web site building tool. Explains how Delphi "rad html generator" work, and presents the CLASS organization (UML Class diagrams)
ajax tutorial AJAX Tutorial : writing an AJAX web application. How AJAX works, using a JavaScript DOM parser, the Indy Web Server, requesting .XML data packets - Integrated development project
asp net master pages Asp.Net 2.0 Master Pages : the new Asp.Net 2.0 allow us to define the page structure in a hierarchical way using Master Pages and Content Pages, in a way similar to tForm inheritance
delphi asp net 20 databases Asp.Net 2.0 and Ado.Net 2.0 : displaying and writing InterBase and Blackfish Sql data using Dbx4, Ado.Net Db and AdoDbxClient. Handling of ListBox and GridView with DataSource components
asp net 20 users roles profiles Asp.Net 2.0 Security: Users, Roles and Profiles : Asp.Net 2.0 offers a vaslty improved support for handling security: new Login Controls, and services for managing Users, grouping Users in Roles, and storing User preferences in Profiles
bayesian spam filter Bayesian Spam Filter : presentation and implementation of a spam elimination tool which uses Bayesian Filtering techniques
TCP/IP
tcp ip sniffer project to capture and display the packets travelling on the Ethernet network of your PC.
sniffing interbase traffic capture and analysis of Interbase packets. Creation of a database and test table, and comparison of the BDE vs Interbase Express Delphi components
socket programming the simplest Client Server example of TCP / IP communication using Windows Sockets with Delphi
delphi socket architecture the organization of the ScktComp unit, with UML diagrams and a simple Client Server file transfer example using tClientSocket and tServerSocket
Object Oriented Programming Components
delphi virtual constructor VIRTUAL CONSTRUCTORS together with CLASS references and dynamic Packages allow the separation between a main project and modules compiled and linked in later. The starting point for Application Frameworks and Plugins
delphi generics tutorial Delphi Generics Tutorial : using Generics (parameterized types) in Delphi : the type parameter and the type argument, application of generics, constraints on INTERFACEs or CONSTRUCTORs
UML Patterns
the lexi editor delphi source code of the Gof Editor: Composite, Decorator, Iterator, Strategy, Visitor, Command, with UML diagrams
factory and bridge patterns presentation and Delphi sources for the Abstract Factory and Bridge patterns, used in the Lexi Document Editor case study from the GOF book
gof design patterns delphi source code of the 23 Gof (GAMMA and other) patterns: Composite, Decorator, Iterator, Strategy, Visitor, Command
Debug and Test
Graphic
delphi 3d designer build a 3d volume list, display it in perspective and move the camera, the screen or the volumes with the mouse.
writing a flash player build your own ShockWave Flash movie Player, with pause, custom back and forward steps, snapshots, resizing. Designed for analyzing .SWF demos.
Utilities
the coliget search engine a Full Text Search unit allowing to find the files in a directory satisfying a complex string request (UML AND Delphi OR Patters)
treeview html help viewer Treeview .HTML Help Viewer : the use of a Treeview along with a WebBrowser to display .HTML files alows both structuring and ordering of the help topics. This tool was used to browse the Delphi PRISM Wiki help.
Delphi utilities
delphi net bdsproj structure and analysis of the .BDSPROJ file with the help of a small Delphi .XML parser
dccil bat generator generation of the .BAT for the Delphi DCCIL command line compiler using the .BDSPROJ
dfm parser a Delphi Project analyzing the .DFM file and building a memory representation. This can be used for transformations of the form components
dfm binary to text a Delphi Project converting all .DFM file from a path from binary to ascii format
component to code generate the component creation and initialization code by analyzing the .DFM. Handy to avoid installing components on the Palette when examining new libraries
exe dll pe explorer presents and analyzes the content of .EXE and .DLL files. The starting point for extracting resources, spying .DLL function calls or injecting additional functionalities
dll and process viewer analyze and display the list of running processes, with their associated DLLs and Memory mapped files (Process Walker)
Controls
find memo a tMemo with "find first", "find next", "sort", "save" capabilities
Helper units
windows environment read and write Windows Environment strings
stdin stdout send and receive strings from a GUI application to a CONSOLE application




8 - The author

Felix John COLIBRI works at the Pascal Institute. Starting with Pascal in 1979, he then became involved with Object Oriented Programming, Delphi, Sql, Tcp/Ip, Html, UML. Currently, he is mainly active in the area of custom software development, Delphi Consulting and Delph training, and is a frequent speaker at Borland Developer Conferences. His web site features tutorials, technical papers about programming with full downloadable source code, and the description and calendar of forthcoming Delphi, Interbase, Asp.Net, Ado.Net and OOP  /  UML training sessions.
Created: nov-04. Last updated: oct-10 - 61 articles, 121.ZIP sources, 954 figures
Copyright © Felix J. Colibri   http://www.felix-colibri.com 2004 - 2010. All rigths reserved
Back:    Home  Papers  Training  Delphi developments  Links  Download
the Pascal Institute

Felix J COLIBRI

+ Home
  + articles_with_sources
    + database
    + web_internet_sockets
    + oop_components
    + uml_design_patterns
    + debug_and_test
    + graphic
    + controls
    + colibri_utilities
      – delphi_net_bdsproj
      – dccil_bat_generator
      – coliget_search_engine
      – dfm_parser
      – dfm_binary_to_text
      – component_to_code
      – exe_dll_pe_explorer
      – dll_process_viewer
      – the_alsacian_notation
      – html_help_viewer
      – cooking_the_code
      – events_record_playback
    + colibri_helpers
    + delphi
    + compilers
  + delphi_training
  + delphi_developments
  + sweet_home
  – download_zip_sources
  + links
Contacts
Site Map
– search :

RSS feed  
Blog