Tailoring the Build Process to Generate the Processing Script

It is relatively easy to integrate CScout into an existing IDE (such as Eclipse) or to provide an alternative method for specifying workspaces by directly creating a processing script from existing Makefiles. A processing script (what results from compiling a workspace file) is a C file containing a number of #pragma preprocessor directives. CScout uses the following pragmas:
#pragma echo "STRING"
Will display the STRING on CScout's standard output when that part of the file is reached.

Example:

#pragma echo "Processing workspace date\n"
#pragma ro_prefix "STRING"
Will add STRING to the list of filename prefixes that mark read-only files. This is a global setting.

Example:

#pragma ro_prefix "C:\gcc"
#pragma define_immutable macro definition
Will define a macro similarly to a #define command, but mark the definition as one that cannot be later be undefined or redefined. This is useful for defining macros that handle compiler extensions so that code cannot change them.
#pragma project "STRING"
Will set the current project to STRING. All identifiers and files processed from then on will belong to the given project.

Example:

#pragma project "date"
#pragma block_enter
Will enter a nested scope block. Two blocks are supported, the first block_enter will enter the project scope (linkage unit); the second encountered nested block_enter will enter the file scope (compilation unit).

#pragma block_exit
Will exit a nested scope block. The number of block_enter pragmas should match the number of block_exit pragmas and there should never be more than two block_enter pragmas in effect.

#pragma process "STRING"
Will analyze (CScout's equivalent to compiling) the C source file named STRING.

Example:

#pragma process "date.d"
#pragma pushd "STRING"
Will set the current directory to STRING, saving the previous current directory in a stack. From that point onward, all relative file accesses will start from the given directory.

Example:

#pragma pushd "cp"
#pragma popd
Will restore the current directory to the one in effect before a previously pushed directory. The number of pushd pragmas should match the number of popd pragmas.

#pragma includepath "STRING"
Will add STRING to the list of directories used for searching included files (the include path).

Example:

#pragma includepath "/usr/lib/gcc-lib/i386-redhat-linux/2.96/include"
#pragma clear_include
Will clear the include path, allowing the specification of a new one.

#pragma clear_defines
Will clear all defined macros allowing the specification of new ones. Should normally be executed before processing a new file. Note that macros can be defined using the normal #define C preprocessor directive.
The following is a complete example of a CScout processing script.
// workspace bin
#pragma echo "Processing workspace bin\n"
#pragma ro_prefix "/usr/include"
#pragma echo "Entering directory /usr/src/bin"
#pragma pushd "/usr/src/bin"
// project date
#pragma echo "Processing project date\n"
#pragma project "date"
#pragma block_enter
#pragma echo "Entering directory date"
#pragma pushd "date"
// file date.c
#pragma echo "Processing file date.c\n"
#pragma block_enter
#pragma clear_defines
#pragma clear_include
#include "/home/dds/src/cscout/cscout_defs.h"
#include "/home/dds/src/cscout/cscout_incs.h"
#pragma process "date.c"
#pragma block_exit
#pragma echo "Done processing file date.c\n"
#pragma echo "Exiting directory date\n"
#pragma popd
#pragma block_exit
#pragma echo "Done processing project date\n"
#pragma echo "Exiting directory /usr/src/bin\n"
#pragma popd
#pragma echo "Done processing workspace bin\n"

Case Study: Processing the FreeBSD Kernel (the Hard Way)

As a further example consider the steps for applying CScout on the FreeBSD kernel, before the existence of the csmake command.
  1. Configure a LINT or GENERIC version of each architecture's kernel.
    Example: config LINT
  2. Go to the compilation directory, update the dependencies (make depend) and compile (make). This step is used to create all automatically generated C and header files. Also during this step note the include path used, in order to provide CScout with the same specification.
  3. Remove the generated object files to force a make operation to rebuild them (rm *.o).
  4. Replace the C compiler invocation command in the Makefile with an appropriate series of shell commands.
    .include "$S/conf/kern.pre.mk"
    The code below was added after the line above
    NORMAL_C= echo '\#pragma echo "Processing file ${.IMPSRC}\n"' >>kernel.cs ;\
          echo '\#pragma block_enter' >>kernel.cs ;\
          echo '\#pragma clear_defines' >>kernel.cs ;\
          echo '\#pragma clear_include' >>kernel.cs ;\
          echo '\#include "cscout_defs.h"' >>kernel.cs ;\
          for i in $(INCLUDES) ; \
          do \
                  case $$i in \
                  -nostdinc) continue ;; \
                  -I-) continue ;; \
                  esac ; \
                  i=`echo $$i | sed 's/-I//'` ; \
                  echo '\#pragma includepath "'$$i'"' >>kernel.cs ; \
          done ; \
          echo '\#define _KERNEL 1' >>kernel.cs ;\
          echo '\#pragma process "opt_global.h"' >>kernel.cs ;\
          echo '\#pragma process "${.IMPSRC}"' >>kernel.cs ;\
          echo '\#pragma block_exit' >>kernel.cs ;\
          echo '\#pragma echo "Done processing file ${.IMPSRC}\n"' >>kernel.cs
    
  5. Create a cscout_incs.h file for each different architecture.
  6. Remove kernel.cs The existing file documents the way to do it.
  7. Run make on the custom Makefile
  8. Repeat for each different architecture
  9. Create a top-level CScout file to process all architectures:
    #pragma echo "Processing workspace FreeBSD kernel\n"
    
    #pragma echo "Entering directory sys/i386/compile/LINT\n"
    #pragma pushd "sys/i386/compile/LINT"
    #pragma echo "Processing project i386\n"
    #pragma project "i386"
    #pragma block_enter
    #include "kernel.cs"
    #pragma echo "Exiting directory sys/i386/compile/LINT\n"
    #pragma popd
    #pragma echo "Done processing project i386\n"
    #pragma block_exit
    
    #pragma echo "Entering directory sys/amd64/compile/GENERIC\n"
    // [...]
    // and so on for all architectures
    // [...]
    #pragma echo "Exiting directory sys/sparc64/compile/LINT\n"
    #pragma popd
    #pragma echo "Done processing project sparc64\n"
    #pragma block_exit
    
    Note that the block_enter and block_exit pragmas are furnished by this top-level file.
The run of the above specification (2 million unique lines) took 330 CPU minutes on a Rioworks HDAMA (AMD64) machine (2x1.8GHz Opteron 244 (in UP mode) - AMD 8111/8131 chipset, 8192MB mem) and required 1474MB of RAM. These are the complete metrics:

CScout Home

File Metrics

Writable Files

Number of files: 4310

File metricTotalMinMaxAvg
Number of characters625057700100834514502
Comment characters159217520850593694
Space characters79364010739681841
Number of line comments19040
Number of block comments1762530433740
Number of lines2063096027336478
Length of longest line3370490186778
Number of C strings13251901929630
Number of defined functions2958403336
Number of preprocessor directives26754202733662
Number of directly included files35408016088
Number of C statements67982504465157