Source Code Views

You can view a file's source code in five different forms:

1. The plain source code, will only provide you the file's code text
2. The source code with unprocessed regions marked, will enable you to see which parts of the file was not processed due to conditional compilation instructions. You may want to use the marked parts as a guide to construct a more inclusive workspace definition (perhaps by processing the project multiple times, with different preprocessor options).

   360 #if defined(__GNUC__) && defined(__STDC__)   361 static __inline int __sputc(int _c, FILE *_p) {   362         if (--_p->_w >= 0 || (_p->_w >= _p->_lbfsize && (char)_c != '\n'))   363                 return (*_p->_p++ = _c);   364         else   365                 return (__swbuf(_c, _p));   366 }   367 #else   368 /*   369  * This has been tuned to generate reasonable code on the vax using pcc.   370  */   371 #define __sputc(c, p) \   372         (--(p)->_w < 0 ? \   373                 (p)->_w >= (p)->_lbfsize ? \   374                         (*(p)->_p = (c)), *(p)->_p != '\n' ? \   375                                 (int)*(p)->_p++ : \   376                                 __swbuf('\n', p) : \   377                         __swbuf((int)(c), p) : \   378                 (*(p)->_p = (c), (int)*(p)->_p++))   379 #endif   380

3. Source code with identifier hyperlinks, will provide you with a page of the file's code text where each identifier is represented as a hyperlink leading to the identifier's page. The following is a representative example.

   int copy_fifo(from_stat, exists)         struct stat *from_stat;         int exists; {         if (exists && unlink(to.p_path)) {                 warn("unlink: %s", to.p_path);                 return (1);         }         if (mkfifo(to.p_path, from_stat->st_mode)) {                 warn("mkfifo: %s", to.p_path);                 return (1);         }         return (pflag ? setfile(from_stat, 0) : 0); }

4. As the above display can be overwhelming, you may prefer to browse the source code with hyperlinks only to project-global writable identifiers, which are typically the most important identifiers. Consider again how the above example would be displayed:

   int copy_fifo(from_stat, exists)         struct stat *from_stat;         int exists; {         if (exists && unlink(to.p_path)) {                 warn("unlink: %s", to.p_path);                 return (1);         }         if (mkfifo(to.p_path, from_stat->st_mode)) {                 warn("mkfifo: %s", to.p_path);                 return (1);         }         return (pflag ? setfile(from_stat, 0) : 0); }

5. Source code with hyperlinks to function and macro declarations provides you hyperlinks to the function pages for each function declaration (implicit or explict) and macro definition. Again, here is an example:

   #if !defined(_ANSI_SOURCE) && !defined(_POSIX_SOURCE) int     digittoint __P((int)); int     isascii __P((int)); int     isblank __P((int)); int     ishexnumber __P((int)); int     isideogram __P((int)); int     isnumber __P((int)); int     isphonogram __P((int)); int     isrune __P((int)); int     isspecial __P((int)); int     toascii __P((int)); #endif __END_DECLS #define __istype(c,f)    (!!__maskrune((c),(f))) #define isalnum(c)       __istype((c), _CTYPE_A|_CTYPE_D) #define isalpha(c)       __istype((c), _CTYPE_A) #define iscntrl(c)       __istype((c), _CTYPE_C) #define isdigit(c)       __isctype((c), _CTYPE_D) /* ANSI -- locale independent */ #define isgraph(c)       __istype((c), _CTYPE_G) #define islower(c)       __istype((c), _CTYPE_L) #define isprint(c)       __istype((c), _CTYPE_R) #define ispunct(c)       __istype((c), _CTYPE_P) #define isspace(c)       __istype((c), _CTYPE_S) #define isupper(c)       __istype((c), _CTYPE_U) #define isxdigit(c)      __isctype((c), _CTYPE_X) /* ANSI -- locale independent */ #define tolower(c)       __tolower(c) #define toupper(c)       __toupper(c)

File Metrics

All files

Read-only files

Writable files

Files containing unused project-scoped writable identifiers

In our example, the following list is generated:

Files containing unused file-scoped writable identifiers

In our example, the following list is generated:

Writable .c files without any statements

In our example, the result set only contains the processing script (the compiled workspace definition file).

Writable files containing unprocessed lines

In our case the results are:

Writable files containing strings

In our case the results are:

Writable .h files with #include directives

In our example, the result is:

Generic File Queries

You specify the query through the following form:

You start by specifying whether the file should be writable (i.e. typically part of your application) and/or readable (i.e. typically part of the compiler or system). Next come a series of metrics CScout collects for each file. For each metric (e.g. the number of comments) you can specify an operator ==, !=, < or > and a number to match that metric against. Thus to locate files without any comments you would specify
Number of block comments == 0.

On the left of each metric you can specify whether that metric will be used to sort the resulting file list. In that case, the corresponding number will appear together with each file listed. A separate option allows you to specify that files should be sorted in the reverse order.

You can request to see files matching any of your specifications (Match any of the above) or to see files matching all your specifications (Match all of the above).

Sometimes you may only want to search in a subset of files; you can then specify a regular expression that filenames should match (or not match) against: "File names should (not) match RE".

Finally, you can also specify a title for your query. The title will then appear on the result document annotating the results, and will also provide you with a sensible name when creating a bookmark to it.

Include Graphs

Two global options specify the format of the include graph and the content on each graph's node. Through these options you can obtain graphs in

• plain text form: suitable for processing with other tools,
• HTML: suitable for browsing via CScout,
• dot: suitable for generating high-quality graphics files,
• SVG: suitable for graphical browsing, if your browser supports this format, and
• GIF: suitable for viewing on SVG-challenged browsers.

including file -> included file

Two links on the main page (file include graph - writable files and file include graph - all files) can give you the include graphs of the complete program. For programs larger than a hundred thousand lines, these graphs are only useful in their textual form. In their graphical form, even with node information disabled, they can only serve to give you a rough idea of how the program is structured. The following image depicts how writable (non-system) files are included in the awk source code.

and the following is a part of the include file structure of the Windows Research Kernel

More useful are typically the include graphs that can be generated for individual files. These can allow you to see what paths can possibly lead to the inclusion of a given file (include graph of all including files) or what files a given file includes (include graph of all included files). (call graph of all callers), which functions can be reached starting from a given function, and how functions in a given file relate to each other.

As an example, the following diagram depicts all files that main.c includes

while the following diagrams shows all the files including (directly or indirectly) proto.h.

C Namespaces

int i = 3;

foo()
{
	int i = 7;
}

main()
{
	foo();
	printf("%d\n", i);
}

In addition, C also partitions a program's identifiers into four namespaces. Identifiers in one namespace, are also considered different from identifiers in another. The four namespaces are:

1. Tags for a struct/union/enum
2. Members of struct/union (actually a separate namespace is assigned to each struct/union)
3. Labels
4. Ordinary identifiers (termed objects in the C standard)

/* structure tag */
struct id {
	int id;		/* structure member */
};

/* Different structure */
struct id2 {
	char id;	/* structure member */
};

/* ordinary identifier */
id()
{
id:	/* label */
}

Normally when you want to locate or change an identifier name, you only consider identifiers in the same scope and namespace. Sometimes however, a C preprocessor macro can semantically unite identifiers living in different namespaces, so that changes in one of them should be propagated to the others. The most common case involves macros that access structure members.

struct s1 {
	int id;
} a;

struct s2 {
	char id;
} b;

#define getid(x) ((x)->id)

main()
{
	printf("%d %c", getid(a), getid(b));
}

Finally, the C preprocessor's token concatenation feature can result in identifiers that should be treated for substitution purposes in separate parts. Consider the following example:

int xleft, xright;
int ytop, ybottom;

#define coord(a, b) (a ## b)

main()
{
	printf("%d %d %d %d\n",
		coord(x, left),
		coord(x, right),
		coord(y, top),
		coord(y, bottom));
}

Identifier Elements

• If the identifier is read-only (i.e. it appears in at least one read-only file)
• The C namespace(s) it appears in (the same identifier can be a member of multiple namespaces)
• Whether the identifier is visible at file or project scope
• Whether the identifier is a typedef (typedef's belong to the ``ordinary identifier'' namespace, but are obviously important, so CScout will tag them as such).
• Whether the identifier crosses a file boundary, i.e. it appears in more than one file
• Whether the identifier is unused i.e. it appears in exactly one location
• The identifier's number of occurences in all the workspace's files
• The projects the identifier appears in
• The function names the identifier forms; the link "function page" will provide you more details regarding the function
• An option to substitute the identifier's name with a different name

Finally, the identifier's page will list the writable and all files the specific identifier appears in. Clicking on the ``marked source'' hyperlink will display the respective file's source code with only the given identifier marked as a hyperlink. By pressing your browser's tab key you can then see where the given identifier is used. In our example the cp.c source code with the copy_file identifier marked would appear as follows:

Identifier Metrics

You can use these metrics to compare characteristics of different projects, adherance to coding standards, or to identify identifier classes with abnormally short or long names. The ratio between the distinct number of identifiers and the total number of identifiers is the number of times each identifier is used. Notice the difference in our case between the read-only identifiers (which are mostly declarations) and the writable identifiers (which are actually used).

All identifiers

Read-only identifiers

Writable identifiers

File-spanning writable identifiers

Unused project-scoped writable identifiers

Unused file-scoped writable identifiers

Unused writable macros

Generic Identifier Queries

• The properties (namespace, scope, instances) of the identifier
• Whether the specified properties should be treated
  • as a disjunction (match any marked),
  • as a conjunction (match all marked),
  • as a negation excluding all identifiers matching any property (exclude marked), or
  • as an exact match specification matching only identifiers that match exactly the properties specified (exact match)
• A regular expression against which identifier names should match (or not match)
• A regular expression that filenames in which identifiers occur should match (or not match)
• A query title to be used for naming the result page. The title will appear on the result document annotating the results, and will also provide you with a sensible name when creating a bookmark to it.

• the identifiers that match the specific query,
• the files containing identifiers that match the query, or
• the functions containing identifiers that match the query.

As an example, the following query could be used to identify unused file-scoped writable identifiers, but excluding the copyright and rcsid identifiers:

Function Elements

From this page you can refactor the function's arguments (more on this in the next section) and obtain the following data.

• The identifier or identifiers composing the function name. These can be modified (from the corresponding identifier page) to change the function's name.
• The function's declaration. This may be an implicit declaration (the location of its first use). CScout only maintains the location of one declaration for each function. You can locate additional points of declaration by looking at the places where the corresponding identifier is used. The "marked source" link allows you to see the declaration as a hyperlink in the file where it occurs. In many browsers pressing the tab key on that page will lead you directly to the function's declaration.
• The function's definition (if a definition was found). Library functions obviously will not have a definition associated with them.
• The number of functions this function directly calls. These are the functions (C functions and function-like macros) that appear inside the function's body.
• A list allowing you to explore interactively the tree of called functions. The tree will appear in the following form:
  
  Each plus or minus box will open or close the list of called functions. Each function name is a hyperlink to the corresponding function page.
• A list of all called functions. This list includes all functions that can be called, starting from the function we are examining. On the right of each function is a hyperlink to a call graph of the path(s) leading from the function being examined to the function listed.
• A call graph of all called functions (explained in a following section).
• A page allowing you to explore interactively all callers. These are the functions that directly call the function we are examining. The functionality of this page is the same as that of the one for exploring the called functions.
• A list of all callers. These are all functions that can directly or indirectly call the function we are examining.
• A call graph of all callers (explained in a following section).
• A call graph of all the function's callers and called functions showing the function in context (explained in a following section).
• A comprehensive set of metrics regarding the function (only for defined functions and macros).

All Functions

Project-scoped writable functions

File-scoped writable functions

Writable functions that are not directly called

Writable functions that are called exactly once

Refactoring Function Arguments

To refactor the function's arguments, one simply enters in the text box a template describing the argument replacement pattern. The template consists of text, which is copied verbatim as a function's argument, and elements starting with the operator @, which have a special meaning. The combined effect of this template mechanism allows you to

• Introduce new arguments
• Remove existing arguments
• Change the arguments' order

@N

pastes the original Nth argument passed to the function. Thus, @1 will get replaced with the function's first argument. Specifying in a template for a function taking two arguments "@2, @1" will swap their order, while specifying "@1, sizeof(@1), stdin" as the arguments for gets will refactor them in a form suitable for calling fgets (if the original argument refers to a fixed-size character array).

@.N

pastes the Nth argument and all subsequent ones, separated by commas. This is useful for handling functions with a variable number of arguments, like printf. Specifying in a template for the printf function "stdout, @1, @.2" will introduce an extra first parameter, named stdout. (Presumably the function will also be renamed to fprintf.)

@+N{...}

pastes the text in the braces, if the specific function being replaced has an Nth argument. The text in the braces can include arbitrary text, including nested @ operators.

@-N{...}

pastes the text in the braces, if the specific function being replaced does not have an Nth argument.

Note that the refactorings will take place on all instances where the identifier is found to match the function or macro. This includes declarations and definitions (which might require some hand-editing if arguments are introduced), and the appearance of the name in the replacement text of a macro, when that macro is used in a way that makes the function match the one being refactored. The replacements will not be performed to function calls that are executed through a function pointer.

Call Graphs

Two global options specify the format of the call graph and the content on each graph's node. Through these options you can obtain graphs in

• plain text form: suitable for processing with other tools,
• HTML: suitable for browsing via CScout,
• dot: suitable for generating high-quality graphics files,
• SVG: suitable for graphical browsing, if your browser supports this format, and
• GIF: suitable for viewing on SVG-challenged browsers.

calling function -> called function

Two links on the main page (function and macro call graph, and non-static function call graph) can give you the call graphs of the complete program. For any program larger than a few thousand lines, these graphs are only useful in their textual form. In their graphical form, even with node information disabled, they can only serve to give you a rough idea of how the program is structured. The following image depicts how the three different programs we analyzed in the bin example relate to each other.

More useful are the call graphs that can be generated for individual functions or files. These can allow you to see what paths can possibly lead to a given function (call graph of all callers), which functions can be reached starting from a given function, the function in context, and how functions in a given file relate to each other.

As an example, the following diagram depicts all paths leading to the setfile function.

Correspondingly, the functions that can be reached starting from the copy_file function appears in the following diagram.

while the following shows the function setsymtab in context, depicting all the paths leading to it (callers) and leaving from it (called functions).

Finally, the following is an example of how the functions in a single file (parse.c) relate to each other.

Generic Function Queries

On the top you can specify whether each function you want listed:

• is a C function
• is a function-like macro
• has a writable declaration
• has a read-only declaration
• is visible in the whole project scope
• is visible only in a file scope
• has a definition body.

As is the case in file queries, next comes a series of metrics CScout collects for each defined function. For each metric (e.g. the number of comments) you can specify an operator ==, !=, < or > and a number to match that metric against. Thus to locate functions containing goto statement you would specify
Number of goto statements != 0.

On the left of each metric you can specify whether that metric will be used to sort the resulting file list. In that case, the corresponding number will appear together with each file listed. A separate option allows you to specify that files should be sorted in the reverse order.

Similarly to the identifier query, you can also specify whether the specified properties should be treated

• as a disjunction (match any marked),
• as a conjunction (match all marked),
• as a negation excluding all identifiers matching any property (exclude marked), or
• as an exact match specification matching only identifiers that match exactly the properties specified (exact match)

In addition you can specify:

• That the function should have a specified number of direct callers.
• A regular expression against which function names should match (or not match)
• A regular expression against which the names of calling functions should match (or not match)
• A regular expression against which the names of any called functions should match (or not match)
• A regular expression that filenames in which functions are declared occur should match (or not match)
• A query title to be used for naming the result page. The title will appear on the result document annotating the results, and will also provide you with a sensible name when creating a bookmark to it.

Global Options

File and Identifier Pages

Show Only True Identifier Classes

Show Associated Projects

Show Lists of Identical Files

Source Listings

Show Line Numbers

Tab Width

Refactoring

Allow the renaming of read-only identifiers

Allow the refactoring of function arguments of read-only functions

Check for renamed identifier clashes when saving refactored code

Queries

Case-insensitive File Name Regular Expression Match

Query Result Lists

Number of Entries on a Page

Show File Lists With File Name in Context

Sort Identifiers Starting from their Last character

Call and File Dependency Graphs

Call Graph Links Should Lead to Pages of

• Plain text: suitable for processing with other text tools.
• HTML: suitable for interactive browsing
• dot: suitable for processing with GraphViz dot into different graphics formats, like PNG, MIF, VRML, and EPS. Dot files can also be processed as graphs using the AT&T gpr program
• SVG: suitable for interactively browsing the graphical representation of the call graph. This option requires your browser to support the rendering of SVG (directly or via a plugin, such as Adobe's (http://www.adobe.com/svg/)), and the existence of the AT&T GraphViz (http://www.graphviz.org) dot program in your executable file path.
• GIF: suitable for directly viewing relatively small images.

Call Graphs Should Contain

• Only edges, will not display anything on the node. This option can be used in the graphics formats (dot, SVG, GIF) to provide an overall picture of the program's call structure.
• Function names: only include the function names. Functions with the same name will still be separately listed, but you will have to follow their hyperlinks to see where they are defined.
• File and function names: the name of the file where a function is declared will precede the name of the function.
• Path and function names: the complete file path of the file where a function is declared will precede the name of the function.

File Graphs Should Contain

• Only edges, will not display anything on the node. This option can be used in the graphics formats (dot, SVG, GIF) to provide an overall picture of the program's file dependency structure.
• File names: only include the file names. Files with the same name will still be separately listed, but you will have to follow their hyperlinks to see where they are defined.
• Path and file names: the complete path of each path will be show.

Maximum number of call levels in a graph

Maximum dependency depth in a file graph

Include URLs in dot output

Graph options

Node options

Edge options

The graph, node, and edge options can be used to fine tune the graph's look. See the GraphViz documentation (http://www.graphviz.org/doc/info/attrs.html) for more details. For instance, the following diagram

was created using

Saved Files

When Saving Modified Files Replace

• continue operating CScout, even after the changes have been saved, and
• easilly back out changes your are not satisfied with.

Note than when this option is specified the existing and new locations of the file must reside on the same drive and partition (under Windows) or file system (under Unix).

Editing

xterm -c "$VISUAL +/'%s' '%s'"

echo Ignoring search for "%s" & start notepad "%s"

start  C:\Progra~1\Vim\vim70\gvim.exe +/"%s" "%s"

Option Files

When CScout starts-up it will attempt to load the options file by searching in $CSCOUT_HOME, $HOME/.cscout, or .cscout in the current directory.

The options file is text based and contains key-value pairs. The order of the entries is not significant. This is an example of an options file.

show_true: 1
show_projects: 1
show_identical_files: 1
show_line_number: 0
tab_width: 8
rename_override_ro: 1
refactor_fun_arg_override_ro: 1
file_icase: 0
entries_per_page: 20
fname_in_context: 1
sort_rev: 0
cgraph_type: s
cgraph_show: n
fgraph_show: n
cgraph_depth: 5
fgraph_depth: 5
cgraph_dot_url: 0
sfile_re_string: sfile_repl_string:
sfile_repl_string: entries_per_page:
start_editor_cmd: start  C:\Progra~1\Vim\vim71\gvim.exe +/"%s" "%s"

Operations

Identifier Replacements

The following is an example of the identifier replacements page. You see all identifiers for which replacements have been specified. All specified replacements are originally active. If a particular replacement appears to be causing problems you can deactivate it from this page. In addition, you can change the replaced name of any of the replaced identifiers. Finally, clicking on an identifier name will lead you to the corresponding identifier page.

Select Active Project

Save Changes and Continue

Exit - Saving Changes

Exit - Ignore Changes

Hand-Editing

The automatic global identifier replacement and the hand-editing of files are mutualy exclusive operations. Once either of the two is performed the other ceases to be available. This is done to protect the integrity of the underlying source code. Furthermore, all CScout's operations, such as queries and source code listings, are always performed on a snapshot of the source code taken just before a file is edited by hand.

Interfacing with Version Management Systems

As an example, for a system managed with Perforce (https://www.perforce.com/) the following commands could be used:

cscout_checkout

#!/bin/sh
p4 edit $1

cscout_checkin

#!/bin/sh
p4 submit -d 'CScout identifier name refactoring' $1

Language Extensions

1. Preprocessor directives can appear within a call to a function-like macro (gcc)
2. Initializers and compound literals can be empty (gcc)
3. The alignof operator can be used on types (gcc)
4. A declaration expression as the first element of a for statement (C99)
5. The restrict qualifier and the inline specifier (C99)
6. The _Bool type (C99)
7. The _Thread_local storage class specifier (C11)
8. Initialization designators (C99)
9. Array initialization designators can include ranges (gcc)
10. ANSI-style function definitions can be nested (gcc) (gcc also allows nested K&R-style function definitions)
11. The equals sign following an initializer designator is optional (gcc)
12. Array and structure initialization (gcc)
13. Compound literals (C99)
14. Declarations can be intermixed with statements (C99).
15. Recognise __atribute__(__unused__) for determining which identifiers should not be reported as unused (gcc).
16. // line comments (common extension)
17. __asm__ blocks (gcc)
18. enum lists ending with a comma (common extension)
19. Anonymous struct/union members (gcc, Microsoft C)
20. Allow case expression ranges (gcc).
21. An enumeration list can be empty (Microsoft C)
22. Allow braces around scalar initializers (common extension).
23. Indirect goto targets and the label address-of operator (gcc).
24. __typeof keyword (gcc)
25. A compound statement in brackets can be an expression (gcc)
26. Macros expanding from /##/ into // are then treated as a line comment (Microsoft C)
27. Exception handling using the __try __except __finally __leave keywords (Microsoft C)
28. #include_next preprocessor directive (gcc)
29. #warning preprocessor directive (gcc)
30. Variable number of arguments preprocessor macros (support for both the gcc and the C99 syntax)
31. Allow empty member declarations in aggregates (gcc)
32. long long type (common extension)
33. A semicolon can appear as a declatation (common extension)
34. An aggregate declaration body can be empty (gcc)
35. Aggregate member initialization using the member: value syntax (gcc)
36. Statement labels do not require a statement following them (gcc)
37. #ident preprocessor directive (gcc)
38. Allow assignment to case expressions (common extension)
39. Accept an empty translation unit (common extension).
40. Support locally declared labels (__label__) (gcc).
41. The second argument of a conditional expression can be omitted (gcc).
42. Dereferencing a function yields a function (common extension).

Processing Yacc Files

CScout processes yacc files as follows:

• All terminal and non-terminal names are considered to be defined as file-scoped identifiers in a special yacc-only namespace.
• All terminal symbols are also marked as an undefined macro. Thus, processing the yacc-generated file y.tab.h immediately after the grammar (i.e. in the same scope) will unify the terminal symbols with the corresponding macro definitions throughout the program.
• Members of the %union construct are defined as members of the YYSTYPE union typedef. These are then considered to be accessed in all %type and the precedence specification constructs, as well as the explicit type specification through the $<tag># construct.
• No yacc-specific macros (yyerrok, YYABORT, YYACCEPT, etc) are defined. Feel free to define anything required to silence CScout as a macro in the workspace definition file.

CScout is designed to process well-formed modern-style yacc files. All rules must be terminated with a semicolon (apparently this is optional in the original yacc version). The accepted grammar appears below.

body:
	defs '%%' rules tail
	;

tail:
	/* Empty */
	| '%%' c_code
	;

defs:
	/* Empty */
	| defs def
	;

def:
	'%start' IDENTIFIER
	| '%union' '{' member_declaration_list  '}' 
	| '%{' c_code '%}'
	| rword name_list_declaration
	;

rword:
	'%token'
	| '%left'
	| '%right'
	| '%nonassoc'
	| '%type'
	;

tag:
	/* Empty: union tag is optional */
	| '<' IDENTIFIER '>'
	;

name_list_declaration:
	tag name_number
	| name_list_declaration opt_comma name_number
	;

opt_comma:
	/* Empty */
	| ','
	;

name_number:
	name
	| name INT_CONST
	;

name:
	IDENTIFIER
	| CHAR_LITERAL
	;

/* rules section */

rules:
	rule
	| rules rule
	;

rule:
	IDENTIFIER ':'  rule_body_list ';'
	;

rule_body_list:
	rule_body
	| rule_body_list '|' rule_body
	;

rule_body:
	id_action_list prec
	;

id_action_list:
	/* Empty */
	| id_action_list name
        | id_action_list '{' c_code '}' 
	;

prec:
	/* Empty */
	| '%prec' name
	| '%prec' name  '{' c_code '}'
	;

variable:
	'$$'
	| '$' INT_CONST
	| '$-' INT_CONST
		{ $$ = basic(b_int); }
	| '$<' IDENTIFIER '>' variable_suffix
		{ $$ = $3; }
	;

variable_suffix:
	'$'
	| INT_CONST
	| '-' INT_CONST
	;

Regular Expression Syntax

Regular expressions (``REs''), as defined in IEEE Std 1003.2 (``POSIX.2''), come in two forms: modern REs (roughly those of egrep(1); 1003.2 calls these ``extended'' REs) and obsolete REs (roughly those of ed(1); 1003.2 ``basic'' REs). CScout has adopted the use of modern (extended) REs.

A (modern) RE is one= or more non-empty= branches, separated by `|'. It matches anything that matches one of the branches.

A branch is one= or more pieces, concatenated. It matches a match for the first, followed by a match for the second, etc.

A piece is an atom possibly followed by a single= `*', `+', `?', or bound. An atom followed by `*' matches a sequence of 0 or more matches of the atom. An atom followed by `+' matches a sequence of 1 or more matches of the atom. An atom followed by `?' matches a sequence of 0 or 1 matches of the atom.

A bound is `{' followed by an unsigned decimal integer, possibly followed by `,' possibly followed by another unsigned decimal integer, always fol- lowed by `}'. The integers must lie between 0 and RE_DUP_MAX (255=) inclusive, and if there are two of them, the first may not exceed the second. An atom followed by a bound containing one integer i and no comma matches a sequence of exactly i matches of the atom. An atom fol- lowed by a bound containing one integer i and a comma matches a sequence of i or more matches of the atom. An atom followed by a bound containing two integers i and j matches a sequence of i through j (inclusive) matches of the atom.

An atom is a regular expression enclosed in `()' (matching a match for the regular expression), an empty set of `()' (matching the null string)=, a bracket expression (see below), `.' (matching any single character), `^' (matching the null string at the beginning of a line), `$' (matching the null string at the end of a line), a `\' followed by one of the characters `^.[$()|*+?{\' (matching that character taken as an ordinary character), a `\' followed by any other character= (matching that character taken as an ordinary character, as if the `\' had not been present=), or a single character with no other significance (matching that character). A `{' followed by a character other than a digit is an ordinary character, not the beginning of a bound=. It is illegal to end an RE with `\'.

A bracket expression is a list of characters enclosed in `[]'. It nor- mally matches any single character from the list (but see below). If the list begins with `^', it matches any single character (but see below) not from the rest of the list. If two characters in the list are separated by `-', this is shorthand for the full range of characters between those two (inclusive) in the collating sequence, e.g. `[0-9]' in ASCII matches any decimal digit. It is illegal= for two ranges to share an endpoint, e.g. `a-c-e'. Ranges are very collating-sequence-dependent, and portable programs should avoid relying on them.

To include a literal `]' in the list, make it the first character (fol- lowing a possible `^'). To include a literal `-', make it the first or last character, or the second endpoint of a range. To use a literal `-' as the first endpoint of a range, enclose it in `[.' and `.]' to make it a collating element (see below). With the exception of these and some combinations using `[' (see next paragraphs), all other special charac- ters, including `\', lose their special significance within a bracket expression.

Within a bracket expression, a collating element (a character, a multi- character sequence that collates as if it were a single character, or a collating-sequence name for either) enclosed in `[.' and `.]' stands for the sequence of characters of that collating element. The sequence is a single element of the bracket expression's list. A bracket expression containing a multi-character collating element can thus match more than one character, e.g. if the collating sequence includes a `ch' collating element, then the RE `[[.ch.]]*c' matches the first five characters of `chchcc'.

Within a bracket expression, a collating element enclosed in `[=' and `=]' is an equivalence class, standing for the sequences of characters of all collating elements equivalent to that one, including itself. (If there are no other equivalent collating elements, the treatment is as if the enclosing delimiters were `[.' and `.]'.) For example, if `x' and `y' are the members of an equivalence class, then `[[=x=]]', `[[=y=]]', and `[xy]' are all synonymous. An equivalence class may not= be an end- point of a range.

Within a bracket expression, the name of a character class enclosed in `[:' and `:]' stands for the list of all characters belonging to that class. Standard character class names are:

   alnum    digit    punct
   alpha    graph    space
   blank    lower    upper
   cntrl    print    xdigit

There are two special cases= of bracket expressions: the bracket expres- sions `[[:<:]]' and `[[:>:]]' match the null string at the beginning and end of a word respectively. A word is defined as a sequence of word characters which is neither preceded nor followed by word characters. A word character is an alnum character (as defined by ctype(3)) or an underscore. This is an extension, compatible with but not specified by IEEE Std 1003.2 (``POSIX.2''), and should be used with caution in soft- ware intended to be portable to other systems.

In the event that an RE could match more than one substring of a given string, the RE matches the one starting earliest in the string. If the RE could match more than one substring starting at that point, it matches the longest. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the RE taking priority over ones starting later. Note that higher-level subexpressions thus take priority over their lower-level component subexpressions.

Match lengths are measured in characters, not collating elements. A null string is considered longer than no match at all. For example, `bb*' matches the three middle characters of `abbbc', `(wee|week)(knights|nights)' matches all ten characters of `weeknights', when `(.*).*' is matched against `abc' the parenthesized subexpression matches all three characters, and when `(a*)*' is matched against `bc' both the whole RE and the parenthesized subexpression match the null string.

If case-independent matching is specified, the effect is much as if all case distinctions had vanished from the alphabet. When an alphabetic that exists in multiple cases appears as an ordinary character outside a bracket expression, it is effectively transformed into a bracket expres- sion containing both cases, e.g. `x' becomes `[xX]'. When it appears inside a bracket expression, all case counterparts of it are added to the bracket expression, so that (e.g.) `[x]' becomes `[xX]' and `[^x]' becomes `[^xX]'.

Access Control

To allow other hosts to connect CScout features an access control list. The list is specified in a file called acl which should be located in $CSCOUT_HOME, $HOME/.cscout, or .cscout in the current directory. The list contains lines with IP numeric addresses prefixed by an A (allow) or D (deny) prefix and a space. Matching is performed by comparing a substring of a machine's IP address against the specified access rule. Thus an entry such as

A 128.135.11.

• If the address matches an allowed entry the machine will be allowed access.
• If no allowed entries have been specified, the machine will be allowed access unless it has been denied access. (i.e. you can not use a deny entry to exclude a machine from an allowed group)

Obfuscation Back-end

Each source code file is obfuscated by

• Giving meaningless names to all identifiers: macros, arguments, ordinary identifiers, structure tags, structure members, and labels.
• Removing comments.
• Removing extraneous whitespace.

Before running CScout to obfuscate, make a complete backup copy of your source code files, and store them in a secure place; preferably off-line. Once the source code files are obfuscated and overwritten, there is no way to get back their original contents.

To obfuscate the workspace, first ensure that CScout can correctly process the complete set of its source code files. Use the "unprocessed lines" metric of each file to verify that no parts of a file are left unprocessed; unprocessed regions will not be obfuscated. You can easily increase the coverage of CScout's processing by including in the workspace multiple projects with different defined directives.

Also ensure that all your project's files are considered writable, and no files outside your project (for example system headers) are considered writable. This will allow CScout to rename your identifier names, but keep the names of library-defined identifiers (for example printf) unchanged.

Finally, run CScout with the switch -o. For each writable workspace file CScout will create a file ending in .obf that will contain the obfuscated version of its contents. The files are not overwritten, providing you with another countermeasure against accidentally destroying them. To overwrite the original source with the obfuscated one, use the following Unix command:

find . -name '*.obf' |
sed 's/\\/\//g;s/\(.*\)\.obf$/mv "\1.obf" "\1"/' |
sh

SQL Back-end

To generate the SQL script simply run CScout with the switch -s dialect, where the argument specifies the SQL dialect (for example, mysql, or postgresql). The SQL script will appear in CScout's standard output, allowing you to directly pipe the results into the database's client. For example, say the database you would want to create for your project was called myproj.
For MySQL you would write:

(
	echo "create database myproj; use myproj ;"
	cscout -s mysql myproj.cs
) | mysql

createdb -U username myproj
cscout -s postgres myproj.cs | psql -U username myproj

cscout -s hsqldb myproj.cs |
java -classpath $HSQLDBHOME/lib/hsqldb/hsqldb.jar org.hsqldb.util.SqlTool --rcfile $HSQLDBHOME/lib/hsqldb/sqltool.rc mem -

Schema of the Generated Database

Table IDS

Details of interdependant identifiers appearing in the workspace.

Table FILES

File details.

Table TOKENS

Instances of identifier tokens within the source code.

Table COMMENTS

Comments in the code.

Table STRINGS

Strings in the code.

Table REST

Remaining, non-identifier source code.

Table LINEPOS

Line number offsets within each file.

Table PROJECTS

Project details.

Table IDPROJ

Identifiers appearing in projects.

Table FILEPROJ

Files used in projects.

Table DEFINERS

Included files defining required elements for a given compilation unit and project.

Table INCLUDERS

Included files including files for a given compilation unit and project.

Table PROVIDERS

Included files providing code or data for a given compilation unit and project.

Table INCTRIGGERS

Tokens requiring file inclusion for a given compilation unit and project.

Table FUNCTIONS

C functions and function-like macros.

Table FUNCTIONMETRICS

Metrics of defined functions and macros.

Table FUNCTIONID

Identifiers comprising a function's name.

Table FCALLS

Function calls.

Table FILECOPIES

Files occuring in more than one copy.

Note 1: INTEGER on 32-bit architectures, BIGINT on 64-bit archiectures.

Examples of SQL Queries

Find identifiers of a given type (typedefs, in this case):

select name from
ids left join tokens on ids.eid = tokens.eid
where ids.typedef = true

Number of different files that use a given identifier:

select name, count(*) as cf from (
 select fid, tokens.eid, count(*) as c from
 tokens
 group by
 eid, fid) as cl inner join ids on
cl.eid = ids.eid
group by ids.eid, ids.name
order by cf desc;

Number of times an identifier occurs in a single file:

SELECT IDS.NAME AS INAME, FILES.NAME AS FNAME, COUNT(*) AS C FROM TOKENS
INNER JOIN IDS ON
IDS.EID = TOKENS.EID
INNER JOIN FILES ON
TOKENS.FID = FILES.FID
GROUP BY IDS.EID, TOKENS.FID
ORDER BY C DESC;

Number of times an identifier occurs in the workspace:

select name, count(*) as c from tokens
inner join ids on
ids.eid = tokens.eid
group by eid
order by c desc

Reconstitute the file with fid = 4:

select s from
(select name as s, foffset  from ids inner join tokens on
ids.eid = tokens.eid where fid = 4
union select code as s, foffset from rest where fid = 4
union select comment as s, foffset from comments where fid = 4
union select string as s, foffset from strings where fid = 4
)
order by foffset

sed -e '/^[0-9][0-9]* rows/d' |
tr -d '\n' |
sed 's/\\u0000d/\
/g'

sed -e '/^[0-9][0-9]* rows/d' |
tr -d '\n\r' |
sed 's/\\u0000d\\u0000a/\
/g'

Show the projects each identifier belongs to:

select IDS.NAME, PROJECTS.NAME from IDS
INNER JOIN IDPROJ ON IDS.EID = IDPROJ.EID
INNER JOIN PROJECTS ON IDPROJ.PID = PROJECTS.PID
ORDER BY IDS.NAME;

Show the included files required by other files for each compilation unit and project.

select
	projects.name as projname,
	cufiles.name as cuname,
	basefiles.name as basename,
	definefiles.name as defname
from
	definers inner join projects on definers.pid = projects.pid
	inner join files as cufiles on definers.cuid=cufiles.fid
	inner join  files as basefiles on definers.basefileid=basefiles.fid
	inner join files as definefiles on definers.definerid = definefiles.fid;

Speed-up processing:

create index teid on tokens(eid)
create index tfid on tokens(fid)

Obtain identifiers common between files participating in a define/use relationship:

SELECT
    tokensa.eid,
    min(ids.name) as identifier,
    min(filesb.name) as defined,
    min(filesa.name) as used
FROM definers
INNER JOIN tokens AS tokensa ON definers.basefileid = tokensa.fid
INNER JOIN tokens AS tokensb ON definers.definerid = tokensb.fid
INNER JOIN ids ON ids.eid = tokensa.eid
INNER JOIN files as filesa ON tokensa.fid = filesa.fid
INNER JOIN files as filesb ON tokensb.fid = filesb.fid
WHERE tokensa.eid = tokensb.eid
GROUP BY tokensa.eid, definerid, basefileid
ORDER BY defined, identifier

Create a function and macro call graph:

SELECT source.name AS CallingFunction, dest.name AS CalledFunction
FROM fcalls
INNER JOIN functions AS source ON fcalls.sourceid = source.id
INNER JOIN functions AS dest ON fcalls.destid = dest.id

Details of the Collected Metrics

Metrics Common to Files and Functions

Number of C preprocessor directives

See note 1.

Number of processed C preprocessor conditionals (ifdef, if, elif)

See note 1.

Number of defined C preprocessor function-like macros

See note 1.

Number of defined C preprocessor object-like macros

See note 1.

Number of preprocessed tokens

Although during preprocessing whitespace is considered a valid token, this metric does not take whitespace tokens into account. This makes it easy to compare the number of preprocessed tokens with the number of compiled tokens. The two metrics are equal if no macro expansion takes place.

Number of compiled tokens

See note 1.

File-Specific Metrics

Number of statements

This metric measures number of statements parsed while processing the file, including statements generated by macro expansion. See note 1.

Number of defined project-scope functions

See note 1.

Number of defined file-scope (static) functions

See note 1.

Number of defined project-scope variables

See note 1.

Number of defined file-scope (static) variables

See note 1.

Number of complete aggregate (struct/union) declarations

Also includes complete declarations without a tag. See note 1.

Number of declared aggregate (struct/union) members

See note 1.

Number of complete enumeration declarations

See note 1.

Number of declared enumeration elements

See note 1.

Number of directly included files

This counts the number of header files that were directly included while processing the project's source files. If each file is processed exactly once, the metric is roughly similar to the number of #include directives in the project's files. See also note 1.

Function-Specific Metrics

Number of statements or declarations

Nested statements are counted recursively. Thus

while (a)
	if (b)
		c();

counts as three statements.

Number of operators

See note 2.

Number of unique operators

See note 2.

Number of if statements

See note 3.

Number of else clauses

See note 3.

Number of switch statements

See note 3.

Number of case labels

See note 3.

Number of default labels

See note 3.

Number of break statements

See note 3.

Number of for statements

See note 3.

Number of while statements

This metric does not include the do .. while form. See note 3.

Number of do statements

See note 3.

Number of continue statements

See note 3.

Number of goto statements

See note 3.

Number of return statements

See note 3.

Total number of object and object-like identifiers

Also includes macros.

Number of unique object and object-like identifiers

Also includes macros.

Number of global namespace occupants at function's top

This metric measures the namespace pollution in the object namespace at the point before entering a function. Its value is the count of all macros as well as objects with file and project-scope visibility that are declared at the point it is measured. See note 1. See note 4.

Number of parameters

See note 1.

Maximum level of statement nesting

In order to avoid excessively inflating this metric when measuring sequences of the form

if (a) {
	...
} else if (b) {
	...
} else if (c) {
	...
} else
	...
}

this metric does not take into account the nesting of else clauses. Thus the above code will be given a nesting level of 1, rather than 3, which is implied by the following (actual) reading of the code.

if (a) {
	...
} else
	if (b) {
		...
	} else
		if (c) {
			...
		} else
			...
		}

See note 1. See note 4.

Fan-in (number of calling functions)

This is also listed under a function's details for functions that are not defined (and have not metrics associated with them).

Cyclomatic complexity (control statements)

This metric, CC₁ measures the number of branch points in the function. In order to avoid misleadingly high values that occur from even trivial switch statements, this metric measures the complexity of a switch statement as 1.

Extended cyclomatic complexity (includes branching operators)

This metric, CC₂, takes into account the nodes introduced by the Boolean-AND, boolean-OR, and conditional evaluation operators.

Maximum cyclomatic complexity (includes branching operators and all switch branches)

This metric, CC₃, considers each case label as a separate node.

Structure complexity (Henry and Kafura)

This metric is calculcated as follows.
C_p = (fan_in * fan_out)²

Halstead complexity

This metric is calculcated as follows.
HC = (number_of_operators + number_of_operands) * log₂( unique_number_of_operators + unique_number_of_operands)
Where operands are object identifiers, macros, numeric and character constants. For the purpose of determining unique operands, each numeric or character constant is considered a separate operand.

Information flow metric (Henry and Selig)

This metric is calculcated as follows.
HC_p = CC₁ * C_p