A brief [f]lex tutorial

flex (and lex): Overview

Scanner generators:
 Helps write programs whose control flow is
directed by instances of regular expressions in
the input stream.

Input: a set of Output: C code

regular expressions lex (or flex) implementing a
+ actions scanner:
function: yylex()
file: lex.yy.c

A quick tutorial on Lex 2

Using flex
file: lex.yy.c

lex input spec yylex()

(regexps + lex {
…
actions)
}

compiler
user
supplies
driver
code

main() {…}
or
parser() {…}

A quick tutorial on Lex 3

Input file must be named as filename.l
For compiling the lex file we use
lex filename.l
For executing
cc lex.yy.c
./a.out

A quick tutorial on Lex 4

flex: input format
An input file has the following structure:

definitions
required %%
rules optional
%%
user code

Shortest possible legal flex input:

%%

A quick tutorial on Lex 5

Definitions
 A series of:
 name definitions, each of the form
name definition
e.g.:
DIGIT [0-9]
CommentStart "/*"
ID [a-zA-Z][a-zA-Z0-9]*
 start conditions
 stuff to be copied verbatim into the flex output
(e.g., declarations, #includes):
– enclosed in %{ … }%, or
– indented
A quick tutorial on Lex 6
Rules
 The rules portion of the input contains a
sequence of rules.
 Each rule has the form
pattern action
where:
 pattern describes a pattern to be matched on the
input
 pattern must be un-indented
 action must begin on the same line.

A quick tutorial on Lex 7

Patterns
 Essentially, extended regular expressions.
 Syntax: similar to grep (see man page)
 <<EOF>> to match “end of file”
 Character classes:
– [:alpha:], [:digit:], [:alnum:], [:space:], etc. (see man
page)
 {name} where name was defined earlier.
 “start conditions” can be used to specify that a
pattern match only in specific situations.

A quick tutorial on Lex 8

Example
A flex program to read a file of (positive) integers and
compute the average:
%{
#include <stdio.h>
#include <stdlib.h>
%}
dgt [0-9]
%%
{dgt}+ return atoi(yytext);
%%
void main()
{
int val, total = 0, n = 0;
while ( (val = yylex()) > 0 ) {
total += val;
n++;
}
if (n > 0) printf(“ave = %d\n”, total/n);
}

A quick tutorial on Lex 9

Example
A flex program to read a file of (positive) integers and
compute the average:
%{ Definition for a digit
definitions

#include <stdio.h> (could have used builtin definition [:digit:] instead)

#include <stdlib.h>
%}
dgt [0-9]
%% Rule to match a number and return its value to
the calling routine
rules

{dgt}+ return atoi(yytext);

%%
void main()
{
int val, total = 0, n = 0;
user code

while ( (val = yylex()) > 0 ) { Driver code

total += val; (could instead have been in a separate file)
n++;
}
if (n > 0) printf(“ave = %d\n”, total/n);
}

A quick tutorial on Lex 10

Example
A flex program to read a file of (positive) integers and
compute the average:
%{
definitions

#include <stdio.h> defining and using a name

#include <stdlib.h>
%}
dgt [0-9]
%%
rules

{dgt}+ return atoi(yytext);

%%
void main()
{
int val, total = 0, n = 0;
user code

while ( (val = yylex()) > 0 ) {

total += val;
n++;
}
if (n > 0) printf(“ave = %d\n”, total/n);
}

A quick tutorial on Lex 11

Example
A flex program to read a file of (positive) integers and
compute the average:
%{
definitions

#include <stdio.h> defining and using a name

#include <stdlib.h>
%}
dgt [0-9]
%% char * yytext;
rules

{dgt}+ return atoi(yytext); a buffer that holds the input

%% characters that actually match the
void main() pattern
{
int val, total = 0, n = 0;
user code

while ( (val = yylex()) > 0 ) {

total += val;
n++;
}
if (n > 0) printf(“ave = %d\n”, total/n);
}

A quick tutorial on Lex 12

Example
A flex program to read a file of (positive) integers and
compute the average:
%{
definitions

#include <stdio.h> defining and using a name

#include <stdlib.h>
%}
dgt [0-9]
%% char * yytext;
rules

{dgt}+ return atoi(yytext); a buffer that holds the input

%% characters that actually match the
void main() pattern
{
int val, total = 0, n = 0;
user code

while ( (val = yylex()) > 0 ) {

Invoking the scanner: yylex()
total += val;
Each time yylex() is called, the
n++;
scanner continues processing
}
the input from where it last left
if (n > 0) printf(“ave = %d\n”, total/n); off.
} Returns 0 on end-of-file.

A quick tutorial on Lex 13