/usr/share/doc/alex/html/introduction.html is in alex 3.1.0-1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | <html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>Chapter 2. Introduction</title><link rel="stylesheet" type="text/css" href="fptools.css"><meta name="generator" content="DocBook XSL Stylesheets V1.78.1"><link rel="home" href="index.html" title="Alex User Guide"><link rel="up" href="index.html" title="Alex User Guide"><link rel="prev" href="license.html" title="1.6. License"><link rel="next" href="syntax.html" title="Chapter 3. Alex Files"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 2. Introduction</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="license.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="syntax.html">Next</a></td></tr></table><hr></div><div class="chapter"><div class="titlepage"><div><div><h1 class="title"><a name="introduction"></a>Chapter 2. Introduction</h1></div></div></div><p>Alex is a tool for generating lexical analysers in Haskell,
given a description of the tokens to be recognised in the form of
regular expressions. It is similar to the tools
lex and flex for C/C++.</p><p>Alex takes a description of tokens based on regular
expressions and generates a Haskell module containing code for
scanning text efficiently. Alex is designed to be familiar to
existing lex users, although it does depart from lex in a number
of ways.</p><div class="figure-float" style="float: left;"><div class="figure"><a name="fig-tokens"></a><p class="title"><b>Figure 2.1. A simple Alex specification.</b></p><div class="figure-contents"><pre class="programlisting">{
module Main (main) where
}
%wrapper "basic"
$digit = 0-9 -- digits
$alpha = [a-zA-Z] -- alphabetic characters
tokens :-
$white+ ;
"--".* ;
let { \s -> Let }
in { \s -> In }
$digit+ { \s -> Int (read s) }
[\=\+\-\*\/\(\)] { \s -> Sym (head s) }
$alpha [$alpha $digit \_ \']* { \s -> Var s }
{
-- Each action has type :: String -> Token
-- The token type:
data Token =
Let |
In |
Sym Char |
Var String |
Int Int
deriving (Eq,Show)
main = do
s <- getContents
print (alexScanTokens s)
}</pre></div></div><br class="figure-break"></div><p>A sample specification is given in <a class="xref" href="introduction.html#fig-tokens" title="Figure 2.1. A simple Alex specification.">Figure 2.1, “A simple Alex specification.”</a>. The first few lines between the
<code class="literal">{</code> and <code class="literal">}</code> provide a code scrap
(some inlined Haskell code) to be placed directly in the output,
the scrap at the top of the module is normally used to declare the
module name for the generated Haskell module, in this case
<code class="literal">Main</code>.</p><p>The next line, <code class="literal">%wrapper "basic"</code> controls
what kind of support code Alex should produce along with the basic
scanner. The <code class="literal">basic</code> wrapper selects a scanner
that tokenises a <code class="literal">String</code> and returns a list of
tokens. Wrappers are described fully in <a class="xref" href="api.html" title="Chapter 5. The Interface to an Alex-generated lexer">Chapter 5, <i>The Interface to an Alex-generated lexer</i></a>.</p><p>The next two lines define the <code class="literal">$digit</code> and
<code class="literal">$alpha</code> macros for use in the token
definitions.</p><p>The ‘<code class="literal">tokens :-</code>’ line ends the
macro definitions and starts the definition of the scanner.</p><p>The scanner is specified as a series of token definitions
where each token specification takes the form of</p><pre class="programlisting"><em class="replaceable"><code>regexp</code></em> { <em class="replaceable"><code>code</code></em> }</pre><p>The meaning of a this rule is "if the input matches
<em class="replaceable"><code>regexp</code></em>, then return
<em class="replaceable"><code>code</code></em>". The code part along with the
braces can be replaced by simply
‘<code class="literal">;</code>’, meaning that this token should
be ignored in the input stream. As you can see, we've used this
to ignore whitespace in our example.</p><p>Our scanner is set up so that the actions are all functions
with type <code class="literal">String->Token</code>. When the token is
matched, the portion of the input stream that it matched is passed
to the appropriate action function as a
<code class="literal">String</code>.</p><p>At the bottom of the file we have another code fragment,
surrounded by braces <code class="literal">{ ... }</code>. In this
fragment, we declare the type of the tokens, and give a
<code class="literal">main</code> function that we can use for testing it;
the <code class="literal">main</code> function just tokenises the input and
prints the results to standard output.</p><p>Alex has kindly provided the following function which we can
use to invoke the scanner:</p><pre class="programlisting">alexScanTokens :: String -> [Token]</pre><p>Alex arranges for the input stream to be tokenised, each of
the action functions to be passed the appropriate
<code class="literal">String</code>, and a list of <code class="literal">Token</code>s
returned as the result. If the input stream is lazy, the output
stream will also be produced lazily<a href="#ftn.idp57299376" class="footnote" name="idp57299376"><sup class="footnote">[1]</sup></a>.</p><p>We have demonstrated the simplest form of scanner here,
which was selected by the <code class="literal">%wrapper "basic"</code> line
near the top of the file. In general, actions do not have to have
type <code class="literal">String->Token</code>, and there's no requirement
for the scanner to return a list of tokens.</p><p>With this specification in the file
<code class="literal">Tokens.x</code>, Alex can be used to generate
<code class="literal">Tokens.hs</code>:</p><pre class="screen">$ alex Tokens.x</pre><p>If the module needed to be placed in different file,
<code class="literal">Main.hs</code> for example, then the output filename
can be specified using the <code class="option">-o</code> option:</p><pre class="screen">$ alex Tokens.x -o Main.hs</pre><p>The resulting module is Haskell 98 compatible. It can also
be readily used with a <a class="ulink" href="http://www.haskell.org/happy/" target="_top">Happy</a> parser.</p><div class="footnotes"><br><hr style="width:100; text-align:left;margin-left: 0"><div id="ftn.idp57299376" class="footnote"><p><a href="#idp57299376" class="para"><sup class="para">[1] </sup></a>that is, unless you
have any patterns that require a long lookahead.</p></div></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="license.html">Prev</a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="syntax.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">1.6. License </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> Chapter 3. Alex Files</td></tr></table></div></body></html>
|