Context-Free Grammars

A context-free grammar (CFG) is a set of recursive rewriting rules (or productions) used to generate patterns of strings.

A CFG consists of the following components:

a set of terminal symbols, which are the characters of the alphabet that appear in the strings generated by the grammar.
a set of nonterminal symbols, which are placeholders for patterns of terminal symbols that can be generated by the nonterminal symbols.
a set of productions, which are rules for replacing (or rewriting) nonterminal symbols (on the left side of the production) in a string with other nonterminal or terminal symbols (on the right side of the production).
a start symbol, which is a special nonterminal symbol that appears in the initial string generated by the grammar.

To generate a string of terminal symbols from a CFG, we:

Begin with a string consisting of the start symbol;
Apply one of the productions with the start symbol on the left hand size, replacing the start symbol with the right hand side of the production;
Repeat the process of selecting nonterminal symbols in the string, and replacing them with the right hand side of some corresponding production, until all nonterminals have been replaced by terminal symbols.

A CFG for Arithmetic Expressions

An example grammar that generates strings representing arithmetic expressions with the four operators +, -, *, /, and numbers as operands is:

<expression> --> number
<expression> --> ( <expression> )
<expression> --> <expression> + <expression>
<expression> --> <expression> - <expression>
<expression> --> <expression> * <expression>
<expression> --> <expression> / <expression>

The only nonterminal symbol in this grammar is <expression>, which is also the start symbol. The terminal symbols are {+,-,*,/,(,),number}. (We will interpret "number" to represent any valid number.)

The first rule (or production) states that an <expression> can be rewritten as (or replaced by) a number. In other words, a number is a valid expression.

The second rule says that an <expression> enclosed in parentheses is also an <expression>. Note that this rule defines an expression in terms of expressions, an example of the use of recursion in the definition of context-free grammars.

The remaining rules say that the sum, difference, product, or division of two <expression>s is also an expression.

Generating Strings from a CFG

In our grammar for arithmetic expressions, the start symbol is <expression>, so our initial string is:

      <expression>

Using rule 5 we can choose to replace this nonterminal, producing the string:

      <expression> * <expression>

We now have two nonterminals to replace. We can apply rule 3 to the first nonterminal, producing the string:

      <expression> + <expression> * <expression>

We can apply rule two to the first nonterminal in this string to produce:

      (<expression>) + <expression> * <expression>

If we apply rule 1 to the remaining nonterminals (the recursion must end somewhere!), we get:

      (number) + number * number

This is a valid arithmetic expression, as generated by the grammar.

When applying the rules above, we often face a choice as to which production to choose. Different choices will typically result in different strings being generated.

Given a grammar G with start symbol S, if there is some sequence of productions that, when applied to the initial string S, result in the string s, then s is in L(G), the language of the grammar.

CFGs with Epsilon Productions

A CFG may have a production for a nonterminal in which the right hand side is the empty string (which we denote by epsilon). The effect of this production is to remove the nonterminal from the string being generated.

Here is a grammar for balanced parentheses that uses epsilon productions.

      P --> ( P )
      P --> P P
      P --> epsilon

We begin with the string P. We can replace P with epsilon, in which case we have generated the empty string (which does have balanced parentheses). Alternatively, we can generate a string of balanced parentheses within a pair of balanced parentheses, which must result in a string of balanced parentheses. Alternatively, we can concatenate two strings of balanced parentheses, which again must result in a string of balanced parentheses.

This grammar is equivalent to:

      P --> ( P ) | P P | epsilon

We use the notational shorthand '|', which can be read as "or", to represent multiple rewriting rules within a single line.

CFG Examples

A CFG describing strings of letters with the word "main" somewhere in the string:

  <program> --> <letter*> m a i n <letter*>
  <letter*> --> <letter> <letter*> | epsilon
  <letter> --> A | B | ... | Z | a | b ... | z

A CFG for the set of identifiers in Pascal:

  <id> --> <L> <LorD*> 
  <LorD*> --> <L> <LorD*> | <D> <LorD*> | epsilon
  <L> --> A | B | ... | Z | a | b ... | z
  <D> --> 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

A CFG describing real numbers in Pascal:

  <real> --> <digit> <digit*> <decimal part> <exp>
  <digit*> --> <digit> <digit*> | epsilon
  <decimal part> --> '.' <digit> <digit*> | epsilon
  <exp> --> 'E' <sign> <digit> <digit*> | epsilon
  <sign> --> + | - | epsilon
  <digit> --> 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

A CFG for C++ compound statements:

  <compound stmt> --> { <stmt list> }
  <stmt list> --> <stmt> <stmt list> | epsilon
  <stmt> --> <compound stmt>
  <stmt> --> if ( <expr> ) <stmt>
  <stmt> --> if ( <expr> ) <stmt> else <stmt>
  <stmt> --> while ( <expr> ) <stmt>
  <stmt> --> do <stmt> while ( <expr> ) ;
  <stmt> --> for ( <stmt> <expr> ; <expr> ) <stmt>
  <stmt> --> case <expr> : <stmt>
  <stmt> --> switch ( <expr> ) <stmt>
  <stmt> --> break ; | continue ;
  <stmt> --> return <expr> ; | goto <id> ;

Finding all the Strings Generated by a CFG

There are several ways to generate the (possibly infinite) set of strings generated by a grammar. We will show a technique based on the number of productions used to generate the string.

Find the strings generated by the following CFG:

      <S> --> w c d <S> | b <L> e | s
      <L> --> <L> ; <S> | <S>

0. Applying at most zero productions, we cannot generate any strings.

1. Applying at most one production (starting with the start symbol) we can generate {wcd<S>, b<L>e, s}. Only one of these strings consists entirely of terminal symbols, so the set of terminal strings we can generate using at most one production is {s}.

2. Applying at most two productions, we can generate all the strings we can generate with one production, plus any additional strings we can generate with an additional production.

  {wcdwcd<S>, wcdb<L>e, wcds, b<S>e, b<L>;<S>e,s}

The set of terminal strings we can generate with at most two productions is therefore {s, wcds}.

3. Applying at most three productions, we can generate:

  {wcdwcdwcd<S>, wcdwcdb<L>e, wcdwcds, wcdb<L>;<S>e,
   wcdb<S>e, bwcd<S>e, bb<L>ee, bse, b<L>;<S>Se,
   b<S><S>e, b<L>wcd<S>e, b<L>b<L>ee, b<L>se }

The set of terminal strings we can generate with at most three productions is therefore {s, wcds, wcdwcds, bse}.

We can repeat this process for an arbitrary number of steps N, and find all the strings the grammar can generate by applying N productions.

CFGs vs Regular Expressions

Context-free grammars are strictly more powerful than regular expressions.

Any language that can be generated using regular expressions can be generated by a context-free grammar.
There are languages that can be generated by a context-free grammar that cannot be generated by any regular expression.

As a corollary, CFGs are strictly more powerful than DFAs and NDFAs.

The proof is in two parts:

Given a regular expression R , we can generate a CFG G such that L(R) == L(G).
We can define a grammar G for which there there is no FA F such that L(F) == L(G).

Simulating a Regular Expression with a CFG

To show that CFGs are at least as powerful as regular expressions, we show how to simulate a RE using a CFG. The construction is similar to the one used to simulate a regular expression with a FA; we build the CFG G in pieces, where each piece corresponds to the operands and operators in the regular expression.

Assume the RE is a single operand. Then if RE is epsilon or a character in the alphabet, add to G the production
```
      <RE> --> RE
  
```
If RE is null, don't add a production.
Assume the RE is R1R2. Add to G the production
```
      <RE> --> <R1> <R2>
  
```
and create productions for regular expressions R1 and R2.
Assume the RE is R1 | R2. Add to G the production
```
      <RE> --> <R1> | <R2>
  
```
and create productions for regular expressions R1 and R2.
Assume the RE is R1*. Add to G the production
```
      <RE> --> <R1> <RE> | epsilon
  
```
and create productions for regular expression R1.

Example: RE to CFG

We will build a CFG G for the RE (0|1)*111.

First the operands:

      <0> --> 0
      <1> --> 1

Now the innermost operator, union:

      <R1> --> <0> | <1>

Now the closure operator:

      <R2> --> <R1> <R2> | epsilon

Now the concatenation operators:

      <RE> --> R2 R3 R4 R5
      <R3> --> <1>
      <R4> --> <1>
      <R5> --> <1>

The final grammar G is:

      <RE> --> R2 R3 R4 R5
      <R2> --> <R1> <R2> | epsilon
      <R1> --> <0> | <1>
      <R3> --> <1>
      <R4> --> <1>
      <R5> --> <1>
      <0> --> 0
      <1> --> 1

A CFG with no Corresponding RE

Recall that FA cannot count. Thus, no FA can recognize the language {0^n 1^n|n>=1} (i.e., the set of strings containing one or more zeros followed by an equal number of ones).

Assume such an FA exists, and it has N states. What happens when the input string has N+1 zeros in it, followed by N+1 ones?

Since the FA only has N states, we must visit some state sT twice on seeing N+1 zeros.
The FA cannot know whether we are entering sT for the first time, when we've seen i < N zeros, or the second time, when we've seen j > i zeros.
There must be a path from sT to an accepting state, since the input string is in the language.
The FA will accept an input string without an equal number of zeros and ones, since i != j, and there is a path to an accepting state from sT on the remaining input.

This language is generated by the following CFG:

<S> --> 0 <S> 1
<S> --> 01

We can prove that this grammar generates the language by induction on n, the number of zeros and ones in the string.

For the basis step, n = 1, and the string is 01. This string is generated by applying the second production once.
For the inductive step, assume we can generate O^n1^n. The last production applied must have been production 2, so the string must have been 0^(n-1)<S>1^(n-1). If we apply production 1 and then production 2, we get 0^n<S>1^n, and then 0^(n+1)1^(n+1). Thus, we can generate all strings of the form {0^n 1^n|n>=1}.
Since we can only apply production 1 some number of times followed by production 2, these are the only strings generated by the grammar.