andy/mpc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Readme updates

Browse Source
This commit is contained in:
Daniel Holden
2014-02-12 20:51:45 +00:00
parent 084bdea367
commit a25dc955a0
1 changed files with 128 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files

268
README.md
View File

@@ -37,45 +37,40 @@ _mpc_ provides a number of features that this project does not offer, and also o
* _mpc_ Doesn't pollute the namespace
Demonstration
=============
Quickstart
==========
In the below example I create a parser for basic mathematical expressions.
Here is how one would use _mpc_ to create a parser for a basic mathematical expression language.
```c
#include "mpc.h"
mpc_parser_t *Expr = mpc_new("expression");
mpc_parser_t *Prod = mpc_new("product");
mpc_parser_t *Value = mpc_new("value");
mpc_parser_t *Maths = mpc_new("maths");
void parse_maths(const char *input) {
mpca_lang(
" \
expression : <product> (('+' | '-') <product>)*; \
product : <value> (('*' | '/') <value> )*; \
value : /[0-9]+/ | '(' <expression> ')'; \
maths : /^/ <expression> /$/; \
",
Expr, Prod, Value, Maths);
mpc_parser_t *Expr = mpc_new("expression");
mpc_parser_t *Prod = mpc_new("product");
mpc_parser_t *Value = mpc_new("value");
mpc_parser_t *Maths = mpc_new("maths");
mpca_lang(
" \
expression : <product> (('+' | '-') <product>)*; \
product : <value> (('*' | '/') <value> )*; \
value : /[0-9]+/ | '(' <expression> ')'; \
maths : /^/ <expression> /$/; \
",
Expr, Prod, Value, Maths);
mpc_result_t r;
if (!mpc_parse("<parse_maths>", input, Maths, &r)) {
mpc_ast_print(r.output);
mpc_ast_delete(r.output);
} else {
mpc_err_print(r.error);
mpc_err_delete(r.error);
}
mpc_cleanup(4, Expr, Prod, Value, Maths);
mpc_result_t r;
if (mpc_parse("input", input, Maths, &r)) {
mpc_ast_print(r.output);
mpc_ast_delete(r.output);
} else {
mpc_err_print(r.error);
mpc_err_delete(r.error);
}
mpc_cleanup(4, Expr, Prod, Value, Maths);
```
If you were to input `"(4 * 2 * 11 + 2) - 5"` into this function, the output would look something like this:
If you were to set `input` to the string `(4 * 2 * 11 + 2) - 5`, the printed output would look like this.
```
>:
@@ -144,7 +139,7 @@ Matches any single given character in the range `s` to `e` (inclusive)
* * *
```c
mpc_parser_t *mpc_oneof(const char* s);
mpc_parser_t *mpc_oneof(const char *s);
```
Matches any single given character in the string `s`
@@ -152,7 +147,7 @@ Matches any single given character in the string `s`
* * *
```c
mpc_parser_t *mpc_noneof(const char* s);
mpc_parser_t *mpc_noneof(const char *s);
```
Matches any single given character not in the string `s`
@@ -167,7 +162,7 @@ Matches any single given character satisfying function `f`
* * *
```c
mpc_parser_t *mpc_string(const char* s);
mpc_parser_t *mpc_string(const char *s);
```
Matches exactly the string `s`
@@ -188,8 +183,8 @@ Consumes no input, always successful, returns `NULL`
* * *
```c
mpc_parser_t *mpc_fail(const char* m);
mpc_parser_t *mpc_failf(const char* fmt, ...);
mpc_parser_t *mpc_fail(const char *m);
mpc_parser_t *mpc_failf(const char *fmt, ...);
```
Consumes no input, always fails with message `m`.
@@ -197,7 +192,7 @@ Consumes no input, always fails with message `m`.
* * *
```c
mpc_parser_t *mpc_failf(const char* fmt, ...);
mpc_parser_t *mpc_failf(const char *fmt, ...);
```
Consumes no input, always fails with formatted message given by `fmt` and following parameters.
@@ -213,7 +208,7 @@ Consumes no input, always successful, returns the result of function `f`
* * *
```c
mpc_parser_t *mpc_lift_val(mpc_val_t* x);
mpc_parser_t *mpc_lift_val(mpc_val_t *x);
```
Consumes no input, always successful, returns `x`
@@ -261,7 +256,7 @@ Run a parser on some pipe (such as `stdin`).
* * *
```c
int mpc_parse_contents(const char* filename, mpc_parser_t *p, mpc_result_t* r);
int mpc_parse_contents(const char *filename, mpc_parser_t *p, mpc_result_t *r);
```
Run a parser on the contents of some file.
@@ -281,8 +276,8 @@ Here are the main combinators and how to use then.
* * *
```c
mpc_parser_t *mpc_expect(mpc_parser_t *a, const char* e);
mpc_parser_t *mpc_expectf(mpc_parser_t *a, const char* fmt, ...);
mpc_parser_t *mpc_expect(mpc_parser_t *a, const char *e);
mpc_parser_t *mpc_expectf(mpc_parser_t *a, const char *fmt, ...);
```
Returns a parser that runs `a`, and on success returns the result of `a`, while on failure reports that `e` was expected.
@@ -291,7 +286,7 @@ Returns a parser that runs `a`, and on success returns the result of `a`, while
```c
mpc_parser_t *mpc_apply(mpc_parser_t *a, mpc_apply_t f);
mpc_parser_t *mpc_apply_to(mpc_parser_t *a, mpc_apply_to_t f, void* x);
mpc_parser_t *mpc_apply_to(mpc_parser_t *a, mpc_apply_to_t f, void *x);
```
Returns a parser that applies function `f` (optionality taking extra input `x`) to the result of parser `a`.
@@ -404,7 +399,7 @@ typedef mpc_val_t*(*mpc_fold_t)(int,mpc_val_t**);
This takes a list of pointers to data values and must return some combined or folded version of these data values. It must ensure to free and input data that is no longer used once after combination has taken place.
Case Study - C Identifier
Case Study - Identifier
=========================
Combinator Method
@@ -486,7 +481,7 @@ Building parsers in the above way can have issues with self-reference or cyclic-
* * *
```c
mpc_parser_t *mpc_new(const char* name);
mpc_parser_t *mpc_new(const char *name);
```
This will construct a parser called `name` which can then be used as input to others, including itself, without fear of being deleted. Any parser created using `mpc_new` is said to be _retained_. This means it will behave differently to a normal parser when referenced. When deleting a parser that includes a _retained_ parser, the _retained_ parser will not be deleted along with it. To delete a retained parser `mpc_delete` must be used on it directly.
@@ -526,35 +521,35 @@ Library Reference
Common Parsers
--------------
* `mpc_soi(void);` Matches only the start of input, returns `NULL`
* `mpc_eoi(void);` Matches only the end of input, returns `NULL`
* `mpc_whitespace(void);` Matches any whitespace character `" \f\n\r\t\v"`
* `mpc_whitespaces(void);` Matches zero or more whitespace characters
* `mpc_blank(void);` Matches whitespaces and frees the result, returns `NULL`
* `mpc_newline(void);` Matches `'\n'`
* `mpc_tab(void);` Matches `'\t'`
* `mpc_escape(void);` Matches a backslash followed by any character
* `mpc_digit(void);` Matches any character in the range `'0'` - `'9'`
* `mpc_hexdigit(void);` Matches any character in the range `'0'` - `'9'` as well as `'A'` - `'F'` and `'a'` - `'f'`
* `mpc_octdigit(void);` Matches any character in the range `'0'` - `'7'`
* `mpc_digits(void);` Matches one or more digit
* `mpc_hexdigits(void);` Matches one or more hexdigit
* `mpc_octdigits(void);` Matches one or more octdigit
* `mpc_lower(void);` Matches and lower case character
* `mpc_upper(void);` Matches any upper case character
* `mpc_alpha(void);` Matches and alphabet character
* `mpc_underscore(void);` Matches `'_'`
* `mpc_alphanum(void);` Matches any alphabet character, underscore or digit
* `mpc_int(void);` Matches digits and returns an `int*`
* `mpc_hex(void);` Matches hexdigits and returns an `int*`
* `mpc_oct(void);` Matches octdigits and returns an `int*`
* `mpc_number(void);` Matches `mpc_int`, `mpc_hex` or `mpc_oct`
* `mpc_real(void);` Matches some floating point number as a string
* `mpc_float(void);` Matches some floating point number and returns a `float*`
* `mpc_char_lit(void);` Matches some character literal surrounded by `'`
* `mpc_string_lit(void);` Matches some string literal surrounded by `"`
* `mpc_regex_lit(void);` Matches some regex literal surrounded by `/`
* `mpc_ident(void);` Matches a C style identifier
* `mpc_soi;` Matches only the start of input, returns `NULL`
* `mpc_eoi;` Matches only the end of input, returns `NULL`
* `mpc_whitespace` Matches any whitespace character `" \f\n\r\t\v"`
* `mpc_whitespaces` Matches zero or more whitespace characters
* `mpc_blank` Matches whitespaces and frees the result, returns `NULL`
* `mpc_newline` Matches `'\n'`
* `mpc_tab` Matches `'\t'`
* `mpc_escape` Matches a backslash followed by any character
* `mpc_digit` Matches any character in the range `'0'` - `'9'`
* `mpc_hexdigit` Matches any character in the range `'0'` - `'9'` as well as `'A'` - `'F'` and `'a'` - `'f'`
* `mpc_octdigit` Matches any character in the range `'0'` - `'7'`
* `mpc_digits` Matches one or more digit
* `mpc_hexdigits` Matches one or more hexdigit
* `mpc_octdigits` Matches one or more octdigit
* `mpc_lower` Matches and lower case character
* `mpc_upper` Matches any upper case character
* `mpc_alpha` Matches and alphabet character
* `mpc_underscore` Matches `'_'`
* `mpc_alphanum` Matches any alphabet character, underscore or digit
* `mpc_int` Matches digits and returns an `int*`
* `mpc_hex` Matches hexdigits and returns an `int*`
* `mpc_oct` Matches octdigits and returns an `int*`
* `mpc_number` Matches `mpc_int`, `mpc_hex` or `mpc_oct`
* `mpc_real` Matches some floating point number as a string
* `mpc_float` Matches some floating point number and returns a `float*`
* `mpc_char_lit` Matches some character literal surrounded by `'`
* `mpc_string_lit` Matches some string literal surrounded by `"`
* `mpc_regex_lit` Matches some regex literal surrounded by `/`
* `mpc_ident` Matches a C style identifier
Useful Parsers
@@ -567,14 +562,14 @@ Useful Parsers
* `mpc_stripr(mpc_parser_t *a);` Matches `a` striping any whitespace to the right
* `mpc_strip(mpc_parser_t *a);` Matches `a` striping any surrounding whitespace
* `mpc_tok(mpc_parser_t *a);` Matches `a` and strips any trailing whitespace
* `mpc_sym(const char* s);` Matches string `s` and strips any trailing whitespace
* `mpc_sym(const char *s);` Matches string `s` and strips any trailing whitespace
* `mpc_total(mpc_parser_t *a, mpc_dtor_t da);` Matches the whitespace stripped `a`, enclosed in the start and end of input
* `mpc_between(mpc_parser_t *a, mpc_dtor_t ad, const char* o, const char* c);` Matches `a` between strings `o` and `c`
* `mpc_between(mpc_parser_t *a, mpc_dtor_t ad, const char *o, const char *c);` Matches `a` between strings `o` and `c`
* `mpc_parens(mpc_parser_t *a, mpc_dtor_t ad);` Matches `a` between `"("` and `")"`
* `mpc_braces(mpc_parser_t *a, mpc_dtor_t ad);` Matches `a` between `"<"` and `">"`
* `mpc_brackets(mpc_parser_t *a, mpc_dtor_t ad);` Matches `a` between `"{"` and `"}"`
* `mpc_squares(mpc_parser_t *a, mpc_dtor_t ad);` Matches `a` between `"["` and `"]"`
* `mpc_tok_between(mpc_parser_t *a, mpc_dtor_t ad, const char* o, const char* c);` Matches `a` between `o` and `c`, where `o` and `c` have their trailing whitespace striped.
* `mpc_tok_between(mpc_parser_t *a, mpc_dtor_t ad, const char *o, const char *c);` Matches `a` between `o` and `c`, where `o` and `c` have their trailing whitespace striped.
* `mpc_tok_parens(mpc_parser_t *a, mpc_dtor_t ad);` Matches `a` between trailing whitespace stripped `"("` and `")"`
* `mpc_tok_braces(mpc_parser_t *a, mpc_dtor_t ad);` Matches `a` between trailing whitespace stripped `"<"` and `">"`
* `mpc_tok_brackets(mpc_parser_t *a, mpc_dtor_t ad);` Matches `a` between trailing whitespace stripped `"{"` and `"}"`
@@ -584,35 +579,35 @@ Useful Parsers
Apply Functions
---------------
* `void mpcf_dtor_null(mpc_val_t* x);` Empty destructor. Does nothing
* `mpc_val_t* mpcf_ctor_null(void);` Returns `NULL`
* `mpc_val_t* mpcf_ctor_str(void);` Returns `""`
* `mpc_val_t* mpcf_free(mpc_val_t* x);` Frees `x` and returns `NULL`
* `mpc_val_t* mpcf_int(mpc_val_t* x);` Converts a decimal string `x` to an `int*`
* `mpc_val_t* mpcf_hex(mpc_val_t* x);` Converts a hex string `x` to an `int*`
* `mpc_val_t* mpcf_oct(mpc_val_t* x);` Converts a oct string `x` to an `int*`
* `mpc_val_t* mpcf_float(mpc_val_t* x);` Converts a string `x` to a `float*`
* `mpc_val_t* mpcf_escape(mpc_val_t* x);` Converts a string `x` to an escaped version
* `mpc_val_t* mpcf_escape_regex(mpc_val_t* x);` Converts a regex `x` to an escaped version
* `mpc_val_t* mpcf_escape_string_raw(mpc_val_t* x);` Converts a raw string `x` to an escaped version
* `mpc_val_t* mpcf_escape_char_raw(mpc_val_t* x);` Converts a raw character `x` to an escaped version
* `mpc_val_t* mpcf_unescape(mpc_val_t* x);` Converts a string `x` to an unescaped version
* `mpc_val_t* mpcf_unescape_regex(mpc_val_t* x);` Converts a regex `x` to an unescaped version
* `mpc_val_t* mpcf_unescape_string_raw(mpc_val_t* x);` Converts a raw string `x` to an unescaped version
* `mpc_val_t* mpcf_unescape_char_raw(mpc_val_t* x);` Converts a raw character `x` to an unescaped version
* `void mpcf_dtor_null(mpc_val_t *x);` Empty destructor. Does nothing
* `mpc_val_t *mpcf_ctor_null(void);` Returns `NULL`
* `mpc_val_t *mpcf_ctor_str(void);` Returns `""`
* `mpc_val_t *mpcf_free(mpc_val_t *x);` Frees `x` and returns `NULL`
* `mpc_val_t *mpcf_int(mpc_val_t *x);` Converts a decimal string `x` to an `int*`
* `mpc_val_t *mpcf_hex(mpc_val_t *x);` Converts a hex string `x` to an `int*`
* `mpc_val_t *mpcf_oct(mpc_val_t *x);` Converts a oct string `x` to an `int*`
* `mpc_val_t *mpcf_float(mpc_val_t *x);` Converts a string `x` to a `float*`
* `mpc_val_t *mpcf_escape(mpc_val_t *x);` Converts a string `x` to an escaped version
* `mpc_val_t *mpcf_escape_regex(mpc_val_t *x);` Converts a regex `x` to an escaped version
* `mpc_val_t *mpcf_escape_string_raw(mpc_val_t *x);` Converts a raw string `x` to an escaped version
* `mpc_val_t *mpcf_escape_char_raw(mpc_val_t *x);` Converts a raw character `x` to an escaped version
* `mpc_val_t *mpcf_unescape(mpc_val_t *x);` Converts a string `x` to an unescaped version
* `mpc_val_t *mpcf_unescape_regex(mpc_val_t *x);` Converts a regex `x` to an unescaped version
* `mpc_val_t *mpcf_unescape_string_raw(mpc_val_t *x);` Converts a raw string `x` to an unescaped version
* `mpc_val_t *mpcf_unescape_char_raw(mpc_val_t *x);` Converts a raw character `x` to an unescaped version
Fold Functions
--------------
* `mpc_val_t* mpcf_null(int n, mpc_val_t** xs);` Returns `NULL`
* `mpc_val_t* mpcf_fst(int n, mpc_val_t** xs);` Returns first element of `xs`
* `mpc_val_t* mpcf_snd(int n, mpc_val_t** xs);` Returns second element of `xs`
* `mpc_val_t* mpcf_trd(int n, mpc_val_t** xs);` Returns third element of `xs`
* `mpc_val_t* mpcf_fst_free(int n, mpc_val_t** xs);` Returns first element of `xs` and calls `free` on others
* `mpc_val_t* mpcf_snd_free(int n, mpc_val_t** xs);` Returns second element of `xs` and calls `free` on others
* `mpc_val_t* mpcf_trd_free(int n, mpc_val_t** xs);` Returns third element of `xs` and calls `free` on others
* `mpc_val_t* mpcf_strfold(int n, mpc_val_t** xs);` Concatenates all `xs` together as strings and returns result
* `mpc_val_t* mpcf_maths(int n, mpc_val_t** xs);` Examines second argument as string to see which operator it is, then operators on first and third argument as if they are `int*`.
* `mpc_val_t *mpcf_null(int n, mpc_val_t** xs);` Returns `NULL`
* `mpc_val_t *mpcf_fst(int n, mpc_val_t** xs);` Returns first element of `xs`
* `mpc_val_t *mpcf_snd(int n, mpc_val_t** xs);` Returns second element of `xs`
* `mpc_val_t *mpcf_trd(int n, mpc_val_t** xs);` Returns third element of `xs`
* `mpc_val_t *mpcf_fst_free(int n, mpc_val_t** xs);` Returns first element of `xs` and calls `free` on others
* `mpc_val_t *mpcf_snd_free(int n, mpc_val_t** xs);` Returns second element of `xs` and calls `free` on others
* `mpc_val_t *mpcf_trd_free(int n, mpc_val_t** xs);` Returns third element of `xs` and calls `free` on others
* `mpc_val_t *mpcf_strfold(int n, mpc_val_t** xs);` Concatenates all `xs` together as strings and returns result
* `mpc_val_t *mpcf_maths(int n, mpc_val_t** xs);` Examines second argument as string to see which operator it is, then operators on first and third argument as if they are `int*`.
Case Study - Maths Language
@@ -628,7 +623,7 @@ As an example of this power, we can specify a simple maths grammar, that ouputs
We start with a fold function that will fold two `int *` into a new `int *` based on some `char *` operator.
```c
mpc_val_t* fold_maths(int n, mpc_val_t **xs) {
mpc_val_t *fold_maths(int n, mpc_val_t **xs) {
int **vs = (int**)xs;
@@ -647,38 +642,31 @@ mpc_val_t* fold_maths(int n, mpc_val_t **xs) {
And then we use this to specify a basic grammar, which folds together any results.
```c
int parse_maths(char* input) {
mpc_parser_t *Expr = mpc_new("expr");
mpc_parser_t *Factor = mpc_new("factor");
mpc_parser_t *Term = mpc_new("term");
mpc_parser_t *Maths = mpc_new("maths");
mpc_parser_t *Expr = mpc_new("expr");
mpc_parser_t *Factor = mpc_new("factor");
mpc_parser_t *Term = mpc_new("term");
mpc_parser_t *Maths = mpc_new("maths");
mpc_define(Expr, mpc_or(2,
mpc_and(3, fold_maths,
Factor, mpc_oneof("*/"), Factor,
free, free),
Factor
));
mpc_define(Expr, mpc_or(2,
mpc_and(3, fold_maths, Factor, mpc_oneof("*/"), Factor, free, free),
Factor
));
mpc_define(Factor, mpc_or(2,
mpc_and(3, fold_maths, Term, mpc_oneof("+-"), Term, free, free),
Term
));
mpc_define(Term, mpc_or(2, mpc_int(), mpc_parens(Expr, free)));
mpc_define(Maths, mpc_enclose(Expr, free));
mpc_result_t r;
if (!mpc_parse("parse_maths", input, Maths, &r)) {
mpc_err_print(r.error);
abort();
}
mpc_define(Factor, mpc_or(2,
mpc_and(3, fold_maths,
Term, mpc_oneof("+-"), Term,
free, free),
Term
));
int result = *r.output;
printf("Result: %i\n", result);
free(r.output);
return result;
}
mpc_define(Term, mpc_or(2, mpc_int(), mpc_parens(Expr, free)));
mpc_define(Maths, mpc_whole(Expr, free));
/* Do Some Parsing... */
mpc_delete(Maths);
```
If we supply this function with something like `(4*2)+5`, we can expect it to output `13`.
@@ -709,22 +697,22 @@ String literals are surrounded in double quotes `"`. Character literals in singl
Parts specified one after another are parsed in order (like `mpc_and`), while parts separated by a pipe `|` are alternatives (like `mpc_or`). Parenthesis `()` are used to specify precedence. `*` can be used to mean zero or more of. `+` for one or more of. `?` for zero or one of. `!` for negation. And a number inside braces `{5}` to means so many counts of.
Rules are specified by rule name, optionally followed by an _expected_ string, followed by a colon `:`, followed by the definition, and ending in a semicolon `;`.
Rules are specified by rule name, optionally followed by an _expected_ string, followed by a colon `:`, followed by the definition, and ending in a semicolon `;`. The flags variable is a set of flags `MPC_LANG_DEFAULT`, `MPC_LANG_PREDICTIVE`, or `MPC_LANG_WHITESPACE_SENSITIVE`. For specifying if the language is predictive or whitespace sensitive.
Like with the regular expressions, this user input is parsed by existing parts of the _mpc_ library. It provides one of the more powerful features of the library.
* * *
```c
mpc_parser_t *mpca_grammar(int flags, const char* grammar, ...);
mpc_parser_t *mpca_grammar(int flags, const char *grammar, ...);
```
This takes in some single right hand side of a rule, as well as a list of any of the parsers it refers to, and outputs a parser that does exactly what is specified by the rule. The flags variable is a set of flags `MPC_LANG_DEFAULT`, `MPC_LANG_PREDICTIVE`, or `MPC_LANG_WHITESPACE_SENSITIVE`. For specifying if the language is predictive or whitespace sensitive.
This takes in some single right hand side of a rule, as well as a list of any of the parsers it refers to, and outputs a parser that does exactly what is specified by the rule.
* * *
```c
mpc_err_t* mpca_lang(int flags, const char* lang, ...);
mpc_err_t* mpca_lang(int flags, const char *lang, ...);
```
This takes in a full language (zero or more rules) as well as any parsers referred to by either the right or left hand sides. Any parsers specified on the left hand side of any rule will be assigned a parser equivalent to what is specified on the right. On valid user input this returns `NULL`, while if there are any errors in the user input it will return an instance of `mpc_err_t` describing the issues.
@@ -740,7 +728,7 @@ This reads in the contents of file `f` and inputs it into `mpca_lang`.
* * *
```c
mpc_err_t* mpca_lang_contents(int flags, const char* filename, ...);
mpc_err_t* mpca_lang_contents(int flags, const char *filename, ...);
```
This opens and reads in the contents of the file given by `filename` and passes it to `mpca_lang`.
@@ -758,14 +746,14 @@ _mpc_ provides some automatic generation of error messages. These can be enhance
Limitations & FAQ
=================
### ASCII
### Does this support Unicode?
Only supports ASCII. Sorry!
_mpc_ Only supports ASCII. Sorry! I welcome contributions as making the library support Unicode is non-trivial.
### Backtracking and Left Recursion
MPC supports backtracking, but will not completely backtrack up a parse tree if it encounters some success on the path it is going. To demonstrate this behaviour examine the following erroneous grammar, intended to parse either a C style identifier, or a C style function call.
_mpc_ supports backtracking, but will not completely backtrack up a parse tree if it encounters some success on the path it is going. To demonstrate this behaviour examine the following erroneous grammar, intended to parse either a C style identifier, or a C style function call.
```
factor : <ident>
@@ -788,7 +776,7 @@ factor : <ident> ('(' <expr>? (',' <expr>)* ')')? ;
```
### Max String Length
### How can I avoid the maximum string literal length?
Some compilers limit the maximum length of string literals. If you have a huge language string in the source file to be passed into `mpca_lang` you might encounter this. The ANSI standard says that 509 is the maximum length allowed for a string literal. Most compilers support greater than this. Visual Studio supports up to 2048 characters, while gcc allocates memory dynamically and so has no real limit.