Index

ANS
API
Debugger
Download
Licensing
Links
Locals
OOP In Ficl
Parse Steps
Release History
Upgrading To 4.0

Unlike every other FORTH we know of, Ficl features an extensible parser chain. The Ficl parser is not a monolithic function; instead, it is comprised of a simple tokenizer and a series of parse steps. A parse step is a step in the parser chain that handles a particular kind of token, acting on the token as appropriate. Example parse steps, in terms of traditional FORTH lore, would be the "number runner" and the "colon compiler".

The Ficl parser works like this:

1. Read in a new token (string of text with no internal whitespace).
2. For each parse step in the chain, call the parse step, passing in the token. If the parse step returns FICL_TRUE, that parse step must have handled the token appropriately; move on to the next token.
3. If the parser tries all the parse steps and none of them return FICL_TRUE, the token is illegal—print an error and reset the virtual machine.

Parse steps can be written as native functions, or as Ficl script functions. New parse steps can be appended to the chain at any time.

These is the default Ficl parser chain, shown in order.

?word

If compiling and local variable support is enabled, attempt to find the token in the local variable dictionary. If found, execute the token's compilation semantics and return FICL_TRUE.

Attempt to find the token in the system dictionary. If found, execute the token's semantics (may be different when compiling than when interpreting) and return FICL_TRUE.

?prefix

This parse step is only active if prefix support is enabled, setting FICL_WANT_PREFIX in ficl.h to a non-zero value. Attempt to match the beginning of the token to the list of known prefixes. If there's a match, execute the associated prefix method and return FICL_TRUE.

?number

Attempt to convert the token to a number in the present BASE. If successful, push the value onto the stack if interpreting, otherwise compile it, then return FICL_TRUE.

?float

This parse step is only active if floating-point number support is enabled, setting FICL_WANT_FLOAT in ficl.h to a non-zero value. Attempt to convert the token to a floating-point number. If successful, push the value onto the floating-point stack if interpreting, otherwise compile it, then return FICL_TRUE.

You can add a parse step in two ways. The first is to write a Ficl word that has the correct stack signature for a parse step:

MY-PARSE-STEP   ( c-addr u -- x*i flag )

where c-addr u are the address and length of the incoming token, and flag is FICL_TRUE if the parse step processed the token and FICL_FALSE otherwise.

Install the parse step using add-parse-step. A trivial example:

: ?silly   ( c-addr u -- flag )
   ." Oh no! Not another  " type cr  true ;
' ?silly add-parse-step
parse-order

The other way to add a parse step is to write it in C and add it into the parse chain with the following function:

void ficlSystemAddPrimitiveParseStep(ficlSystem *system, char *name, ficlParseStep step);

name is the display name of the parse step in the parse chain (as displayed by the Ficl word PARSE-ORDER). step is a pointer to the code for the parse step itself, and must match the following declaration:

typedef int (*ficlParseStep)(ficlVm *vm, ficlString s);

When a native parse step is run, si points to the incoming token. The parse step must return FICL_TRUE if it succeeds in handling the token, and FICL_FALSE otherwise. See ficlVmParseNumber() in system.c for an example.

What's a prefix, anyway? A prefix (contributed by Larry Hastings) is a token that's recognized as the beginning of another token. Its presence modifies the semantics of the rest of the token. An example is 0x, which causes digits following it to be converted to hex regardless of the current value of BASE.

Caveat: Prefixes are matched in sequence, so the more of them there are, the slower the interpreter gets. On the other hand, because the prefix parse step occurs immediately after the dictionary lookup step, if you have a prefix for a particular purpose, using it may save time since it stops the parse process. Also, the Ficl interpreter is wonderfully fast, and most interpretation only happens once, so it's likely you won't notice any change in interpreter speed even if you make heavy use of prefixes.

Each prefix is a Ficl word stored in a special wordlist called <PREFIXES>. When the prefix parse step (?prefix, implemented in C as ficlVmParsePrefix()) is executed, it searches each word in <PREFIXES> in turn, comparing it with the initial characters of the incoming token. If a prefix matches, the parse step returns the remainder of the token to the input stream and executes the code associated with the prefix. This code can be anything you like, but it would typically do something with the remainder of the token. If the prefix code does not consume the rest of the token, it will go through the parse process again (which may be what you want).

Prefixes are defined in prefix.c and in softcore/prefix.fr. The best way to add prefixes is by defining them in your own code, bracketed with the special words START-PREFIXES and END-PREFIXES. For example, the following code would make .( a prefix.

start-prefixes
: .(  .( ;
end-prefixes

The compile-time constant FICL_EXTENDED_PREFIX controls the inclusion of several additional prefixes. This is turned off in the default build, since several of these prefixes alter standard behavior, but you might like them.

• Prefixes and parser extensions are non-standard. However, with the exception of prefix support, Ficl's default parse order follows the standard. Inserting parse steps in some other order will almost certainly break standard behavior.

• The number of parse steps that can be added to the system is limited by the value of FICL_MAX_PARSE_STEPS (defined in sysdep.h). The default maximum number is 8.

• The compile-time constant FICL_EXTENDED_PREFIX controls the inclusion of several additional prefixes. This is turned off in the default build, since several of these prefixes alter standard behavior, but you might like them.

PARSE-ORDER ( -- )

Prints the list of parse steps, in the order in which they are called.

ADD-PARSE-STEP ( xt -- )

Appends a parse step to the parse chain. xt is the address (execution token) of a Ficl word to use as the parse step. The word must be a legal Ficl parse step (see above).

SHOW-PREFIXES ( -- )

Prints the list of all prefixes. Each prefix is a Ficl word that is executed if its name is found at the beginning of a token.

START-PREFIXES ( -- )

Declares the beginning of a series of prefix definitions. Should be followed, eventually, by END-PREFIXES. (All START-PREFIXES does is tell the Ficl virtual machine to compile into the <PREFIXES> wordlist.)

END-PREFIXES ( -- )

Declares the end of a series of prefix definitions. Should only be used after calling START-PREFIXES. (All END-PREFIXES does is tell the Ficl virtual machine to switch back to the wordlist that was in use before START-PREFIXES was called.)