Storing assembly instructions in a list felt nice since that lets me
just fire and forget, but if we want to properly optimize short
branches (i.e. branches that do not require a full JMP) we'll need to
be able to rewrite assembly after linking, and I feel that's best done
by making assembly instructions hold a NEXT rather than poking at a
list clumsily.
The first linking pass was also moved out to a separate function since
we may want to run the first pass twice, first after the initial
assembling and second after expanding optimistic short branches (those
that turned out to be longer branches than 127/128 bytes) into a long
branch snippet (branch with inverted condition skipping over a JMP).
It replaces the IR-FETCHVAR's result with the variable being
fetched. This only works when the only use is in an operation that
does not require a separate fetch be performed, such as those
implemented as CPU instructions.
The compiler middle stage takes high level nodes and produces code in
an intermediate representation more closely resembling assembly code.
Optimizations and the tools for making those are also included. It's
significantly easier to optimize IR than syntax trees or assembly.
Several things need cleaning up, in particular there are things in
jigs.lisp that really should be documented tools, not
jigs (specifically the compilation setup and finalization).
It ingests tokens via a "token stream", and feeds out a node
tree. It's a mostly handwritten recursive descent parser with the
occasional Lisp macros for convenience.
The new nodes are more directly representational of the syntax, and
code generation will be moved to an intermediate stage with a more
linear (yet still node-based) representation.
