I've found two bugs in SubX libraries:
1. next-word had an out-of-bounds read
2. next-word was skipping comments, because that's what I need during bootstrapping.
I've created a new variant called next-raw-word that doesn't skip comments.
These really need better names.
We're now at the point where 4b.mu has the right structure and returns
identical result to 4a.mu.
Both manual tests described in commit 7222 now work.
To make them work I had to figure out how to copy a file. It
requires a dependency on a new syscall: lseek.
No support for combining characters. Graphemes are currently just utf-8
encodings of a single Unicode code-point. No support for code-points that
require more than 32 bits in utf-8.
Several bugs fixed in the process, and expectation of further bugs is growing.
I'd somehow started assuming I don't need to have separate cases for rm32
as a register vs mem. That's not right. We might need more reg-reg Primitives.
Cleaner abstraction, but adds 3 instructions to our overhead for handles,
including one potentially-hard-to-predict jump :/
I wish I could have put the alloc id in eax for the comparison as well,
to save a few bytes of instruction space. But that messes up the non-null
case.
Mystery solved of why the syntax sugar phases don't work even though they
don't use any functions whose signatures changed in the migration to handles.
The answer: they use the Registers table, and it needs to use handles rather
than raw strings.
Mystery solved of why the syntax sugar phases don't work even though they
don't use any functions whose signatures changed in the migration to handles.
The answer: they use the Registers table, and it currently doesn't use
handles.
Rather than create a whole new set of functions that operate on addresses,
I'm going to create fake handles that are never intended to be reclaimed.
Which raises the question of the best way to do that. I'd like to continue
using string syntax, so I'm going to use a prefix in the payload that can
also be rendered as a string. But all the printable characters start with
0x20, and we don't currently have escape sequences for null or any other
non-printable characters.
I _could_ use newlines, but that seems overly clever. So instead I'll once
again not worry about some hypothetical problem with running out of alloc-ids,
and just carve out half of the id space that can't be used for real alloc
ids. Ascii doesn't use the most significant bit of bytes, so it seems like
a natural separation.
So far it's unclear how to do this in a series of small commits. Still
nibbling around the edges. In this commit we standardize some terminology:
The length of an array or stream is denominated in the high-level elements.
The _size_ is denominated in bytes.
The thing we encode into the type is always the size, not the length.
There's still an open question of what to do about the Mu `length` operator.
I'd like to modify it to provide the length. Currently it provides the
size. If I can't fix that I'll rename it.
At the SubX level we have to put up with null-terminated kernel strings
for commandline args. But so far we haven't done much with them. Rather
than try to support them we'll just convert them transparently to standard
length-prefixed strings.
In the process I realized that it's not quite right to treat the combination
of argc and argv as an array of kernel strings. Argc counts the number
of elements, whereas the length of an array is usually denominated in bytes.
Anytime we create a slice, the first check tends to be whether it's empty.
If we handle ill-formed slices here where start > end, that provides a
measure of safety.
In the Mu translator (mu.subx) we often check for a trailing ':' or ','
and decrement slice->end to ignore it. But that could conceivably yield
ill-formed slices if the slice started out empty. Now we make sure we never
operate on such ill-formed slices.
Layers 0-89 are used in self-hosting SubX.
Layers 90-99 are not needed for self-hosting SubX, and therefore could
use transitional levels of syntax sugar.
Layers 100 and up use all SubX syntax sugar.
Support binary operations with reg/mem and reg operands.
Everything is passing. However, the self-hosting translator now generates
some discrepancies compared to the C++ translator :(
Move stack operations to a layer of their own.
It was some short-term pain to take out the syntax sugar from it, but we
need access to this layer from braces, which can't depend on sugar since
it's part of sugar. Just simpler to keep one clear line and not have to
build sometimes with some sugar but not others.
This undoes 5672 in favor of a new plan:
Layers 000 - 099 are for running without syntax sugar. We use them for
building syntax-sugar passes.
Layers 100 and up are for running with all syntax sugar.
The layers are arranged in approximate order so more phases rely on earlier
layers than later ones.
I plan to not use intermediate syntax sugar (just sigils without calls,
or sigils and calls without braces) anywhere except in the specific passes
implementing them.
Kartik Agaram