I've found two bugs in SubX libraries:
1. next-word had an out-of-bounds read
2. next-word was skipping comments, because that's what I need during bootstrapping.
I've created a new variant called next-raw-word that doesn't skip comments.
These really need better names.
We're now at the point where 4b.mu has the right structure and returns
identical result to 4a.mu.
Both manual tests described in commit 7222 now work.
To make them work I had to figure out how to copy a file. It
requires a dependency on a new syscall: lseek.
No support for combining characters. Graphemes are currently just utf-8
encodings of a single Unicode code-point. No support for code-points that
require more than 32 bits in utf-8.
1000+ LoC spent; just 300+ excluding tests.
Still one known gap; we don't check the entirety of an array's element
type if it's a compound. So far we just check if say both sides start with
'addr'. Obviously that's not good enough.
Several bugs fixed in the process, and expectation of further bugs is growing.
I'd somehow started assuming I don't need to have separate cases for rm32
as a register vs mem. That's not right. We might need more reg-reg Primitives.
I thought I'd done this in the previous commit, but I hadn't. And, what's
more, there was a bug that seemed pretty tough for a time. Turns out my
self-hosted translator doesn't support '.' comment tokens in data segments.
Hopefully I'm past the valley of the shadow of death now.
"I HAVE NO TOOLS BECAUSE I’VE DESTROYED MY TOOLS WITH MY TOOLS."
-- James Mickens (https://www.usenix.org/system/files/1311_05-08_mickens.pdf)
Cleaner abstraction, but adds 3 instructions to our overhead for handles,
including one potentially-hard-to-predict jump :/
I wish I could have put the alloc id in eax for the comparison as well,
to save a few bytes of instruction space. But that messes up the non-null
case.
Mystery solved of why the syntax sugar phases don't work even though they
don't use any functions whose signatures changed in the migration to handles.
The answer: they use the Registers table, and it needs to use handles rather
than raw strings.
Mystery solved of why the syntax sugar phases don't work even though they
don't use any functions whose signatures changed in the migration to handles.
The answer: they use the Registers table, and it currently doesn't use
handles.
Rather than create a whole new set of functions that operate on addresses,
I'm going to create fake handles that are never intended to be reclaimed.
Which raises the question of the best way to do that. I'd like to continue
using string syntax, so I'm going to use a prefix in the payload that can
also be rendered as a string. But all the printable characters start with
0x20, and we don't currently have escape sequences for null or any other
non-printable characters.
I _could_ use newlines, but that seems overly clever. So instead I'll once
again not worry about some hypothetical problem with running out of alloc-ids,
and just carve out half of the id space that can't be used for real alloc
ids. Ascii doesn't use the most significant bit of bytes, so it seems like
a natural separation.
So far it's unclear how to do this in a series of small commits. Still
nibbling around the edges. In this commit we standardize some terminology:
The length of an array or stream is denominated in the high-level elements.
The _size_ is denominated in bytes.
The thing we encode into the type is always the size, not the length.
There's still an open question of what to do about the Mu `length` operator.
I'd like to modify it to provide the length. Currently it provides the
size. If I can't fix that I'll rename it.
At the SubX level we have to put up with null-terminated kernel strings
for commandline args. But so far we haven't done much with them. Rather
than try to support them we'll just convert them transparently to standard
length-prefixed strings.
In the process I realized that it's not quite right to treat the combination
of argc and argv as an array of kernel strings. Argc counts the number
of elements, whereas the length of an array is usually denominated in bytes.
If indexing into a type with power-of-2-sized elements we can access them
in one instruction:
x/reg1: (addr int) <- index A/reg2: (addr array int), idx/reg3: int
This translates to a single instruction because x86 instructions support
an addressing mode with left-shifts.
For non-powers-of-2, however, we need a multiply. To keep things type-safe,
it is performed like this:
x/reg1: (offset T) <- compute-offset A: (addr array T), idx: int
y/reg2: (addr T) <- index A, x
An offset is just an int that is guaranteed to be a multiple of size-of(T).
Offsets can only be used in index instructions, and the types will eventually
be required to line up.
In the process, I have to expand Input-size because mu.subx is growing
big.
Anytime we create a slice, the first check tends to be whether it's empty.
If we handle ill-formed slices here where start > end, that provides a
measure of safety.
In the Mu translator (mu.subx) we often check for a trailing ':' or ','
and decrement slice->end to ignore it. But that could conceivably yield
ill-formed slices if the slice started out empty. Now we make sure we never
operate on such ill-formed slices.
Layers 0-89 are used in self-hosting SubX.
Layers 90-99 are not needed for self-hosting SubX, and therefore could
use transitional levels of syntax sugar.
Layers 100 and up use all SubX syntax sugar.
Kartik Agaram