Another commit, another bugfix.
Some snippets from my currently exploding todo list:
- always investigate lookup errors immediately. Beyond the root cause, they should never happen at the moment, while we aren't reclaiming memory.
we should always return a more precise error message. Usually involving null pointer checks.
- on abort, print out stack trace
- emit mapping of labels to addresses during survey
- store a mapping of symbols somewhere in the code image
- stop allocating 1KB per token; expand space for tokens as needed
In the process I spent a long time tracking down a stray TODO in 108write.subx
that I thought would abort but didn't since the switch to baremetal.
Then after I reintroduced that assertion I had to go track down a bunch
of buffer sizes. Stream sizes continue to be a huge mess.
Current plan:
- some way to define macros. For now:
(def f (litmac litfn () (a b) `(+ ,a , b)))
- macroexpand will expand calls by passing them through the cdr
(f 3 4)
macroexpand: ((litfn () (a b) `(+ ,a ,b)) 3 4)
=> (+ 3 4)
eval: (+ 3 4) => 7
I don't understand why a second line in the keyboard is visible now where
it wasn't before. That whole aspect has unclear desires. What exactly do
I want to happen on newlines?
I keep running into one hole in Mu's memory-safety since dropping the Linux
dependency: null pointers no longer error when dereferenced. Here the problem
manifests as aliasing: lots of gap buffers share the same exact data near
address 0, because it was never initialized.
It requires more than 1GB to fill the screen with a chessboard pattern
using the definition in shell/iterative-definitions.limg.
I also speed up the chessboard program by clearing the screen up front
and then only rendering the white pixels.
It took me _way_ too long to realize that I'm not checking for errors within
the loop, and that will cause it to manifest as an infinite loop as inner
evaluations fail to run.
Debugging notes, for posterity:
printing one row of a chessboard pattern over fake screen (chessboard screen 4 0 0 15) gets stuck in an infinite loop halfway through
debug pattern during infinite loop: VWEX. It's still in the loop but it's not executing the body
raw (fill_rect screen 16 0 20 4 15) works fine
same number of calls to fill_rect work fine
replacing calls to fill_rect with pixel inside chessboard2 works fine
at the point of the infinite loop it's repeatedly going through the hline loop
-- BUT it never executes the check of the loop (< lo hi) with lo=20, hi=20. Something is returning 1, but it's not inside <
stream optimization is not implicated
simple test case with a single loop
(
(globals . (
(foo . (fn () (screen i n)
(while (< i n)
(pixel screen 4 4 i)
(pixel screen 5 4 i)
(pixel screen 6 4 i)
(pixel screen 7 4 i)
(set i (+ i 1)))))
))
(sandbox . (foo screen 0 100))
)
simpler (if you reset cursor position before every print):
(
(globals . (
(foo . (fn () (screen i n)
(while (< i n)
(print screen i)
(set i (+ i 1)))))
))
(sandbox . (foo screen 0 210))
)
I now believe it has nothing to do with the check. The check always works.
Sometimes no body is evaluated. And so the set has no effect.
All highly experimental. Current constraints:
* No tail recursion elimination
* No heap reuse
* Keep implementation simple
So it's slow, and I don't want to complicate it to speed it up. So I'm
investing in affordances to help deal with the slowness. However, in the
process I've taken the clean abstraction of a trace ("all you need to do
is add to the trace") and bolted on call counts and debug-prints as independent
mechanisms.
Before: we always drew pixels atop characters, and we only drew pixels
that were explicitly requested.
After: we always draw pixels atop characters, and we only draw pixels that
don't have color 0.
Both semantics should be identical as long as pixels are never drawn atop
characters.
Filling pixels isn't a rare corner case. I'm going to switch to a dense
rather than sparse representation for pixels, but callers will have to
explicitly request the additional memory.
We now have a couple of protections:
- if we get close to running out of space in the trace we drop in an
error
- if we run out of space in the trace we stop trying to append
- if there are errors we cancel future evaluations
This is already much nicer. You can't do much on the Mu computer, but at
least it gracefully gives up and shows its limitations. On my computer
the Mu shell tries to run computations for about 20s before giving up.
That seems at the outer limit of what interactivity supports. If things
take too long, test smaller chunks.
Among other things, we turned off the trace to significantly speed up the
debug cycle.
State as of https://merveilles.town/@akkartik/106079258606146213
Ohhh, as I save the commit I notice a big problem: I've been editing the
disk image directly because writes to the Mu disk lose indentation. But
I've been forgetting that the state in the Mu disk needs to be pre-evaluated.
So function bindings need extra parens for the environment. The `pixel`
calls in the previous commit message are the first statement in the body,
and they aren't actually considered part of the body right now. No wonder
they don't run.
There are lots of other problems, but this will clarify a lot.
I tried building a function to draw a horizontal line across the screen.
Here's what I have in data.txt:
(
(globals . (
(horline . (fn () (screen y)
(horline_1 screen y 0 (width screen))))
(horline_1 . (fn () (screen y lo hi)
(if (>= lo hi)
()
((fn ()
(pixel screen lo y 12)
(horline_1 screen y (+ lo 1) hi))))))
))
(sandbox . (horline_1 screen 0 0 20))
)
$ dd if=/dev/zero of=data.img count=20160
$ cat data.txt |dd of=data.img conv=notrunc
$ ./translate shell/*.mu && qemu-system-i386 -hda disk.img -hdb data.img
Result: I can't call (horline screen 0) over a fake screen of width 40.
Some stream overflows somewhere after all the tweaks to various fixed-size
buffers scattered throughout the app. Calling horline_1 gets to a 'hi'
column of 20, but not to 30.
I just realized Mu has a pretty big weakness: writes to null pointers don't
error out. Perhaps writes to address 0 do, but address 1 and so on don't?
I need a slightly more sophisticated page table.
Mu's keyboard handling is currently a bit of a mess, and this commit might
be a bad idea.
Ideally keyboards would return Unicode. Currently Mu returns single bytes.
Mostly ASCII. No support for international keyboards yet.
ASCII and Unicode have some keyboard scancodes grandfathered in, that don't
really make sense for data transmission. Like backspace and delete. However,
other keyboard scancodes don't have any place in Unicode. Including arrow keys.
So Mu carves out an exception to Unicode for arrow keys. We'll place the
arrow keys in a part of Unicode that is set aside for implementation-defined
behavior (https://en.wikipedia.org/wiki/C0_and_C1_control_codes#C1_controls):
0x80: left arrow
0x81: down arrow
0x82: up arrow
0x83: right arrow
The order is same as hjkl for mnemonic convenience. I'd _really_ to follow
someone else's cannibalization here. If I find one later, I'll switch to
it.
Applications that blindly assume the keyboard generates Unicode will have
a bad time. Events like backspace, delete and arrow keys are intended to
be processed early and should not be in text.
With a little luck I won't need to modify this convention when I support
international keyboards.
Tested by inserting a call into the shell, but we can't leave it in because
every test ends up clobbering the disk. So it's now time to think about
a testable interface for the disk.
Baremetal is now the default build target and therefore has its sources
at the top-level. Baremetal programs build using the phase-2 Mu toolchain
that requires a Linux kernel. This phase-2 codebase which used to be at
the top-level is now under the linux/ directory. Finally, the phase-2 toolchain,
while self-hosting, has a way to bootstrap from a C implementation, which
is now stored in linux/bootstrap. The bootstrap C implementation uses some
literate programming tools that are now in linux/bootstrap/tools.
So the whole thing has gotten inverted. Each directory should build one
artifact and include the main sources (along with standard library). Tools
used for building it are relegated to sub-directories, even though those
tools are often useful in their own right, and have had lots of interesting
programs written using them.
A couple of things have gotten dropped in this process:
- I had old ways to run on just a Linux kernel, or with a Soso kernel.
No more.
- I had some old tooling for running a single test at the cursor. I haven't
used that lately. Maybe I'll bring it back one day.
The reorg isn't done yet. Still to do:
- redo documentation everywhere. All the README files, all other markdown,
particularly vocabulary.md.
- clean up how-to-run comments at the start of programs everywhere
- rethink what to do with the html/ directory. Do we even want to keep
supporting it?
In spite of these shortcomings, all the scripts at the top-level, linux/
and linux/bootstrap are working. The names of the scripts also feel reasonable.
This is a good milestone to take stock at.
