Even though the standard library is building and passing tests, the
binaries it generates aren't exactly bit for bit identical with the
originals. Comparing using `diff_ntranslate`, it looks like the data
segment starting address isn't computed right in survey.subx
(`compute-addresses`) when I start translating layer 058. Deleting some
tests brings the code segment to a p_offset where bits 8-11 (the lowest
4 bits excluding the lowermost byte) are cleared and everything works.
However, if bits 8-11 are set, then they don't make it to p_vaddr and
p_paddr.
Tried reproducing with a unit test, but the unit test passes fine.
Translates 5k lines of input in 26 seconds.
I'm not sure why I need to grow the label table. It was already 512 entries
long, and I'm only using 373 so far.
Bugfix fourteen: we need different address computation logic for code vs
data labels.
It's really about different categories of instructions having different
address computation logic. This subtle distinction will make good error
messages hard. But that's a problem for later.
Now there's just one example program not translating.
Figured out what's going in with bug fourteen: displacement operands
aren't always used relative to the PC. Does this mean I need to track
instruction boundaries past pack? :'(
No, I just need different logic for labels in code vs data segments.
This was an interesting bug for reminding me of the difference between
the emulator-level trace and the application-level trace. The former has
1.5 million lines, while the latter has a dozen. Luckily, just dumping
the latter immediately made obvious what the issue was.
Though this experience does suggest some further ideas for debugging
tools:
slice trace by line and phase
slice trace by start and end label
debug UI for SubX translator
2D layout: rows = lines of code; columns = translator phases
each 'cell' in this layout contains a list of log lines
shows what came in, what was emitted
easily collapse any cell
These are domain-specific tools. Special-cased to the SubX translator
phases.
Bugfix thirteen: displacement calculations were wrong because current
offset was not being updated properly as words were being read and
emitted.
Now 10/12 example programs are translated correctly.
Bugfix ten: type error in `convert`. I was calling `rewind-stream` on a
`buffered-file`.
examples/ex1 is now just one nibble off the canonical.
I *have* found one missing feature in the self-hosted translator,
though: dquotes doesn't support newlines in strings, even though the C++
version does. dquotes parses them right, but the value initialized in
the data segment is wrong.
Bugfix eight: incorrect segment count in ELF header.
The generated examples/ex1 is still not right. But it has the second
segment now. Or almost all of it. Final byte is missing for some reason.
The result isn't an identical binary to before, and it segfaults when
run. But it's bugfix seven.
A couple of places where we make .subx files a little more strict:
a) All .subx files must define a data segment. Even if they have no
data.
b) All .subx files must define an `Entry` label for the binary to start
at. Earlier we used to default to the start of the code label. That's
not too hard to add; we'd just need to:
i) rename `get` to `get-or-abort`
ii) clone a third variant of `get-or-insert` called `get` that returns
null if the key is not found.
iii) use `get` rather than `get-or-abort` when looking up the `Entry`
label.
map of how far we've gotten by now (functions with '*' independently tested):
✓ compute-offsets*
✓ compute-addresses*
✓ emit-output
✓ emit-headers
✓ emit-elf-header
✓ emit-hex-array*
✓ first emit-elf-program-header-entry
✓ emit-hex-array*
? second emit-elf-program-header-entry
emit-hex-array*
emit-segments*
I carefully logged the segment a label is declared in but forgot to
actually save it in the table. This has been a theoretic concern for
some time, but I've never seen it actually happen until now. SubX is
just too low level.
Now I get past the first two phases but code generation fails to find
the 'Entry' label.
Snapshot at a random moment, showing a new debugging trick: hacking on
the C++ level to dump memory contents on specific labels.
For some reason label 'x' doesn't have a segment assigned by the time we
get to compute-addresses.
'curr-segment-name' is now a string, and it's stored in a register
rather than a global.
Paradoxically, this leaks *less* than before. Before, every call to
`get-or-insert-slice` leaked memory. Now we leak one string for every
new segment. Which is trivial.
I changed the test a little to make it obvious.
Basically there's no way we can compute the segment offset correctly
without knowing the segment name in the previous assertion.
The pseudocode was a mess here :/ I was saving the segment-offset but
tracing the file-offset.
Segments need file offsets (to tweak their starting address).
Labels need segment offsets (to add to segment starting address).
