akkartik
/
mu
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

7843 - clean up README after directory reorg

This commit is contained in:
Kartik K. Agaram 2021-03-03 23:24:33 -08:00
parent 71e4f38129
commit 3b81d74813
8 changed files with 77 additions and 107 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
README.baremetal.md
View File

@ -1,45 +0,0 @@
Some apps written in SubX and Mu. Where the rest of this repo relies on a few
Linux syscalls, the apps in this subdirectory interface directly with hardware.
We still need the top-level and apps to build them.
I'd like to eventually test these programs on real hardware, and to that end
they are extremely parsimonious in the hardware they assume:
0. Lots (more than 640KB/1MB[1]) of RAM
1. Pure-graphics video mode (1024x768 pixels) in 256-color mode. At 8x8
pixels per grapheme, this will give us 160x128 graphemes. But it's still
an open question if it's reasonably widely supported by modern hardware.
If it isn't, I'll downsize.
2. Keyboard. Just a partial US keyboard for now. Main qwerty zone only. No
number pad, no function keys, no ctrl/alt/meta/fn/super/capslck/numlck.
That's it:
* No wifi, no networking
* No multitouch, no touchscreen, no mouse
* No graphics acceleration
* No virtual memory, no memory reclamation
Just your processor, gigabytes of RAM[1], a moderately-sized monitor and a
keyboard. (The mouse should also be easy to provide.)
We can't yet read from or write to disk, except for the initial load of the
program. Enabling access to lots of RAM gives up access to BIOS helpers for
the disk.
These programs don't convert to formats like ELF that can load on other
operating systems. There's also currently no code/data segment separation,
just labels and bytes. I promise not to write self-modifying code. Security
and sandboxing is still an open question.
Programs start executing at address 0x9400. See baremetal/boot.hex for
details.
Mu programs always run all their automated tests first. `main` only runs if
there are no failing tests. See baremetal/mu-init.subx for details.
So far the programs have only been tested in Qemu and Bochs emulators.
[1] Though we might need to start thinking of [the PC memory map](https://wiki.osdev.org/Memory_Map_(x86))
as our programs grow past the first 32MB of memory. Mu doesn't yet make any
attempt to understand how much RAM the underlying computer has. Also, writing
to random locations can damage hardware or corrupt storage devices.

73
README.md
View File

@ -10,18 +10,33 @@ Running the code you want to run, and nothing else.
```sh
$ git clone https://github.com/akkartik/mu
$ cd mu
$ ./translate_mu apps/ex2.mu # emit a.elf
$ ./a.elf # add 3 and 4
$ echo $?
7
$ ./translate life.mu # emit a bootable disk.img
$ qemu-system-i386 disk.img
```
<img alt='screenshot of Game of Life running on the Mu computer' src='html/baremetal-life.png'>
([Colorized sources.](http://akkartik.github.io/mu/html/baremetal/life.mu.html)
This is memory-safe code, and most statements map to a single instruction of
machine code.)
Rather than start from some syntax and introduce layers of translation to
implement it, Mu starts from the processor's instruction set and tries to get
to _some_ safe and clear syntax with as few layers of translation as possible.
The emphasis is on internal consistency at any point in time rather than
compatibility with the past. ([More details.](http://akkartik.name/akkartik-convivial-20200607.pdf))
Tests are a key mechanism here for creating a computer that others can make
their own. I want to encourage a style of active and interactive reading with
Mu. If something doesn't make sense, try changing it and see what tests break.
Any breaking change should break some well-named test somewhere. Consequently,
any manual test should be easy to turn into a reproducible automated test. Mu
is a testbed for providing this guarantee. It exposes testable interfaces for
hardware using dependency injection so that tests can run on -- and make
assertions against -- fake hardware. It also is an experiment in [automated
white-box testing](http://akkartik.name/post/tracing-tests) which promises
robust tests for performance, concurrency, fault-tolerance, etc.
Currently Mu requires a 32-bit x86 processor.
## Goals
@ -60,7 +75,7 @@ In priority order:
## Toolchain
The Mu stack consists of:
- the Mu type-safe language;
- the Mu type-safe and memory-safe language;
- SubX, an unsafe notation for a subset of x86 machine code; and
- _bare_ SubX, a more rudimentary form of SubX without certain syntax sugar.
@ -72,38 +87,31 @@ emulator for Mu's supported subset of x86, that's useful for [debugging SubX
programs](subx_debugging.md).
Mu programs build natively either on Linux or on Windows using [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
For Macs and other Unix-like systems, use the emulator:
For Macs and other Unix-like systems, use the (much slower) emulator:
```sh
$ ./translate_mu_emulated apps/ex2.mu # ~2 mins to emit a.elf
$ ./bootstrap run ./a.elf # run in the emulator
$ echo $?
$ ./translate_mu_emulated ex2.mu # ~2 mins to emit disk.img
```
Mu programs can be written for two very different environments:
* With just a Linux kernel. This is the environment that Mu bootstraps itself
into, and it's the option for programming with stdin, stdout and file
descriptors.
* At the top-level, Mu programs emit a bootable image that runs without an OS
(under emulation; I haven't tested on native hardware yet). There's just a
screen and a keyboard, and that's it. No mouse, no hardware acceleration, no
virtual memory, no process separation, no multi-tasking, no persistent
storage, no network.
* Without an OS, by interacting directly with the screen and keyboard. This
is the option for rudimentary pixel graphics. There's currently no mouse, no
hardware acceleration, no virtual memory, no process separation, no multi-tasking,
no persistent storage, no network. These programs have not yet been tested
on native hardware, only on on Qemu and Bochs. But these _baremetal_
programs build from scratch, without any reliance on C. This is the future.
Here is Conway's Game of Life on Mu:
* The top-level is built using tools created under the linux/ sub-directory.
This sub-directory contains an entirely separate set of standard libraries
intended for building programs that run with just a Linux kernel, reading
from stdin and writing to stdout. The Mu compiler is such a program, at
linux/mu.subx.
```sh
$ ./translate_mu_baremetal baremetal/life.mu # emit disk.img
$ qemu-system-i386 disk.img
```
<img alt='screenshot of Game of Life running on Mu without any intervening Operating System' src='html/baremetal-life.png'>
Use `translate_mu` to build programs for Linux, and `translate_mu_baremetal`
for running without Linux. The standard libraries are totally separate for the
two options, so programs for one won't run on the other.
While I currently focus on programs without an OS, the `linux/` sub-directory
is fairly ergonomic. There's a couple of dozen example programs to try out
there. It is likely to be the option for a network stack in the foreseeable
future; I have no idea how to write to disk or interact on the network without
Linux.
## Syntax
@ -173,15 +181,16 @@ If you're still reading, here are some more things to check out:
$ ./a.elf screen
```
- [How to get your text editor set up for Mu and SubX programs.](editor.md)
- [How to get your text editor set up for Mu and SubX programs.](editor/editor.md)
- [Some tips for debugging SubX programs.](subx_debugging.md)
- [Shared vocabulary of data types and functions shared by Mu programs.](vocabulary.md)
Mu programs can transparently call low-level functions written in SubX.
- [A summary](mu_instructions) of how the Mu compiler translates instructions
to SubX. ([colorized version](http://akkartik.github.io/mu/html/mu_instructions.html))
- [A summary](mu_instructions) of how the Mu compiler translates statements
to SubX. Most Mu statements map to a single x86 instruction.
([colorized version](http://akkartik.github.io/mu/html/mu_instructions.html))
- [Some starter exercises for learning SubX](https://github.com/akkartik/mu/pulls)
(labelled `hello`). Feel free to [ping me](mailto:ak@akkartik.com) with any questions.

27
linux/README.md
View File

@ -1,7 +1,8 @@
A set of standard libraries for building programs that run with just a Linux
kernel. Most programs here read from stdin and write to stdout. One of these
programs is the Mu compiler ([colorized sources](http://akkartik.github.io/mu/html/apps/mu.subx.html)).
Some apps written in SubX and Mu. Check out:
Other apps beyond the Mu toolchain:
* `tile`: [An experimental live-updating postfix shell environment](https://mastodon.social/@akkartik/105108305362341204)
that updates as you type. Prototype. Look at this to see what is currently
@ -17,15 +18,13 @@ Some apps written in SubX and Mu. Check out:
* `factorial*`: A simple program to compute factorials in 5 versions, showing
all the different syntax sugars and what they expand to.
* Code unique to phases of our build toolchain:
* Core SubX: `hex`, `survey_elf`, `pack`, `dquotes`, `assort`, `tests`
* Syntax sugar for SubX: `sigils`, `calls`, `braces`
* More ambitious translator for a memory-safe language (in progress): `mu`
The Mu toolchain is also here in the following phases:
* Core SubX: `hex`, `survey_elf`, `pack`, `dquotes`, `assort`, `tests`
* Syntax sugar for SubX: `sigils`, `calls`, `braces`
* More ambitious translator for a memory-safe language (in progress): `mu`
* Miscellaneous test programs.
All SubX apps include binaries. At any commit, an example's binary should be
identical bit for bit with the result of translating the corresponding `.subx`
file. The binary should also be natively runnable on a Linux system running on
Intel x86 processors, either 32- or 64-bit. If either of these invariants is
violated, it's a bug.
The toolchain includes binaries in the repo. At any commit, the binary should
be identical bit for bit with the result of translating the corresponding
`.subx` file. The binary should also be natively runnable on a Linux system
running on Intel x86 processors, either 32- or 64-bit. If either of these
invariants is violated, it's a bug.

4
linux/bootstrap/README.md
View File

@ -4,4 +4,6 @@ a) An emulator for SubX, the subset of the 32-bit x86 instruction set used by
Mu.
b) A second translator for SubX programs that emits identical binaries to the
self-hosting versions in the parent directory.
self-hosting versions in the parent directory. Having two diverse compilers
(one in a familiar language, one with minimal syscall surface area) that emit
identical binaries should help gain confidence in Mu.

0
bootstrap.md → linux/bootstrap/bootstrap.md
View File

1
subx_debugging.md → linux/subx_debugging.md
View File

@ -27,6 +27,7 @@ rudimentary but hopefully still workable toolkit:
mode (`bootstrap run`):
```
$ cd linux
$ ./translate_subx_debug file1.subx file2.subx ... # generating a.elf
$ ./bootstrap --trace run a.elf arg1 arg2
saving trace to 'last_run'

29
subx_bare.md
View File

@ -107,8 +107,9 @@ reader's burden. Here's the order I've been using after opcodes:
Try running this example now:
```sh
$ ./bootstrap translate init.linux apps/ex3.subx -o apps/ex3
$ ./bootstrap run apps/ex3
$ cd linux
$ bootstrap/bootstrap translate 000init.subx ex3.subx -o ex3
$ bootstrap/bootstrap run ex3
$ echo $?
55
```
@ -116,7 +117,8 @@ $ echo $?
If you're on Linux you can also run it natively:
```sh
$ ./apps/ex3
$ chmod +x ex3
$ ./ex3
$ echo $?
55
```
@ -127,21 +129,22 @@ low-level SubX programs.
## Translating SubX programs
This repo includes two translators for bare SubX. The first is [the bootstrap
translator](bootstrap.md) implemented in C++. In addition, you can use SubX to
translate itself. For example, running natively on Linux:
translator](bootstrap/bootstrap.md) implemented in C++. In addition, you can
use SubX to translate itself. For example, running natively on Linux:
```sh
# generate translator phases using the C++ translator
$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/hex.subx -o hex
$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/survey_elf.subx -o survey_elf
$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/pack.subx -o pack
$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/assort.subx -o assort
$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/dquotes.subx -o dquotes
$ ./bootstrap translate init.linux 0*.subx apps/subx-params.subx apps/tests.subx -o tests
$ cd linux
$ bootstrap/bootstrap translate [01]*.subx subx-params.subx hex.subx -o hex
$ bootstrap/bootstrap translate [01]*.subx subx-params.subx survey_elf.subx -o survey_elf
$ bootstrap/bootstrap translate [01]*.subx subx-params.subx pack.subx -o pack
$ bootstrap/bootstrap translate [01]*.subx subx-params.subx assort.subx -o assort
$ bootstrap/bootstrap translate [01]*.subx subx-params.subx dquotes.subx -o dquotes
$ bootstrap/bootstrap translate [01]*.subx subx-params.subx tests.subx -o tests
$ chmod +x hex survey_elf pack assort dquotes tests
# use the generated translator phases to translate SubX programs
$ cat init.linux apps/ex1.subx |./tests |./dquotes |./assort |./pack |./survey_elf |./hex > a.elf
$ cat 000init.subx ex1.subx |./tests |./dquotes |./assort |./pack |./survey_elf |./hex > a.elf
$ chmod +x a.elf
$ ./a.elf
$ echo $?
@ -158,7 +161,7 @@ Or, running in a VM on other platforms (much slower):
```sh
$ ./translate_subx_emulated init.linux apps/ex1.subx # generates identical a.elf to above
$ ./bootstrap run a.elf
$ bootstrap/bootstrap run a.elf
$ echo $?
42
```

5
x86_approx.md
View File

@ -21,8 +21,9 @@ Here's a test Mu program that prints out the bits for 0.5:
It gives different results when emulated and run natively:
```
$ ./translate_mu_debug x.mu # debug mode = error checking
$ ./bootstrap run a.elf
$ cd linux
$ ./translate_debug x.mu # debug mode = error checking
$ bootstrap/bootstrap run a.elf
0x3f000000 # correct
$ ./a.elf
0x3efff000 # wrong