2019-07-28 16:38:34 +00:00
# Mu: a human-scale computer
2019-07-28 00:41:52 +00:00
2019-10-17 17:57:28 +00:00
Mu is a minimal-dependency hobbyist computing stack (everything above the
2021-03-01 04:34:58 +00:00
processor).
2019-08-18 05:17:39 +00:00
2019-10-17 17:57:28 +00:00
Mu is not designed to operate in large clusters providing services for
millions of people. Mu is designed for _you_ , to run one computer. (Or a few.)
Running the code you want to run, and nothing else.
2019-07-28 00:41:52 +00:00
2020-07-12 23:37:58 +00:00
```sh
$ git clone https://github.com/akkartik/mu
$ cd mu
$ ./translate_mu apps/ex2.mu # emit a.elf
2020-12-15 04:02:21 +00:00
$ ./a.elf # add 3 and 4
2020-07-12 23:37:58 +00:00
$ echo $?
7
```
2019-08-10 04:54:07 +00:00
2020-07-05 22:28:37 +00:00
Rather than start from some syntax and introduce layers of translation to
implement it, Mu starts from the processor's instruction set and tries to get
to _some_ safe and clear syntax with as few layers of translation as possible.
The emphasis is on internal consistency at any point in time rather than
compatibility with the past. ([More details.](http://akkartik.name/akkartik-convivial-20200607.pdf))
2021-03-01 05:29:47 +00:00
Currently Mu requires a 32-bit x86 processor.
2020-01-02 04:49:45 +00:00
2019-08-10 04:54:07 +00:00
## Goals
In priority order:
2019-07-28 00:41:52 +00:00
2020-06-06 22:55:03 +00:00
- [Reward curiosity. ](http://akkartik.name/about )
- Easy to build, easy to run. [Minimal dependencies ](https://news.ycombinator.com/item?id=16882140#16882555 ),
2019-07-28 01:05:34 +00:00
so that installation is always painless.
2020-06-06 22:55:03 +00:00
- All design decisions comprehensible to a single individual. (On demand.)
- All design decisions comprehensible without needing to talk to anyone.
2019-07-28 00:41:52 +00:00
(I always love talking to you, but I try hard to make myself redundant.)
2020-06-06 22:55:03 +00:00
- [A globally comprehensible _codebase_ rather than locally clean code. ](http://akkartik.name/post/readable-bad )
- Clear error messages over expressive syntax.
- Safe.
- Thorough test coverage. If you break something you should immediately see
2019-07-28 00:41:52 +00:00
an error message. If you can manually test for something you should be
able to write an automated test for it.
2020-06-06 22:55:03 +00:00
- Memory leaks over memory corruption.
- Teach the computer bottom-up.
2019-07-28 00:41:52 +00:00
2019-08-10 04:54:07 +00:00
## Non-goals
2020-06-06 22:55:03 +00:00
- Speed. Staying close to machine code should naturally keep Mu fast enough.
- Efficiency. Controlling the number of abstractions should naturally keep Mu
2020-03-15 17:54:25 +00:00
using far less than the gigabytes of memory modern computers have.
2020-06-06 22:55:03 +00:00
- Portability. Mu will run on any computer as long as it's x86. I will
2020-03-15 17:54:25 +00:00
enthusiastically contribute to support for other processors -- in separate
forks. Readers shouldn't have to think about processors they don't have.
2020-06-06 22:55:03 +00:00
- Compatibility. The goal is to get off mainstream stacks, not to perpetuate
2019-07-28 00:41:52 +00:00
them. Sometimes the right long-term solution is to [bump the major version number ](http://akkartik.name/post/versioning ).
2020-06-06 22:55:03 +00:00
- Syntax. Mu code is meant to be comprehended by [running, not just reading ](http://akkartik.name/post/comprehension ).
2019-07-28 00:41:52 +00:00
For now it's a thin veneer over machine code. I'm working on memory safety
before expressive syntax.
2020-07-05 22:28:37 +00:00
## Toolchain
2015-11-11 17:13:40 +00:00
2020-07-05 22:28:37 +00:00
The Mu stack consists of:
- the Mu type-safe language;
- SubX, an unsafe notation for a subset of x86 machine code; and
- _bare_ SubX, a more rudimentary form of SubX without certain syntax sugar.
2015-11-11 17:13:40 +00:00
2020-07-12 23:28:24 +00:00
All Mu programs get translated through these layers into tiny zero-dependency
2021-03-01 05:29:47 +00:00
binaries that run natively. The translators for most levels are built out of
lower levels. The translator from Mu to SubX is written in SubX, and the
translator from SubX to bare SubX is built in bare SubX. There is also an
emulator for Mu's supported subset of x86, that's useful for [debugging SubX
programs](subx_debugging.md).
2021-02-18 16:58:37 +00:00
2021-03-01 04:34:58 +00:00
Mu programs build natively either on Linux or on Windows using [WSL 2 ](https://docs.microsoft.com/en-us/windows/wsl/install-win10 ).
2021-02-18 16:58:37 +00:00
For Macs and other Unix-like systems, use the emulator:
2020-12-25 20:26:05 +00:00
```sh
2021-02-18 07:46:25 +00:00
$ ./translate_mu_emulated apps/ex2.mu # ~2 mins to emit a.elf
2020-12-25 20:26:05 +00:00
$ ./bootstrap run ./a.elf # run in the emulator
$ echo $?
```
2021-03-01 04:34:58 +00:00
Mu programs can be written for two very different environments:
2015-11-11 17:13:40 +00:00
2021-03-01 04:34:58 +00:00
* With just a Linux kernel. This is the environment that Mu bootstraps itself
into, and it's the option for programming with stdin, stdout and file
descriptors.
2015-11-11 17:13:40 +00:00
2021-03-01 04:34:58 +00:00
* Without an OS, by interacting directly with the screen and keyboard. This
is the option for rudimentary pixel graphics. There's currently no mouse, no
hardware acceleration, no virtual memory, no process separation, no multi-tasking,
no persistent storage, no network. These programs have not yet been tested
on native hardware, only on on Qemu and Bochs. But these _baremetal_
2021-03-01 05:29:47 +00:00
programs build from scratch, without any reliance on C. This is the future.
2021-03-01 22:58:11 +00:00
Here is Conway's Game of Life on Mu:
2021-02-18 07:46:25 +00:00
2021-03-01 04:34:58 +00:00
```sh
2021-03-01 22:58:11 +00:00
$ ./translate_mu_baremetal baremetal/life.mu # emit disk.img
2021-03-01 04:34:58 +00:00
$ qemu-system-i386 disk.img
```
2021-02-18 07:46:25 +00:00
2021-03-01 22:58:11 +00:00
< img alt = 'screenshot of Game of Life running on Mu without any intervening Operating System' src = 'html/baremetal-life.png' >
2021-02-18 07:46:25 +00:00
2021-03-01 04:34:58 +00:00
Use `translate_mu` to build programs for Linux, and `translate_mu_baremetal`
for running without Linux. The standard libraries are totally separate for the
two options, so programs for one won't run on the other.
2021-02-18 07:46:25 +00:00
2020-07-05 22:28:37 +00:00
## Syntax
2020-03-14 07:58:52 +00:00
2020-07-05 22:28:37 +00:00
The entire stack shares certain properties and conventions. Programs consist
of functions and functions consist of statements, each performing a single
operation. Operands to statements are always variables or constants. You can't
2021-02-07 17:53:14 +00:00
perform `a + b*c` in a single statement; you have to break it up into two.
Variables can live in memory or in registers. Registers must be explicitly
specified. There are some shared lexical rules. Comments always start with
'#'. Numbers are always written in hex. Many terms can have context-dependent
_metadata_ attached after '/'.
2019-07-27 23:01:55 +00:00
2020-07-05 22:28:37 +00:00
Here's an example program in Mu:
2019-07-27 23:01:55 +00:00
2020-11-17 16:20:27 +00:00
< img alt = 'ex2.mu' src = 'html/ex2.mu.png' width = '400px' >
2014-11-01 23:15:15 +00:00
2020-07-05 22:28:37 +00:00
[More details on Mu syntax → ](mu.md )
2014-11-26 16:30:26 +00:00
2020-07-05 22:28:37 +00:00
Here's an example program in SubX:
2014-11-26 16:30:26 +00:00
2020-07-12 23:37:58 +00:00
```sh
== code
Entry:
# ebx = 1
bb/copy-to-ebx 1/imm32
# increment ebx
43/increment-ebx
# exit(ebx)
e8/call syscall_exit/disp32
```
2020-03-12 02:50:49 +00:00
2020-07-05 22:28:37 +00:00
[More details on SubX syntax → ](subx.md )
2015-08-13 15:27:18 +00:00
2020-07-05 22:28:37 +00:00
## Forks
2020-05-10 20:21:52 +00:00
Forks of Mu are encouraged. If you don't like something about this repo, feel
2020-11-18 03:06:19 +00:00
free to make a fork. If you show it to me, I'll link to it here. I might even
pull your changes into this repo!
2020-05-10 20:21:52 +00:00
2020-06-06 22:55:03 +00:00
- [mu-normie ](https://git.sr.ht/~akkartik/mu-normie ): with a more standard
2020-10-23 18:17:57 +00:00
build system that organizes the repo by header files and compilation units.
2020-11-14 19:28:40 +00:00
Stays in sync with this repo.
2020-10-23 07:18:35 +00:00
- [mu-x86\_64 ](https://git.sr.ht/~akkartik/mu-x86_64 ): experimental fork for
64-bit x86 in collaboration with [Max Bernstein ](https://bernsteinbear.com ).
It's brought up a few concrete open problems that I don't have good solutions
for yet.
2020-11-14 19:28:40 +00:00
- [uCISC ](https://github.com/grokthis/ucisc ): a 16-bit processor being
designed from scratch by [Robert Butler ](https://www.youtube.com/channel/UCh4OpfF7T7UtezGejRTLxCw )
and programmed with a SubX-like syntax.
2020-11-21 01:08:01 +00:00
- [subv ](https://git.s-ol.nu/subv ): experimental SubX-like syntax by [s-ol
bekic](https://mmm.s-ol.nu) for the RISC-V instruction set.
2020-05-10 20:21:52 +00:00
2020-07-05 22:28:37 +00:00
## Desiderata
2019-07-27 23:01:55 +00:00
2020-07-05 22:28:37 +00:00
If you're still reading, here are some more things to check out:
2019-07-28 01:08:45 +00:00
2020-07-05 22:28:37 +00:00
- The references on [Mu ](mu.md ) and [SubX ](subx.md ) syntax, and also [bare
SubX](subx_bare.md) without any syntax sugar.
2019-07-28 01:08:45 +00:00
2021-03-01 04:37:57 +00:00
- [Some 2-minute videos demonstrating Mu programs ](https://archive.org/details/@kartik_agaram ).
7842 - new directory organization
Baremetal is now the default build target and therefore has its sources
at the top-level. Baremetal programs build using the phase-2 Mu toolchain
that requires a Linux kernel. This phase-2 codebase which used to be at
the top-level is now under the linux/ directory. Finally, the phase-2 toolchain,
while self-hosting, has a way to bootstrap from a C implementation, which
is now stored in linux/bootstrap. The bootstrap C implementation uses some
literate programming tools that are now in linux/bootstrap/tools.
So the whole thing has gotten inverted. Each directory should build one
artifact and include the main sources (along with standard library). Tools
used for building it are relegated to sub-directories, even though those
tools are often useful in their own right, and have had lots of interesting
programs written using them.
A couple of things have gotten dropped in this process:
- I had old ways to run on just a Linux kernel, or with a Soso kernel.
No more.
- I had some old tooling for running a single test at the cursor. I haven't
used that lately. Maybe I'll bring it back one day.
The reorg isn't done yet. Still to do:
- redo documentation everywhere. All the README files, all other markdown,
particularly vocabulary.md.
- clean up how-to-run comments at the start of programs everywhere
- rethink what to do with the html/ directory. Do we even want to keep
supporting it?
In spite of these shortcomings, all the scripts at the top-level, linux/
and linux/bootstrap are working. The names of the scripts also feel reasonable.
This is a good milestone to take stock at.
2021-03-04 06:09:50 +00:00
Many of them involve this prototype live-updating programming environment
for a postfix language that I might work on again one day:
```sh
$ cd linux
$ ./translate tile/*.mu
$ ./a.elf screen
```
2021-03-01 04:37:57 +00:00
2020-07-05 22:28:37 +00:00
- [How to get your text editor set up for Mu and SubX programs. ](editor.md )
2019-08-11 02:25:44 +00:00
2020-07-11 16:10:57 +00:00
- [Some tips for debugging SubX programs. ](subx_debugging.md )
2019-10-19 21:44:16 +00:00
2020-07-05 22:28:37 +00:00
- [Shared vocabulary of data types and functions shared by Mu programs. ](vocabulary.md )
Mu programs can transparently call low-level functions written in SubX.
2019-10-19 21:44:16 +00:00
2020-07-05 22:28:37 +00:00
- [A summary ](mu_instructions ) of how the Mu compiler translates instructions
to SubX. ([colorized version](http://akkartik.github.io/mu/html/mu_instructions.html))
2019-10-19 21:44:16 +00:00
2020-07-05 22:28:37 +00:00
- [Some starter exercises for learning SubX ](https://github.com/akkartik/mu/pulls )
(labelled `hello` ). Feel free to [ping me ](mailto:ak@akkartik.com ) with any questions.
2019-10-19 21:44:16 +00:00
2020-07-05 22:28:37 +00:00
- [Commandline reference for the bootstrap C++ program. ](bootstrap.md )
2020-02-01 20:14:12 +00:00
2020-07-05 22:28:37 +00:00
- The [list of x86 opcodes ](subx_opcodes ) supported in SubX: `./bootstrap
help opcodes`.
2019-10-19 22:05:06 +00:00
2020-07-05 22:28:37 +00:00
- [Some details on the unconventional organization of this project. ](http://akkartik.name/post/four-repos )
2019-10-19 21:44:16 +00:00
2020-07-05 22:28:37 +00:00
- Previous prototypes: [mu0 ](https://github.com/akkartik/mu0 ), [mu1 ](https://github.com/akkartik/mu1 ).
2020-01-02 04:49:45 +00:00
2019-07-28 00:41:52 +00:00
## Credits
Mu builds on many ideas that have come before, especially:
- [Peter Naur ](http://akkartik.name/naur.pdf ) for articulating the paramount
problem of programming: communicating a codebase to others;
- [Christopher Alexander ](http://www.amazon.com/Notes-Synthesis-Form-Harvard-Paperbacks/dp/0674627512 )
2020-10-09 00:05:56 +00:00
and [Richard Gabriel ](https://www.dreamsongs.com/Files/PatternsOfSoftware.pdf ) for
2019-07-28 00:41:52 +00:00
the intellectual tools for reasoning about the higher order design of a
codebase;
- [David Parnas ](http://www.cs.umd.edu/class/spring2003/cmsc838p/Design/criteria.pdf )
and others for highlighting the value of separating concerns and stepwise
refinement;
2020-11-07 23:12:53 +00:00
- The folklore of debugging by print and the trace facility in many Lisp
2019-07-28 00:41:52 +00:00
systems;
- Automated tests for showing the value of developing programs inside an
elaborate harness;
2020-12-21 21:23:11 +00:00
On a more tactical level, this project has made progress in a series of bursts
as I discovered the following resources. In autobiographical order, with no
claims of completeness:
2020-07-05 22:28:37 +00:00
- [“Bootstrapping a compiler from nothing” ](http://web.archive.org/web/20061108010907/http://www.rano.org/bcompiler.html ) by Edmund Grumley-Evans.
2020-12-21 21:23:11 +00:00
- [StoneKnifeForth ](https://github.com/kragen/stoneknifeforth ) by [Kragen Sitaker ](http://canonical.org/~kragen ),
including [a tiny sketch of an ELF loader ](https://github.com/kragen/stoneknifeforth/blob/master/386.c ).
2020-07-05 22:28:37 +00:00
- [“Creating tiny ELF executables” ](https://www.muppetlabs.com/~breadbox/software/tiny/teensy.html ) by Brian Raiter.
2020-12-21 21:23:11 +00:00
- [Single-page cheatsheet for the x86 ISA ](https://net.cs.uni-bonn.de/fileadmin/user_upload/plohmann/x86_opcode_structure_and_instruction_overview.pdf )
by Daniel Plohmann ([cached local copy](https://github.com/akkartik/mu/blob/master/cheatsheet.pdf))
- [Minimal Linux Live ](http://minimal.linux-bg.org ) for teaching how to create
2021-02-01 18:29:31 +00:00
a bootable disk image using the syslinux bootloader.
- [“Writing a bootloader from scratch” ](https://www.cs.bham.ac.uk/~exr/lectures/opsys/10_11/lectures/os-dev.pdf )
by Nick Blundell.
2020-12-21 21:23:11 +00:00
- Wikipedia on BIOS interfaces: [Int 10h ](https://en.wikipedia.org/wiki/INT_10H ), [Int 13h ](https://en.wikipedia.org/wiki/INT_13H ).
- [Some tips on programming bootloaders ](https://stackoverflow.com/questions/43786251/int-13h-42h-doesnt-load-anything-in-bochs/43787939#43787939 )
by Michael Petch.
2020-12-24 06:02:06 +00:00
- [xv6, the port of Unix Version 6 to x86 processors ](https://github.com/mit-pdos/xv6-public )
- Some tips on handling keyboard interrupts by [Alex Dzyoba ](https://alex.dzyoba.com/blog/os-interrupts )
and [Michael Petch ](https://stackoverflow.com/questions/37618111/keyboard-irq-within-an-x86-kernel ).
