From 30b2fcf8e6b2b127f769d339ced085ac685c3bcf Mon Sep 17 00:00:00 2001 From: Kartik Agaram Date: Fri, 9 Aug 2019 21:54:07 -0700 Subject: [PATCH] 5502 - package up into a bootable disk image Many thanks to John Davidson for Minimal Linux Live (GPLv3), from which I cribbed gen_iso. --- Readme.md | 83 +++++++++++++++++++++++++++++++----------------- clean | 1 + gen_iso | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++ syslinux.cfg | 6 ++++ 4 files changed, 150 insertions(+), 29 deletions(-) create mode 100755 gen_iso create mode 100644 syslinux.cfg diff --git a/Readme.md b/Readme.md index 40089b09..ea41f8ac 100644 --- a/Readme.md +++ b/Readme.md @@ -2,9 +2,22 @@ * Not designed to operate in large clusters providing services for millions of people. -* Designed for _you_, to run one computer. (Or a few.) +* Designed for _you_, to run one computer. (Or a few.) Running the code you + want to run, and nothing else. -Goals (in priority order): + ```sh + $ git clone https://github.com/akkartik/mu + $ cd mu + # package up a "hello world" binary and Linux kernel into mu.iso + $ ./gen_iso examples/ex6.subx + # wait a few minutes, mostly for the kernel to compile + $ qemu-system-x86_64 -m 256M -cdrom mu.iso -boot d + # print the message followed by kernel panic + ``` + +## Goals + +In priority order: * [Reward curiosity.](http://akkartik.name/about) * Easy to build, easy to run. [Minimal dependencies](https://news.ycombinator.com/item?id=16882140#16882555), @@ -21,7 +34,8 @@ Goals (in priority order): * Memory leaks over memory corruption. * Teach the computer bottom-up. -Non-goals: +## Non-goals + * Efficiency. Clear programs over fast programs. * Portability. Runs on any computer as long as it's x86. * Compatibility. The goal is to get off mainstream stacks, not to perpetuate @@ -30,49 +44,41 @@ Non-goals: For now it's a thin veneer over machine code. I'm working on memory safety before expressive syntax. -So far I have a self-hosted tool (SubX) for writing thoroughly tested x86 -machine code atop a bare Linux kernel. -Eventually you will be able to program in higher-level notations. -Eventually Mu won't need Linux or C. -Eventually the OS interfaces for screen, keyboard, file system and network -will be _dependency-injected_ so that tests can easily insert a fake screen, -keyboard, file system or network. +## What works so far -The rest of this Readme describes SubX. - -## SubX is a simple, minimalist stack for programming your computer. +You get a thin syntax called SubX for programming in (a subset of) x86 machine +code. Here's a program (`examples/ex1.subx`) that returns 42: ```sh - $ git clone https://github.com/akkartik/mu - $ cd mu - $ ./subx # print out a help message + bb/copy-to-EBX 0x2a/imm32 # 42 in hex + b8/copy-to-EAX 1/imm32/exit + cd/syscall 0x80/imm8 ``` -SubX requires a Unix-like environment with a C++ compiler (Linux or BSD or Mac -OS). Running `subx` will transparently compile it as necessary. - -[![Build Status](https://api.travis-ci.org/akkartik/mu.svg?branch=master)](https://travis-ci.org/akkartik/mu) - -You can generate native ELF binaries with it that run on a bare Linux -kernel. No other dependencies needed. +You can generate tiny zero-dependency ELF binaries with it that run on Linux. ```sh - $ ./subx translate examples/ex1.subx -o examples/ex1 + $ ./subx translate examples/ex1.subx -o examples/ex1 # on Linux or BSD or Mac $ ./examples/ex1 # only on Linux $ echo $? 42 ``` +Generating the binary requires a C++ compiler (any version). Running `subx` +will transparently invoke the compiler as necessary. + +[![Build Status](https://api.travis-ci.org/akkartik/mu.svg?branch=master)](https://travis-ci.org/akkartik/mu) + You can run the generated binaries on an interpreter/VM for better error messages. ```sh - $ ./subx run examples/ex1 # on Linux or BSD or OS X + $ ./subx run examples/ex1 # on Linux or BSD or Mac $ echo $? 42 ``` -Emulated runs generate a trace that permits [time-travel debugging](https://github.com/akkartik/mu/blob/master/browse_trace/Readme.md). +Emulated runs can generate a trace that permits [time-travel debugging](https://github.com/akkartik/mu/blob/master/browse_trace/Readme.md). ```sh $ ./subx --debug translate examples/factorial.subx -o examples/factorial @@ -85,8 +91,8 @@ Emulated runs generate a trace that permits [time-travel debugging](https://gith $ ../browse_trace/browse_trace last_run # text-mode debugger UI ``` -You can write tests for your assembly programs. The entire stack is thoroughly -covered by automated tests. SubX's tagline: tests before syntax. +You can write tests for your programs. The entire stack is thoroughly covered +by automated tests. SubX's tagline: tests before syntax. ```sh $ ./subx test @@ -129,9 +135,28 @@ Or, running in a VM on other platforms: 42 ``` +Finally, as described at the top, you can turn it into a bootable disk image +containing just your code and a Linux kernel. You can run the disk image on a +cloud server that supports custom images. [Instructions for Linode.](http://akkartik.name/post/iso-on-linode) + + ```sh + $ sudo apt install build-essential wget libelf-dev xorriso + $ ./gen_iso examples/ex6.subx + $ qemu-system-x86_64 -m 256M -cdrom mu.iso -boot d + ``` + +Some caveats for `gen_iso` which was only created 2019-08-09: + +* It can take a while, mostly to compile the Linux kernel. In my tests, + generating the ISO takes 40 minutes on a computer with 1 core and 2GB RAM. + With 4 cores and 8GB RAM it takes 6 minutes. + +* It currently works on Ubuntu 18.04 but not 19.04. This is because it uses a + kernel version that doesn't work with the default settings of gcc 8. + ## What it looks like -Here is the first example we ran above, a program that just returns 42: +Here is the above example again: ```sh bb/copy-to-EBX 0x2a/imm32 # 42 in hex diff --git a/clean b/clean index 590687a7..2d17099f 100755 --- a/clean +++ b/clean @@ -6,3 +6,4 @@ rm -rf subx.cc subx_bin* *_list rm -rf .until test $# -gt 0 && exit 0 # convenience: 'clean top-level' to leave subsidiary tools alone rm -rf enumerate/enumerate tangle/tangle tangle/*_list */*.dSYM termbox/*.[oa] +rm -rf tmp mu.iso diff --git a/gen_iso b/gen_iso new file mode 100755 index 00000000..ba08e8cb --- /dev/null +++ b/gen_iso @@ -0,0 +1,89 @@ +#!/bin/sh +# Build one or more .subx files into an ELF binary, and package it up into a +# bootable ISO image. +# +# Must be run on Linux. +# +# Dependencies: +# apt install build-essential flex bison libelf-dev libssl-dev xorriso +# +# Based on http://minimal.linux-bg.org (GPLv3) + +set -e + +if [ $# -eq 0 ] +then + echo "Usage: `basename $0` file.subx ..." + exit 1 +fi + +echo "=== constructing initramfs out of SubX binary" +./ntranslate $* +mv a.elf init +chmod +x init +rm -rf tmp/isoimage +mkdir -p tmp/isoimage/boot +echo init | cpio -R root:root -H newc -o | xz -9 --check=none > tmp/isoimage/boot/rootfs.xz + +if [ ! -d kernel ] +then + echo "=== downloading kernel" + test -f tmp/linux-4.14.12.tar.xz || wget https://kernel.org/pub/linux/kernel/v4.x/linux-4.14.12.tar.xz -P tmp + echo "=== unpacking kernel" + tar xf tmp/linux-4.14.12.tar.xz + mv linux-4.14.12 kernel +fi + +echo "=== building kernel" +( cd kernel + + make mrproper -j $NUM_JOBS + make defconfig -j $NUM_JOBS + sed -i "s/.*CONFIG_DEFAULT_HOSTNAME.*/CONFIG_DEFAULT_HOSTNAME=\"mu\"/" .config + # enable overlay support, e.g. merge ro and rw directories (3.18+). + sed -i "s/.*CONFIG_OVERLAY_FS.*/CONFIG_OVERLAY_FS=y/" .config + # enable overlayfs redirection (4.10+). + echo "CONFIG_OVERLAY_FS_REDIRECT_DIR=y" >> .config + # turn on inodes index feature (4.13+). + echo "CONFIG_OVERLAY_FS_INDEX=y" >> .config + # disable all kernel compression options + sed -i "s/.*\\(CONFIG_KERNEL_.*\\)=y/\\#\\ \\1 is not set/" .config + # enable the VESA framebuffer for graphics support + sed -i "s/.*CONFIG_FB_VESA.*/CONFIG_FB_VESA=y/" .config + # disable boot logo + sed -i "s/.*CONFIG_LOGO_LINUX_CLUT224.*/\\# CONFIG_LOGO_LINUX_CLUT224 is not set/" .config + sed -i "s/.*CONFIG_EFI_STUB.*/CONFIG_EFI_STUB=y/" .config + # request that the firmware clear the contents of RAM after reboot (4.14+) + echo "CONFIG_RESET_ATTACK_MITIGATION=y" >> .config + echo "CONFIG_APPLE_PROPERTIES=n" >> .config + + if [ "`grep "CONFIG_X86_64=y" .config`" = "CONFIG_X86_64=y" ] + then + echo "CONFIG_EFI_MIXED=y" >> .config + fi + + make bzImage -j $(grep ^processor /proc/cpuinfo | wc -l) +) +cp kernel/arch/x86/boot/bzImage tmp/isoimage/boot/kernel.xz + +echo "=== downloading syslinux" +test -f tmp/syslinux-6.03.tar.xz || wget https://kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.tar.xz -P tmp +echo "=== unpacking syslinux" +tar xf tmp/syslinux-*.tar.xz -C tmp + +mkdir -p tmp/isoimage/boot/syslinux +cp syslinux.cfg \ + tmp/syslinux-*/bios/core/isolinux.bin \ + tmp/syslinux-*/bios/com32/elflink/ldlinux/ldlinux.c32 \ + tmp/isoimage/boot/syslinux + +echo "=== generating ISO" +# 'hybrid' ISO can also be used on non-optical media such as a disk or USB stick +xorriso -as mkisofs \ + -isohybrid-mbr tmp/syslinux-*/bios/mbr/isohdpfx.bin \ + -c boot/syslinux/boot.cat \ + -b boot/syslinux/isolinux.bin \ + -no-emul-boot \ + -boot-load-size 4 \ + -boot-info-table \ + tmp/isoimage -o mu.iso diff --git a/syslinux.cfg b/syslinux.cfg new file mode 100644 index 00000000..ae2eee79 --- /dev/null +++ b/syslinux.cfg @@ -0,0 +1,6 @@ +DEFAULT mu + +LABEL mu + LINUX /boot/kernel.xz + APPEND vga=nomodeset + INITRD /boot/rootfs.xz