Specification and translation files for forgebuild
Go to file
southerntofu 08c5de5398 Mark broken link TODO so zola builds 2022-02-23 20:33:48 +01:00
i18n Add translation string for info log 2020-11-29 15:57:04 +01:00
tests Add --inbox and --inbox-folder flags 2022-01-12 19:18:57 +01:00
.gitignore Initial commit 2020-07-11 11:46:10 -04:00
LICENSE Initial commit 2020-07-11 11:46:10 -04:00
README.md Mark broken link TODO so zola builds 2022-02-23 20:33:48 +01:00
spec.md Add some TODO in the spec 2021-01-04 16:23:16 +01:00
test.sh Can run tests from another folder 2022-01-05 16:40:08 +01:00

README.md

+++ extra.pipeline = "forgebuild" template = "project.html" +++

forgebuild

NOTE: This is experimental software. Please don't use just yet.

This repository contains the specification, translation strings, and tests for forgebuild. Looking for setup instructions? Head over here.

forgebuild is a command-line program to check for updates on remote repositories (software forges) and potentially trigger some tasks. When those tasks are run, what they do and how they do it is up to you! forgebuild is an essential component to build a CI/CD system, and can be used either as a standalone tool or as part of the forgesuite CI/CD.

forgebuild uses a pull-based update mechanism where tasks are defined outside of the source repository. Unlike push-based CI/CD (Drone/Gitlab CI/Jenkins), you don't need maintainer permission to perform new actions on a software repository by modifying eg. .gitlab-ci.yml in the source tree. Unlike some other pull-based systems, forgebuild is not just a glorified CRON job, as it has built-in support for high-level features so you don't have to reinvent the wheel.

Have you ever struggled to:

  • trigger a task if its associated source received updates? forgebuild TASK (git/mercurial support)
  • trigger many tasks associated to a single repository? forgebuild SOURCE
  • have fine control over submodule updates and task triggering conditions? see submodules and conditional run
  • run tasks on different hosts with different settings using a single source of truth? see per-host config
  • use your favorite programming language to write tasks? see task API
  • ensure PGP signatures across your software supply chain? (coming soon with guix git authenticate)

If the answer is yes to any of these questions, you may enjoy using forgebuild. However, please be aware forgebuild is alpha-quality software. Although it's evolving very quickly and has a growing test suite, it may produce unexpected results and kill the policeman in your head. Please don't rely on it for critical infrastructure until you have audited it: patches to the specification and to the test suite are welcome.

Features

forgebuild aims to be a one-size-fits-all tool for software automation. It empowers newcomers to setup complex workflows: no grand mathematical theory of implementation details, but a strong focus on user experience (specification-driven development). It doesn't get in the way of power users: use your favorite DVCS (git/mercurial) and your favorite programming language.

If the tool isn't easy and obvious to use, or it gets in your way, this is a bug! There are chances you're doing some dubious/crazy stuff, but there's also chances you have good reasons for that and it should be a supported workflow. forgebuild is developed as a specification, and there are different implementations available. As a single maintainer (for the time being), i cannot promise to support every workflow in the reference implementations, but there is space for more test suites for your own usecases, and for your own implementations supporting them!

So far, this is what forgebuild can do for you:

  • Setup your environment on a new machine (sourceless tasks)
  • Track git/mercurial remotes and run some scripts when updates are performed
    • Build, test and deploy all your software projects (websites/infrastructure/programs) in a consistent manner
    • Run your own tasks on repositories maintained by others: don't wait for the maintainer's permission to setup tests and benchmarks
    • Checkout a specific branch/channel for updates
    • Run your own custom script to decide whether a task should run (conditional run)
  • Per-repository submodule update settings
    • Automatic submodule updates for the whole repository
    • Opt-in list of submodules to auto-update (coming soon)
  • Trigger task updates by..
    • Task name
    • Source URL to trigger all tasks tracking a remote repository
    • Submodule URL to trigger all tasks auto-updating a given submodule (coming soon)
  • Setup a very flexible automation system
    • Program output can be altered through localization (i18n translation system)
    • The programming language for your task scripts is your own choice
    • Each task can have very different configuration
  • Integrate with your favorite forging ecosystem
    • Easy integration with forgehook for real-time updates
    • Pull updates from mail patches (coming soon)
    • Support multiple update types (eg pull request, issue comment, push) through standard semantics (coming soon with forgefed)
  • Secure your software supply chain with PGP signatures (coming soon only for git with guix git authenticate)

Setup

Would you like to give forgebuild a try? There is currently only two implementations, with more or less the same features:

More precise setup instructions are contained in those repositories.

Currently, forgebuild is not packaged for any distribution.

Example usecases

In this section, we will introduce common usecases to use forgebuild:

  • dotfiles setup
  • website deployment (and other build artifacts)
  • testing/benchmarking
  • software derivations

Dotfiles setup

You just setup a new machine, or received a new user account on a server, and would like to configure your session? Versioning and publishing your dotfiles is common practice with software like GNU stow. With forgebuild, you can achieve the same kind of setup with more granularity.

Let's assume you have your session configuration stored as a forgebuild basedir on a remote repository:

user@portable:~$ git clone https://tildegit.org/foo/dotfiles-setup ~/.forgebuild
user@portable:~$ forgebuild
[forgebuild] bashrc: PROCESS
[forgebuild] bashrc: RUN
[forgebuild] gitconfig: PROCESS
[forgebuild] gitconfig: RUN
user@portable:~$ # Your bash and git are now configured according to your taste

Now, you have this repository cloned on several machines, but on server1 you need to edit some files with people who use only 8-space tabulations. vim is already configured to your liking on the current machine. Let's create a task to setup vim appropriately on all your machines:

user@portable:~$ # Copy the current vim configuration as default for the task
user@portable:~$ mkdir ~/.forgebuild/config && cp ~/.vimrc ~/.forgebuild/config/.vimrc
user@portable:~$ # Make a special config just for server1
user@portable:~$ mkdir ~/.forgebuild/server1 && cp default_vimrc ~/.forgebuild/server1/.vimrc
user@portable:~$ echo -e "shiftwidth=8\nexpandtab" >> ~/.forgebuild/server1/.vimrc
user@portable:~$ # Create a task to deploy the configuration
user@portable:~$ cat << \EOF > ~/.forgebuild/vimrc
#! /usr/bin/env bash
cp $FORGEBUILDCONF/.vimrc ~/.vimrc
EOF
user@portable:~$ forgebuild
[forgebuild] Using config: /home/user/.forgebuild/config
[forgebuild] vimrc: PROCESS
[forgebuild] vimrc: RUN
user@portable:~$ # vim is still running your great config locally
user@portable:~$ cd ~/.forgebuild && git add vimrc config/ server1/
user@portable:~/.forgebuild$ git commit -m "Added vim configuration"
...
user@portable:~/.forgebuild$ git push && cd ~
...
user@portable:~$ ssh user@server1 "cd ~/.forgebuild;git pull;forgebuild"  
[forgebuild] Using config: /home/user/.forgebuild/server1
[forgebuild] vimrc: PROCESS
[forgebuild] vimrc: RUN
user@portable:~$ # server1 vim is now configured with 8-space tabulations

But maybe you're also working on server3 but there, everything has a great configuration and you would like to configure only git there, not bash or vim. Let's see what we can do:

user@portable:~$ mkdir ~/.forgebuild/server3
user@portable:~$ touch ~/.forgebuild/server3/{bash,vim}.ignore
user@portable:~$ cd ~/.forgebuild && git add server3
user@portable:~/.forgebuild$ git commit -m "Configure server3" && git push; cd ~
...
user@portable:~$ ssh user@server3 "git clone https://tildegit.org/foo/dotfiles-setup .forgebuild; forgebuild" 
[forgebuild] Using config: /home/user/.forgebuild/server3
[forgebuild] git: PROCESS
[forgebuild] git: RUN
user@portable:~$ # git is now configured on server3, but bash/vim previous config remains!

TODO: actually the last example is almost a lie according to current implementations. That is because FORGEBUILDDIR points EITHER to config/ or to a host-specific folder. This example only works because git task doesn't use stuff from config/. Maybe w should have an overlayFS for config? or have FORGEBUILDCONF be an executable which deals with that?

Website deployment (build artifacts)

If you like to keep your website in a version control, you may want to deploy updates automatically (continuous delivery). While forgebuild does not provide signaling from remote forges, you can either run forgebuild manually or via CRON to publish your website. What is true of a website is true of any other build artifacts. You can use forgebuild to build packages, documentation, translations, and many other things.

For example, with zola, you can do:

~$ cd ~/.forgebuild
~/.forgebuild$ echo "https://tildegit.org/forge/site" > site.source
~/.forgebuild$ cat << \EOF > site # Our task script
> #! /usr/bin/env bash
> zola build -o $HOME/sites/forgesite
> EOF # End of task script
~/.forgebuild$ touch site.subupdates # Enable submodule updates for the site 
~/.forgebuild$ chmod +x ~/.forgebuild/site # Exec permission for task
~/.forgebuild$ forgebuild site
[forgebuild] site: PROCESS
[forgebuild] site: CLONE https://tildegit.org/forge/site
[forgebuild] site: RUN
~/.forgebuild$ cat site.log
Building site...
-> Creating 10 pages (4 orphan), 1 sections, and processing 0 images
Done in 33ms.

Now, you want to deploy your website on two different servers for greater censorship resistance. These two servers are reachable over two different URLs, but your website makes heavy use of absolute links. You need to tell your static site generator to build your website for a specific baseURL on every host. When the host is not one of your servers, we assume a fallback http://localhost/ URL.

~/.forgebuild$ mkdir server1 # Create config folder for first server
~/.forgebuild$ echo "https://server1.example/" > server1/baseurl
~/.forgebuild$ mkdir server2
~/.forgebuild$ echo "https://server2.example/" > server2/baseurl
~/.forgebuild$ mkdir config # fallback configuration
~/.forgebuild$ echo "http://localhost/" > config/baseurl
~/.forgebuild$ cat << \EOF > site # Our new task script
> #! /usr/bin/env bash
> zola build -u "$(cat $FORGEBUILDCONF/baseurl)" -o $HOME/sites/forgesite
> EOF # End of task script
~/.forgebuild$ # Test run on our client
~/.forgebuild$ forgebuild -f site
[forgebuild] site: PROCESS
[forgebuild] site: RUN
~/.forgebuild$ grep "http:&#x2F;&#x2F;localhost" $HOME/sites/forgesite/index.html
~/.forgebuild$ [[ $? != 0 ]] && echo "Failed to find localhost baseURL!"
~/.forgebuild$ # The correct baseURL was loaded, let's go ahead
~/.forgebuild$ # We should probably version and push the basedir to a remote
~/.forgebuild$ # But we'll make a simple copy across machines for now
~/.forgebuild$ scp -qr ~/.forgebuild user@server1:
~/.forgebuild$ scp -qr ~/.forgebuild user@server2:
~/.forgebuild$ # Now run forgebuild on the remote servers
~/.forgebuild$ ssh user@server1 forgebuild -f site
[forgebuild] site: PROCESS
[forgebuild] site: RUN
~/.forgebuild$ # Ensure the site was built with the correct baseurl
~/.forgebuild$ checkbase() { curl https://$1/index.html | grep "https:&#x2F;&#x2F;$1" && echo "SUCCESS" }
~/.forgebuild$ checkbase server1.example
SUCCESS
~/.forgebuild$ ssh user@server2 forgebuild -f site
[forgebuild] site: PROCESS
[forgebuild] site: RUN
~/.forgebuild$ checkbase server2.example
SUCCESS
~/.forgebuild$ # All is good for now!

Testing/benchmarking

You are contributing to a free-software project. You start to develop a branch with a set of patches to fix some bugs, or improve performance. You would like other people to be able to run your modified test suite or the benchmarks you just added. However, the maintainers to the project are either unexperienced or relunctant to add a continuous integration pipeline to the project as a whole, or simply can't deal with the burden of reviewing patches to the CI every time someone wants to do an experiment.

With forgebuild, you can setup your own, unprivileged testing/benchmarking infrastructure to run your personal experiments.Better than that, you can publish your forgebuild basedir so others can reproduce your setup and you can compare results. For example, maybe you have forked a static-site generator to implement some new features, and would like to know whether your work negatively impacts performance by running the program's full test suite many times. You can do:

~$ cd ~/.forgebuild
~/.forgebuild$ echo "https://github.com/getzola/zola" > upstream.source
~/.forgebuild$ cat << \EOF > upstream # Our task script
> #! /usr/bin/env bash
> TIMESTAMP="$(date +%s)"
> export TIMEFORMAT="%R"
> # $1 is the task name so we can use the same script with many tasks
> TESTS=$HOME/testresults/$1
> cargo build || exit 1 # Abort the task if build fails
> time cargo bench 2> $TESTS/$TIMESTAMP # Save bench time
> ln -sf $TESTS/{$TIMESTAMP,latest} # Symlink latest results
> EOF # End of task script
~/.forgebuild$ chmod +x upstream # Exec permission for task
~/.forgebuild$ # Clone the benchmarking task for your fork
~/.forgebuild$ echo "https://0xacab.org/foo/zola" > fork.source
~/.forgebuild$ echo "mybranch" > fork.checkout # Checkout fork's mybranch
~/.forgebuild$ ln -s {upstream,fork}
~/.forgebuild$ forgebuild upstream fork
[forgebuild] fork: PROCESS
[forgebuild] fork: CLONE
[forgebuild] fork: RUN
[forgebuild] upstream: PROCESS
[forgebuild] upstream: CLONE
[forgebuild] upstream: RUN
~/.forgebuild$ result() { echo -n "$1: "; cat $HOME/testresults/$1/latest } # Helper function
~/.forgebuild$ compares() { echo "$(result upstream) $(result fork)" }
~/.forgebuild$ compares
upstream: 123.234 fork: 147.211
~/.forgebuild$ # Did we introduce a regression? Try again
~/.forgebuild$ forgebuild -f fork
[forgebuild] fork: PROCESS
[forgebuild] fork: RUN
~/.forgebuild$ compares
upstream: 123.234 fork: 122.231
~/.forgebuild$ # Looks ok, maybe the machine was just busy?
~/.forgebuild$ # Maybe run a few more times just to make sure
~/.forgebuild$ R=0; while [ $R -lt 3 ]; do
> ((R++)); forgebuild -f upstream fork > /dev/null; compares
> done
upstream: 122.213 fork: 123.487
upstream: 124.722 fork: 123.902
upstream: 122.819 fork: 122.984
~/.forgebuild$ # Everything looks ok performance wise!

Now that the benchmarks are working on your machine, you can simply publish your basedir so other people may run the same benchmarks on their own system. Is somebody else working on another patch you'd like to bench? Simply declare their source repository for a new task, symlink this new task to the upstream task, and run benchmark for their patch too!

~/.forgebuild$ echo "https://github.com/forker/zola-fork" > otherfork.source
~/.forgebuild$ ln -s upstream otherfork
~/.forgebuild$ compares() { echo "$(result upstream) $(result fork) $(result otherfork)" }
~/.forgebuild$ forgebuild otherfork
[forgebuild] otherfork: PROCESS 
[forgebuild] otherfork: CLONE https://github.com/forker/zola-fork
[forgebuild] otherfork: RUN
~/.forgebuild$ compares
upstream: 122.819 fork: 122.984 otherfork: 83.281
~/.forgebuild$ # Wow, looks like this contributor did some incredible optimization!
~/.forgebuild$ R=0; while [ $R -lt 3 ]; do
> ((R++)); forgebuild -f upstream fork otherfork > /dev/null; compares
> done
upstream: 121.284 fork: 122.745 otherfork: 83.103
upstream: 123.290 fork: 122.951 otherfork: 83.158
upstream: 122.348 fork: 122.300 otherfork: 82.021
~/.forgebuild$ # Their fork is really faster!

Would you like to fork forker's patch to explore what they did, and try to improve on it? You don't necessarily need Internet access, as long as you already have a copy of their source repository. Simply use a local git repo on your filesystem as source for your new tests.

~/.forgebuild$ # I don't have Internet access anymore, but i'd like to try something
~/.forgebuild$ cp -r .otherfork/ $HOME/projects/otherfork # Copy otherfork source
~/.forgebuild$ rm -rf .fork/ # Remove the source of your current fork
~/.forgebuild$ echo "$HOME/projects/otherfork" > fork.source # Track your local copy
~/.forgebuild$ # MAKE SOME CHANGES IN $HOME/projects/otherfork and commit them
...
~/.forgebuild$ forgebuild fork
[forgebuild] fork: PROCESS
[forgebuild] fork: CLONE
[forgebuild] fork: RUN
~/.forgebuild$ compares
upstream: 122.348 fork: 79.811 otherfork: 82.021
~/.forgebuild$ # Looks like we made it even faster, let's make sure of it
~/.forgebuild$ R=0; while [ $R -lt 3 ]; do # Let's not bench upstream again
> ((R++)); forgebuild -f fork otherfork > /dev/null; compares
> done
upstream: 122.348 fork: 80.121 otherfork: 81.923
upstream: 122.348 fork: 79.758 otherfork: 82.987
upstream: 122.348 fork: 79.991 otherfork: 81.228
~/.forgebuild$ # We did make it even faster!
~/.forgebuild$ # We can publish the results when we have Internet access

Software derivations

forgebuild can be used to release build artifacts. As such, it can be used to produce packages for your software, or live images for your system. Is there a patch you would really want to have upstream, but you and the maintainers have a disagreement over it? Maybe forgebuild can help you.

You can use forgebuild to track upstream commits, and rebase your fork on them automatically, then either publish the build artifacts to your own repository, or perform actions in case of conflicts emerging (maybe send yourself an email).

TODO: write example derivation with package publishing and new conflict

Specification

forgebuild is based on a specification which can be found in the spec.md document. This document is versioned, but is currently outdated. At the moment, forgebuild implementation is more and more guided by integration tests. Test suites contained in the tests/ folder use forgebuild as an end-user would, and check for expected output and side-effects produced by task runs.

In the future, forgebuild will try to reunite the specification with the test suites, because it makes little sense to have tests you don't understand or a specification you can't test. As doctests have proven successful for producing reliable software over time, we will explore how spectests may help produce correct implementations.

Translations

The i18n/ folder contains translation files that can be used across forgebuild implementations. The reference translations (currently french and english only) are stored as JSON files using 2-letter language codes (en.json/fr.json) containing a mapping of translation keys to their localized version:

{
  "hello": "bonjour",
  ... 
}

Translation files used by forgebuild are configured with the LANG environment variable. In most cases, the first two letters of this variable will be used to find corresponding translations (fr_FR.UTF8 -> fr). However, LANG can have two special values:

  • NONE will output the translation key, instead of the actual translation (used for output tests)
  • JSONCONTEXT will output the current context for translations (more on that below)

The translation context is a mapping of keys to values. It contains variables that will be replaced before displaying the translation. It can contain:

  • $i18n_basedir: the forgebuild basedir where the task is located
  • $i18n_config: the folder containing task settings
  • $i18n_task: the task being processed
  • $i18n_source: the source URL of the task being processed

Let's consider the translation string "task_run": "Exécution de $i18n_task". For a task called foo without a source, it would output:

frenchie@portable:~$ forgebuild foo
(...)
Exécution de foo
(...)
frenchie@portable:~$ LANG=NONE forgebuild foo
(...)
task_run
(...)
frenchie@portable:~$ LANG=JSONCONTEXT forgebuild foo
(...)
{"$i18n_basedir":"/home/frenchie/.forgebuild","$i18n_task":"foo"}
(...)

JSON was chosen for simplicity and portability in the early stages of the project, but is not very well-suited for translations. This choice may be reconsidered in the future, or we may develop some scripts to export those translation strings to different formats such as GNU gettext.

TODO: maybe LANG=JSONCONTEXT makes implementation and testing less obvious. It also means the testing suite requires jq for JSON processing. Maybe trying with LANG pointing to an actual JSON file just outputing expected values could do the trick? Also it must make the Rust implementation (and maybe others) more bloated? (should make measurements)

Tests

In the tests/ folder are the implementation tests for forgebuild. In order to run the tests, you need:

  • a bash shell
  • the bats testing framework, packaged on many distributions (TODO: a copy is included here in the repository)
  • GNU grep, for ensuring forgebuild output is correct
  • configured authorship for git and mercurial on your current session (TODO: explain more, or automatically configure it?)

You can either call individual tests directly with bats, or you can use the test.sh wrapper:

frenchie@portable:~$ bats tests/*.bats # IS THE SAME AS
frenchie@portable:~$ ./test.sh

By default, the test suite will be run against forgebuild in the PATH environment variable. If you would like to test another implementation, you can either use the FORGEBUILD environment variable, or pass it as first argument to test.sh:

frenchie@portable:~$ FORGEBUILD=~/forgebuild.rb bats tests/ # IS THE SAME AS
frenchie@portable:~$ FORGEBUILD=~/forgebuild.rb ./test.sh # IS THE SAME AS
frenchie@portable:~$ ./test.sh ~/forgebuild.rb

Example run:

frenchie@portable:~$ git clone https://tildegit.org/forge/build
frenchie@portable:~$ build/test.sh
 ✓ repository is cloned
 ✓ first run triggers task
 ✓ no update does not trigger task
 ✓ forced run triggers task
 ✓ update triggers task
 ✓ host-based settings are loaded
 ✓ unknown host uses default settings
 ✓ submodule is cloned
 ✓ first submodule run triggers task
 ✓ submodule update does not trigger task by default

10 tests, 0 failures

TODO: we should provide nix/guix profiles to setup development environment (with bash, bats and jq)

Implement your own

forgebuild is not complicated software, although powerful it may be. It should be very straightforward to make your own implementation, and to ensure compatibility with existing implementations. You should consider to implement your own if you want:

  • to run forgebuild on a system where neither bash nor rust can operate
  • to understand forgebuild internals, or if you're curious about continuous integration/delivery in general
  • to learn a new programming language, and are looking for a simple/useful project to implement
  • to try out test-driven development on a small-ish project before you dive into it entirely
  • to contribute to the forgesuite ecosystem for user-friendly, anarchist software development

Implementing your own forgebuild isn't hard, and you don't have to implement the whole specification if you don't need to. In the process of writing your own, you will be guided by the test suite. The test suite is divided in three sections:

  • tests/0*.bats: basic implementation (CLI flags, output and translations)
  • tests/1*.bats: support for sourceless tasks and different DVCS (git/mercurial)
  • tests/2*.bats: nice-to-have (host-based config, task environment, logging)

You should try to make tests pass in an alphanumeric order. In this order, you should:

  • tests/01-cli-flags.bats
    • implement dummy translations, where a requested key translates to itself (trans(foo) -> foo)
    • support ERROR, INFO and DEBUG logging levels according to the LOG environment variable
    • detect tasks (non-hidden executable files) from a basedir passed as -b|--basedir, or from ~/.forgebuild/
    • the basedir passed as argument may be a relative path to a folder
    • the basedir passed as argument may or may not have a trailing slash
    • loads tasks provided as arguments (forgebuild task1 task2) in the basedir, or all tasks from the basedir when no task is passed as argument ; error when a task requested as argument does not exist in the basedir
    • display correct output for missing basedir, unknown task, unknown argument, simple task run, no tasks requested, no tasks exist
  • tests/02-translations.bats
    • load translations for LANG environment variable
      • if LANG is a file, load it as JSON translation strings
      • if LANG is NONE, perform dummy output as previously
      • otherwise, load LANG.json from ~/.local/share/forgebuild/i18n/, /usr/local/share/forgebuild/i18n/, or /usr/share/forgebuild/i18n/
    • setup translation context (variable substitution) for basedir ($i18n_basedir), config folder ($i18n_config), task name ($i18n_task), task source ($i18n_source), missing task ($i18n_task), and unknown argument ($i18n_arg)
    • translate output according to the loaded translation strings, performing variable substitution (context expansion) after translation
  • tests/10-sourceless.bats
    • a task without a source (TASK.source) only runs once
    • a sourceless task may run again if it failed previously
    • running sourceless tasks with -f|--force flag does not trigger them after first successful run
    • sourceless tasks are run from the current basedir
  • tests/11-git.bats
    • without a DVCS configured (TASK.dvcs), a task with a source (TASK.source) will clone a remote git repository and run
    • with DVCS configured as git, a task with a source will clone a remote git repository and run
    • when no update happened on the remote, the task does not run again
    • when no update happened, but the previous task run failed, the task is run again
    • when an update happened, the local repository will pull updates and the task runs
    • when the -f|--force flag is passed, the task runs whether updates were performed or not
    • a task with a source is run from its own local copy of the repository
    • passing a source URL as argument to forgebuild triggers all corresponding tasks
    • a source URL may be mixed with task names within arguments, but tasks are not run twice
    • TODO: git submodules
  • tests/12-mercurial.bats
    • same as before, but only when DVCS (TASK.dvcs) is configured as mercurial or hg
  • tests/20-host-config.bats
    • when a task's allowlist is enabled (TASK.hosts), the task only runs when the current hostname is contained on a line in this file, and whitespace is stripped
    • when a folder exists in the basedir matching the hostname (BASEDIR/HOSTNAME), it is passed to the task as FORGEBUILDCONF environment variable; otherwise, BASEDIR/config is passed instead
    • when FORGEBUILDCONF/TASK.ignore exists, the task is not run
  • tests/21-task-env.bats
    • task receives the running forgebuild as FORGEBUILD environment variable
    • task receives the current basedir as FORGEBUILDDIR environment variable
    • task receives the current language (or NONE) as LANG environment variable
    • task receives the current loglevel as LOG environment variable
  • tests/22-conditional-run.bats
    • when a task has a conditional run file (TASK.when) which is a simple file, forgebuild's STDIN will be parsed for forgefed activity type matching the contents of this file (TODO: not implemented yet)
    • when the conditional run file is an executable file, it will be run and receive forgebuild's STDIN to decide whether or not to trigger the task
  • tests/23-logging.bats
    • TODO: decide about logging policy
  • tests/24-symlinks.bats
    • a basedir may be a symlink to another folder
    • a task may be a symlink to another program
    • a task source may be a symlink to another task's source, or to any file containing a source
    • a task's cloned source may be a symlink to another folder (produced by an external program or manual intervention)
    • a host configuration folder may be a symlink to another folder
    • a task conditional run file (TASK.when) may be a symlink to another file/program
  • tests/99-security.bats
    • check that weird strings don't do remote code execution (RCE) or allow to exec files outside the basedir

TODO: How to emulate root for /usr/local/ or ~/.forgebuild/ tests? chroot requires privileges, but maybe we can do it with some some of overlayfs?

Inspirations/Alternatives

If you know of any software similar to forgebuild, please submit a patch to mention it here!

forgebuild was inspired by packaging tools and their utilities to keep track of remote repositories. Notably, it was inspired by Debian's TODO, nix and GNU/Guix. While not aiming for the same level of reproducibility and correctness as nix/guix, forgebuild borrows their approach to rolling-release to implement a user-friendly automation workflow integration with your own tools.

Philosophy

forgebuild was born out of personal needs. It was developed following a philosophy of least-authority and maximum empowerment according to anarchist principles of inseparable freedom & equality.

Software is increasingly used as a means of social control. While developers may be less victim to their tools, we are still often trapped in closed ecosystems or forced to run software in "the cloud", which is just someone's computer we don't control. Beyond the consequences on a personal level, the computing industry is increasingly destroying the environment and enslaving millions of people (in mines and factories) to produce hardware, and negatively impacting the lives of billions of others through surveillance, censorship and dehumanized institutional processes, where an algorithm will decide whether you get social benefits or if you may get bailed out of jail.

This situation is, at least partly, due to the concentration of power in the hands of a tiny minority of wealthy Silicon Valley startup-nation fanboys whose capitalist dreams of exploitation are our worst nightmares. Software developed by such persons will never be able to empower us, because we have wildly-differing interests and concerns:

  • creepy men develop software to spy on girls from their campus (TODO: facebook story) or their own lovers (TODO: spyware link on tech patriarcal control)
  • european/israelian/north-american corporations develop mass-surveillance tools that colonize and oppress people across the globe (TODO: stratfor, hackingteam, amesys..)
  • authoritarian hackers work for their own local government to oppress their own neighbors (great firewall of china, NSA, DGSI) through facial recognition or anti-fraud systems to detect smaller frauds/mistakes but completely ignoring large-scale tax evasion and social fraud by the powerful

forgebuild is anarchist software. As such, it aims for freedom and equality, never one without the other, and attempts to prevent any form of domination or exploitation. We aim for feminist, anticolonial, accessible software development for the masses. Tools are never neutral: we have chosen our side of the struggle.

On the technical side, forgebuild relies on accessible and proven technology. The user interface is a simple set command-line flags (which you can easily build other user interfaces upon) and the filesystem is our configuration database. Using a simple folder, the configuration can be inspected and edited by anyone without special tools. Leveraging filesystem symbolic links allows us to develop a simple-yet-powerful aliasing system without having to develop a complex CLI interface or a custom scripting language.

License

GPLv3?