build/spec.md

forgebuild v0.5

forgebuild is a program to check for updates on remote repositories and trigger some tasks accordingly. There are different implementations available, and this document serves as a reference for the high-level interface these different programs should implement.

If you are looking for a program you may use as an end user, check out forge/build.rs (Rust) or forge/build.sh (bash) instead.

On a high-level, forgebuild is a lightweight and KISS system to start tasks when a remote repository was updated and/or when those tasks haven't run on the current machine yet. In most cases, forgebuild trigger tasks when a remote Git/Mercurial repository was updated, eg. to run a suite of tests or build your static website. But it can serve many other purposes, including setting up your dotfiles on a new machine. Your imagination is the limit!

This is version 0.5 of this specification, published on September 22 2020.

Overview

Any forgehook implementation MUST have the following features:

• check for updates on remote repositories, and start corresponding tasks if there was any update (forgebuild [tasks])
• run one or more tasks, regardless of remote updates (̀forgebuild -f [tasks])
• configure settings per task or per host
• run one-off tasks (sourceless tasks)

forgebuild supports different source backends. Here's an overview of backends that forgehook implementations MUST support, and those that they MAY additionally support:

• (REQUIRED) sourceless: no remote source, task is executed only once per host
• (REQUIRED) git
• mercurial (hg)
• pijul

Vocabulary

• basedir: folder where forgebuild will look for tasks
• task: a program which may be run by forgebuild
• task parameter: a file in the basedir, which has the name of the task followed by a dot, and the name of the parameter (t.s for a parameter s on a task t)
• task/host setting: a file in the configuration directory

Summary

If you can't afford to read the whole document, here's what you need to know. A task is an executable file, which may have additional parameters. For a given task t, there may be several parameters in the basedir (only t itself is mandatory):

• t: the executable script
• t.source: the source URL for the repo to track
• t.dvcs: the source backend to clone the repository, can be either git or mercurial (default: git)
• t.checkout: a specific branch/commit to checkout
• t.hosts: line-separated list of hosts on which this task should run (opt-in, see below for opt-out)

There may also be settings folders for consumption by tasks. This folder is either the config folder in the basedir, or a folder named after the current host ($HOSTNAME). If the settings folder contains a file named t.skip (skip setting), then the task will be skipped on this host (opt-out).

Command-Line Interface (CLI)

Base directory (basedir)

forgebuild MUST support passing an arbitrary basedir (tasks directory) through the -b|--basedir flag. If no such directory is provided, forgebuild MUST use $HOME/.forgebuild/. In both cases, forgebuild MUST support relative paths and symlinks.

Running specific tasks

forgebuild MAY receive an arbitrary number of task names to trigger (eg. forgebuild run_tests deploy_site). In this case, other tasks MUST NOT be triggered. If no such task names are passed as arguments, then all tasks MUST be triggered.

Forced run

When the -f|--force flag is passed, forgebuild MUST run the requested tasks, whether the task source has received updates or not. However, if the task doesn't have a source, it MUST be skipped unless its name is specified.

For example, forgebuild -f will run all tasks who have a source, ignoring sourceless tasks. forgebuild -f foo bar will run tasks foo and bar, even if they are sourceless tasks who have already run.

Tasks

Discovery

All executable files within the basedir, as well as symlinks within the basedir pointing to an executable file, MUST be considered tasks, except for files containing a dot (hidden files and task.when).

Ordering

Tasks SHOULD be run sequentially, in alphanumeric order. Additionally, a forgebuild implementation MAY implement an opt-in mechanism to run tasks in parallel, but MUST respect alphanumeric order for starting those parallel tasks.

TODO: This could be standardized as a task dependency system.

Settings

Tasks MUST receive an environment variable $FORGEBUILDCONF pointing to a folder where they may find arbitrary files, usually containing settings/parameters. If the basedir contains a folder matching the current $HOSTNAME, then this folder MUST be considered the configuration folder. Otherwise, the config/ folder in the basedir MUST be used, even if it does not exist.

In this document, a task setting s for a task t refers to the presence/content of the file t.s in the basedir.

Sources

Defining sources

Tasks MAY have associated parameters defining a remote source repository to track. These parameters are source, dvcs and checkout:

• source: the URL of the source repository
• dvcs: the version control system (git|mercurial) to clone the repository
• checkout: which commit/branch to checkout after cloning

Submodules of the source repository MUST be cloned when cloning said repository.

TODO: PGP security here? task.key contains the commit where a guix authorisation file is initialized

Managing submodules

Tasks MAY have a subupdates setting. When they do, submodule updates will trigger the task. Otherwise, only source repository updates will trigger the task.

TODO: The future version will more look like this:

Tasks **MAY** have a `subupdates` setting controlling how submodule updates affect the task. It can have the following values:

- file does not exist: submodules are not updated automatically
- `update`: submodules are updated, but tasks are only run when the main repositories is updated
- `trigger`: submodule updates trigger tasks

TODO: actually, submodule update always triggers task when subupdates is defined. If the file is not empty, it contains one entry per lines of submodules to autoupdate. Others will not be updated.
TODO: does auto-updating subs break when main repo is updated? what could be a strategy here? when the main repo is updated, check if it tries to update submodules and if so revert back the submodule to its upstream version so we can merge the update, then update submodules again according to subupdates policy?