||2 years ago|
|bin||2 years ago|
|databases||2 years ago|
|docs||2 years ago|
|endpoints||2 years ago|
|public||2 years ago|
|triggers||2 years ago|
|README.md||2 years ago|
|setup.sh||2 years ago|
Simple and interoperable CI/CD system
forgehook is a collection of scripts following a simple interface to build a full Continuous Integration/Delivery platform. forgehook provides first-class multi-user experience for tilde/pubnix servers, and easy integration with your custom tooling.
These scripts are not intended to be used for generic webhooks, but for those produced indicating changes on a software repository. If you are looking for a more generic solution, please take a look at webhook or webhookd instead.
Note: Some specific parts of forgehook do not yet correspond to what you will find here. That's because I'm taking some time to step back and think about what's implemented so far and how it should end up, before i deepdive into the code again. The inconsistencies are marked with TODO notes.
As of April 28, the reference implementation for
forgehook-db (databases/unix) works fine, so does
forgehook-notify and the default
forgehook-trigger (triggers/git-build). No endpoint is implemented yet.
Introduction to webhooks
Webhooks are simple web pings performed by a client to inform a server that something happened (push model). In the context of a forge (such as Gitea), a webhook additionally contains information as JSON payload about what changed on the repository, as well as an HTTP signature in order to authenticate the client who sent the webhook via a shared secret.
Note: If you are not running a web-based forge such as those mentioned above, but run your own git server, you do not need webhooks to build a CI/CD system. All you need are server-side git hooks as explained in the docs.
In this section, you will find:
- an overview of the core concepts of forgehook
- a short intro to the
forgehookcommand-line interface (CLI)
- a setup guide
- a configuration guide
forgehook is a collection of scripts to let users manage their subscriptions to remote repositories (currently git only) and trigger something when a legitimate updated notification is received. Typically, forgehook is intended to be run alongside git-build.sh to automatically trigger build tasks when an update is perfomed on the repository.
Simplicity and extensibility are core concerns of forgehook, focusing on defining standard interfaces between different components so that you can reimplement each part of it to better suit your needs. Sharing of your tricks is highly encouraged! And remember, if the tool gets in your way, it's a bug so please report it.
The three components of forgehook are:
- endpoints which receive and validate webhooks of dubious authenticity (through a shared secret) (TODO: no endpoint is implemented yet!)
- databases which retrieve/store subscriptions and secrets
- triggers which does stuff as a subscribed user in case of update
Although there can be many endpoints, there can only ever be one database and one trigger configured for the system. These components are interchangeable as they follow simple interfaces described in the Endpoints, Triggers and Databases sections.
forgehook aims to be secure through simplicity and auditability. Security concerns are addressed in a dedicated section.
In most cases, you can just do ./setup.sh from the repository folder and it will setup everything just fine as long as you are a sudoer
TODO: introduce the setup.sh script and describe manual steps for installing
Configurating the forgehook user can only be done at setup time. Please refer to the setup docs.
TODO: Configuration for endpoint (none at the moment), databases (only unix support at the moment) and triggers (only git-build at the moment)
So you want to subscribe to updates on a remote repository? This is a quick introduction to the
forgehook CLI. The complete reference can be found in the docs/cli.md file.
Subscribing to a repository
To subscribe to a remote repository, use the
forgehook add REPO command, where
REPO is the URL of this repository.
If there is already a secret shared with the repository, the command will simply subscribe you to updates. However, if no secret has been configured yet, it will either extract it from your first argument (
forgehook add REPO SECRET) or generate one for you.
Note: If you just registered a secret, don't forget to give it to your forge.
$ webhook add "https://tildegit.org/tilde-fr/infra" [webhook] Your secret for https://tildegit.org/tilde-fr/infra is now: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX # <--- Hex-encoded /dev/urandom
TODO: Currently, no ownership check is performed on the repo (see issue #6)
To unsubscribe from a repository, simply use the
forgehook remove command:
$ webhook unsubscribe "https://tildegit.org/tilde-fr/infra" [webhook] Unsubscribed from https://tildegit.org/tilde-fr/infra [webhook] Users can still subscribe to this remote. To remove it entirely, run: webhook remove "https://tildegit.org/tilde-fr/infra"
list command lists your current subscriptions. If a URL is passed as argument, the command returns the list of users currently subscribed to the repository.
$ webhook list https://tildegit.org/tilde-fr/infra\tXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Note: For repositories you do not own, the
list command only returns one repository URLs per line. However, additional information separated by tabulations (
\t) are output for repositories you own, as explained in the CLI reference.
TODO: currently, only listing one's subscriptions is supported
Viewing/changing a secret
For a repository you own, you can always view your secret with the
forgehook secret URL command. If you give it an additional
SECRET, it will replace the current secret with this value:
$ webhook secret https://tildegit.org/tilde-fr/infra [webhook] Your secret for https://tildegit.org/tilde-fr/infra is now: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX $ $ webhook secret "https://tildegit.org/tilde-fr/infra" YYYYYYYYYYYYYYYY [webhook] Your secret for https://tildegit.org/tilde-fr/infra is now: YYYYYYYYYYYYYYYY
There are two entry points for the forgehook system:
a user running the
forgehookscript to manage their subscriptions/secrets
an endpoint running
forgehook-notifyto announce a legitimate update was received for a remote
forgehook-notify takes the URL of the updated repository as argument, checks for current user subscription, and runs the trigger (
/usr/local/bin/forgehook-trigger) as each user currently subscribed. The forgehook CLI interface is further described here.
Endpoints are simple services vouching for repository updates, usually by verifying a shared secret. Shortly, they are trusted 3rd party tools querying the local database in order to validate update notifications submitted by a remote forge (usually through a webhook). When a request is validated, an endpoint notifies the current trigger through the
The steps performed by an endpoint are:
- Extract repository URL and secret from the remote notification (eg. webhook)
- Query the local database for a secret matching this URL
- If nothing is found, the program stops
- Compare the remote and local secrets
- If they don't match, the program stops
forgehook-notifywith the remote URL as argument
There can be as many endpoints as you like to suit your needs. This allows to receive updates from multiple sources, such as different web forges, or even from a local
post-receive git hook. However, remember your system is only as secure as your endpoints, as explained in the Security section.
Triggers are simple scripts running as a user who subscribed to a task, thanks to sudo magic performed by
forgehook-notify. Trigger configuration is system-wide and there is only one trigger at any given moment (
A trigger receives a repository URL as argument. This repository is assumed to have legitimately received an update, as bad-faith requests have been filtered by the endpoint who received the original request.
A trigger may:
- send a mail or some sort of notification for the update
- trigger a test/deployment using a 3rd party tool such as git-build.sh
git-build is supported as trigger. More may come in the future, and contributions are welcome! Notably, it should be possible to integrate Github/Gitlab CI scripts as a trigger.
Note: Triggers are by nature unprivileged and only runs code the user should consent to (having subscribed to a repository). Running tests or building websites should not require complex infrastructure, as long as you have an account on a machine somewhere.
Databases are simple programs storing information about repository ownership, secrets and subscriptions. They may operate over an SQL or LDAP database, as long as they respect the
forgehook CLI interface described here. Database configuration is systemwide and there is only one database at any given moment (
The provided reference implementation for a forgehook database is a flat-file database managed by bash scripts, located in
databases/unix.sh, and documented here.
TODO: Explain sudo tricks and suggest everyone should read the code in its entirely because it's brief