specs/forgehook

+++ extra.pipeline = "endpoints" template = "project.html" +++

Webhook endpoints

This repository contains documentation and tests about running webhook endpoints as part of a forge suite setup. There are tests for CLI webhook validation programs (such as whck), as well as for web validation programs (HTTP endpoints such as endpoints.php). A certain number of conventions (environment variables and folder structure) should make it "just work" for different kinds of setups including shared hosts with unprivileged accounts. For setup information, please head over to the Setup section.

Below, you will find specifications for the CLI and Web programs. For the reference of supported webhook sources and their respective property, please read the Sources section

Web interface

A webhook HTTP endpoint is a web validator that receives requests from remote services, and validates them using a CLI validator as described in the Command-line interface section. A web validator implements support for various Sources to extract the claimed secret as well as the webhook content. For the moment, two different verification schemes (used by several sources) are supported:

• hmac-sha256 header signature: used by Github and Gitea to sign the body (JSON payload)
• token header: used by Gitlab without signatures, so the body (payload) is ignored

The web validator returns HTTP status code 200 when validation succeeded, or any other exit code if it fails. In the future, error codes could be standardized.

Conventions

In order to facilitate deployment in varied setups, each web validator respects the following conventions:

• returns HTTP code 200 when validation succeeded, or any other exit code if it fails. In the future, error codes could be standardized
• the CLI validator used to validate incoming webhooks is generated from the following possibilities, in order of preference:
  • from the WHCK environment variable (automated tests)
  • from the ./bin/whck program, which is not tracked in git (if you want to override whck from PATH)
  • from whck in the current PATH (for proper install)
• a bin/server script builds the web validator:
  • returning 0 upon success (or if the code doesn't require building) and outputting a complete path to the executable starting the program
  • any other exit code indicates failure
• a spec submodule contains this very repository (forge/endpoints)
• a bin/test.sh script starts the tests: ensures the spec submodule is cloned, and starts the spec/test_web.sh with the following arguments:
  • the full path to the executable (generated by bin/server)
  • (optional) the full path to bin/whck if it exists (user override), or the full path to whck from current PATH, or nothing if neither of those options matches
• the executable starts the web server on localhost, on a port which is configured, in this order of precedence:
  • by a FORGEHOOKPORT environment variable (used in tests)
  • by a first positional argument passed to the script (for manual use)

Testing

If you want to run tests for an existing implementation, please run the bin/test.sh script from the implementation's repository.

NOTE: Running tests requires the bats testing framework. You can usually find it in your distro repositories. If not, head over to the bats repository.

The test_web.sh script is used for testing web validators (such as endpoints.php). It takes two optional positional arguments:

• a path to a web validator executable (as defined in the Conventions section); if no argument is provided, the implementation's bin/server script will be executed to find a path to the executable; the bin/server script will be looked up on these paths, in order of precedence:
  • bin/server, relative to the directory from which you called the tests (eg. if you're running specs/test_web.sh from endpoints.php repository)
  • ../bin/server, relative to the directory containing test_web.sh (eg. if you're running ./test_web.sh from specs submodule in endpoints.php repository)
  • if no path was matched, the test suite fails to start
• a path to a CLI validator executable; if no argument is provided, the following paths will be attempted, in order of precedence:
  • the output of whck/bin/cli, relative to the current working dir (eg. calling specs/test_web.sh from endpoints.php repository)
  • the output of ../whck/bin/cli, relative to the current working dir (eg. calling ./test_web.sh from specs submodule in endpoints.php repository)
  • if no path was matched, the test suite fails to start

To run tests, simply call bin/test.sh from your implementation repository. If you call tests using specs/test_web.sh directly, it will assume you either want to test specific implementations (passed by arguments), or that you want to test the implementations in the parent repository. If you want the web endpoint tests to use the whck from PATH, please use the endpoint's bin/test.sh script.

Note: to test that the test suites can be called from several folders using different kind of relative/absolute paths, use the test_tests_web.sh script.:

Sources

Name	Payload	Secret type	Secret location	Repo location
Gitea	JSON body	hmac-sha256 sig	header:X_GITEA_SIGNATURE	repository>html_url
Github	JSON body	hmac-sha256 sig	header:X_HUB_SIGNATURE	repository>html_url
Gitlab	JSON body	token	header:X_GITLAB_TOKEN	project->git_http_url

Setup

Now it gets tricky... In all cases you need some kind of shell account! Talk about sudo, suid, please etc.

Unprivileged user, web server running as user

Unprivileged user, web server running as system account

Privileged user (requires root)

TODO

• reimplement calling forgehook-notify from endpoints.php
• add test to check it's actually called
• split READMEs separately for CLI and HTTP endpoint
• maybe have all specs/tests/READMEs (with forgebuild) in a single repo?
• make higher-level helper functions that can be shared across test files and implementation, for better consistency