Commit Graph

529 Commits

Author SHA1 Message Date
agnostic-apollo 7b9b9b6224 fix(run-docker.sh): Fix docker exec not passing kill signals (ctrl+c) to commands in some cases leaving processes still running
If `--tty` is not passed to `docker exec` because stdout is not available (`[ ! -t 1 ]`), like due to redirection to file (`&> build.log`) or if stdin is not available (`< /dev/null`), then docker does not forward kill signals to the process started and they remain running.

To fix the issue, the `DOCKER_EXEC_PID_FILE_PATH` env variable with the value `/tmp/docker-exec-pid-<timestamp>` is passed to the process called with `docke exec` and the process started stores its pid in the file path passed. Traps are set in `run-docker.sh` that runs the `docker exec` command to receive any kills signals, and if it does, it runs another `docker exec` command to read the pid of the process previously started from `DOCKER_EXEC_PID_FILE_PATH` and then kills it and all its children.

See Also:

https://github.com/docker/cli/issues/2607
https://github.com/moby/moby/issues/9098
https://github.com/moby/moby/pull/41548
https://stackoverflow.com/questions/41097652/how-to-fix-ctrlc-inside-a-docker-container

Also passing `--init` to `docker run` to "Run an init inside the container that forwards signals and reaps processes", although it does not work for above cases, but may helpful in others. The `--init` flag changes will only engage on new container creation.

https://docs.docker.com/engine/reference/run/#specify-an-init-process

https://docs.docker.com/engine/reference/commandline/run/

```
./scripts/run-docker.sh ./build-package.sh -f libjpeg-turbo  &> build.log
^C
$ ./scripts/run-docker.sh ps -efww
Running container 'termux-package-builder' from image 'termux/package-builder'...
UID          PID    PPID  C STIME TTY          TIME CMD
builder        1       0  0 05:48 pts/0    00:00:00 bash
builder     9243       0  0 06:01 pts/1    00:00:00 bash
builder    28127       0  0 06:12 ?        00:00:00 /bin/bash ./build-package.sh -f libjpeg-turbo
builder    28141   28127  0 06:12 ?        00:00:00 /bin/bash ./build-package.sh -f libjpeg-turbo
builder    28449   28141  1 06:12 ?        00:00:00 ninja -w dupbuild=warn -j 8
builder    28656   28449  0 06:12 ?        00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28657   28656 79 06:12 ?        00:00:01 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28694   28449  0 06:12 ?        00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28695   28694 89 06:12 ?        00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28728   28449  0 06:12 ?        00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28729   28728  0 06:12 ?        00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28731   28449  0 06:12 ?        00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28734   28731  0 06:12 ?        00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28740   28449  0 06:12 ?        00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28741   28740  0 06:12 ?        00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28744       0  0 06:12 pts/2    00:00:00 ps -efww
builder    28748   28449  0 06:12 ?        00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28752   28748  0 06:12 ?        00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28753   28449  0 06:12 ?        00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28754   28753  0 06:12 ?        00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder    28755   28449  0 06:12 ?        00:00:00 ninja -w dupbuild=warn -j 8
$ ./scripts/run-docker.sh ./build-package.sh -f libjpeg-turbo  &> build.log
$ ./scripts/run-docker.sh ./build-package.sh -f libjpeg-turbo
Running container 'termux-package-builder' from image 'termux/package-builder'...
ERROR: Another build is already running within same environment.
```
2022-07-28 15:27:20 +05:00
agnostic-apollo cd2928cbe1 fix(build-package.sh): Do not use dependencies while building from repo if it hosts packages built for a different package name than TERMUX_APP_PACKAGE 2022-07-28 15:27:20 +05:00
Leonid Pliushch 1fe284d818
build-package.sh: fix syntax error 2022-07-08 12:01:49 +03:00
Butta b8bfe98f10 upgpkg(swift): 5.6.1 to 5.6.2
Also, add a termux_setup_swift() bash function and move downloading the prebuilt
Swift toolchain for linux there.
2022-06-16 21:31:13 +05:30
agnostic-apollo 0275d9bff3 changed(scripts|main/termux-tools): Use TERMUX_APP_PACKAGE_MANAGER instead of TERMUX_MAIN_PACKAGE_FORMAT
Make changes as per new design implemented in termux/termux-app@b950efec and termux/termux-app#2740

The package build and termux-tools scripts use current package manager for custom logic. The `termux-tools/termux-setup-package-manager` script has been added that will now be used to provide backward compatibility for termux-app `< 0.119.0` (when its released) and validate the package manager. It will also ensure the variable in not unset to prevent `unbound variable` errors if `set -u` is being used by calling scripts.

Closes #10782
2022-05-23 12:14:55 +05:00
Henrik Grimler 56dbb9cf80
CI: specify distribution in repo.json as well
x11-repo uses distribution x11, and root-repo distribution root.
Store this information in the json file as well, and parse it to set
both REPOSITORY_NAME and REPOSITORY_DISTRIBUTION.

Also remove unnecessary `< repo.json` from jq command, jq accepts the
file as an argument.

This fixes upload issues to x11-repo and root-repo.
2022-04-20 16:37:36 +02:00
Henrik Grimler 4f3c709d78
build-package.sh: spaces -> tabs in indentation 2022-04-20 13:08:42 +02:00
Yaksh Bariya f0ea9b922b
monorepo: apply changes suggested by buttaface 2022-04-18 13:57:19 +05:30
Yaksh Bariya 52c383b4d1
chore: switch to monorepo 2022-04-18 13:57:13 +05:30
Aditya Alok f58de40f2e
feat(build-package.sh): create debscripts for haskell packages
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
2022-03-16 15:32:12 +01:00
Aditya Alok 49c3e79513
feat(setup ghc cross compiler): make it work independently from other build steps
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
2022-03-16 15:31:59 +01:00
Aditya Alok 225a560590
feat(build-package.sh): source haskell build scripts
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
2022-03-16 15:31:53 +01:00
Yaksh Bariya e5b54ebb84
build-package.sh: Fix typo for comment for termux_setup_nodejs
Node.js is not 'ninja build system', but a JavaScript Runtime based on V8

:P
2022-03-12 11:35:01 +05:30
Henrik Grimler 59f1c7b183
Revert "feat(ghc): enable cross-compiling haskell packages on CI"
This will need some more work.  Currently
termux_create_haskell_debscripts is run for all packages, even
non-haskell ones, which causes errors in postinst and prerm scripts in
an ugly way.

This reverts commit dc2474b2c2.
2022-03-09 07:40:36 +01:00
Aditya Alok dc2474b2c2 feat(ghc): enable cross-compiling haskell packages on CI
- `termux_setup_ghc_cross_compiler` should not be called by build.sh, as it installs cross-compiler
in $TERMUX_PREFIX (otherwise ghc adds wrong `rpaths` to built libs and executables)

Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
2022-03-09 10:01:57 +09:00
Tee KOBAYASHI 9849f01987 new function: termux_setup_python_crossenv 2022-03-02 12:00:30 +09:00
Henrik Grimler f66938e355
scripts/build/setup: add termux_setup_nodejs function
To be used by packages that need nodejs during build (gitea and
frida-server for example).

Use LTS version for now (gitea had issues with latest version).
2022-03-01 09:03:54 +01:00
Maxython 465159a72e
build-package.sh: change logic for package manager selection to be more stable
termux_step_setup_variables: added logic with TERMUX_PACKAGE_FORMAT
for easier compilation.
2022-02-11 22:38:40 +01:00
Henrik Grimler 0d5118c71d
build-package.sh: skip importing Fornwall's key
We aren't subscribed to any repos signed with his key.
2022-02-04 19:47:24 +01:00
xeffyr a978da484f update termux-keyring 2022-01-31 20:15:35 +02:00
Leap of Azzam 18fbc56c72 add package: zig
Signed-off-by: Leap of Azzam <leapofazzam@gmail.com>
2022-01-25 20:52:58 +09:00
Ivan Max ece5419863
build-package.sh: setting up to work with pacman (#8451) 2022-01-14 15:16:37 +02:00
Henrik Grimler 150008c502
build-package: import and trust grimler's new gpg key
I missed this and broke the CI build system.
2021-12-08 10:07:21 +01:00
Henrik Grimler af3f791645
build-package: use PACKAGE_FORMAT debian if not specified 2021-10-22 09:27:15 +02:00
Henrik Grimler 26da480d36 build-package: mv code for creating llvm-config
After https://github.com/termux/termux-packages/commit/e34ae5da030d,
llvm-config was created before termux_step_get_dependencies, meaning
that it gets overwritten if libllvm is installed.

Fix this by creating llvm-config in a new function
termux_step_override_config_scripts, that is run right after
termux_step_get_dependencies.
2021-09-17 12:02:53 +02:00
Leonid Pliushch 07852c874b
build-package.sh: allow to select package formats between debian and pacman 2021-09-10 19:55:28 +03:00
YAKSH BARIYA 2d38944910
new package: gn (#7290) 2021-09-03 20:46:03 +03:00
Henrik Grimler 6445645d7e build-package: mv code for setting up src and build to to its own function
%ci:no-build
2021-08-24 21:40:46 +02:00
Henrik Grimler ecd214881a build-package: add new arg -c for "continue build"
As replacement for TERMUX_PKG_QUICK_REBUILD. Running
./build-package.sh -c <package> starts a build for <package>, but does
not extract and patch the source from scratch. Instead it sets up the
build variables and starts from termux_step_make.

When working on a big package that can take hours to build it is
convenient to be able to build until there is an error, then apply
some new patch (manually) to the source, and then continue from where
the build failed.
2021-08-24 21:40:46 +02:00
Henrik Grimler 23530a540c build-package: create timestamp file after get_dependencies
Fixes issue introduced in e34ae5da03.
2021-08-21 22:37:51 +02:00
Lucinda May Phipps e6166f11d0
move termux_step_get_dependencies up one step 2021-08-21 21:25:26 +01:00
Henrik Grimler e34ae5da03 build-package: mv code for getting deps out of step_start_build
Into new function termux_step_get_dependencies.
2021-08-21 20:59:24 +02:00
Henrik Grimler 2c70621a98 build-package: rename TERMUX_DEBUG to TERMUX_DEBUG_BUILD
To make it more self-explanatory.
2021-08-21 20:55:57 +02:00
Leonid Pliushch 90e2de6fcc
build-package.sh: add hook to prevent execution of sudo
Do not let privileged commands appear in build.sh. Any environment
configuration should be done by scripts like setup-ubuntu.sh. Execution
of arbitrary sudo commands could mess up host system, in case if not
building package in Docker container or VM.
2021-07-07 23:27:54 +03:00
Leonid Pliushch 3337b9ffee
build-package.sh: compatibility with "reproducible builds"
Though git method of getting persistent timestamp won't work in submodules
and snapshots and will fallback to current date.
2021-05-19 13:19:23 +03:00
Leonid Pliushch c9664c11ba
build-package.sh: never use git outside of package build process.
termux-packages is not guaranteed to be always used as git repository.
Relying on git here may lead to issues.

P.S. Avoid unfinished work on master branch!
2021-05-19 13:12:27 +03:00
Henrik Grimler b3b5a3f35f mv checks from termux_step_create_datatar to termux_step_massage
By moving the checks we can make the function more general, and
thereby re-use it for creating subpackages and potentially hostbuild
tools packages.

tests: building libandroid-support and build-essential.
2021-05-16 20:08:37 +02:00
Lucy Phipps d5313fab26
get SOURCE_DATE_EPOCH from git log time
start of work towards reproducible builds
2021-05-11 20:36:23 +01:00
Leonid Pliushch 501ce516cd misc hardcoded home/prefix fixes 2020-10-30 22:48:55 +02:00
Leonid Pliushch 47eca5ed01 build-package.sh: enable offline mode when file ./build-tools/.installed is present
Once ./scripts/setup-offline-bundle.sh was executed, all sources and build tools
should be downloaded and build-package.sh should be able to use them.
2020-10-23 01:57:33 +03:00
Henrik Grimler bafdf9d9c0 Rename termux_extract_src_archive to termux_unpack_src_archive
%ci:no-build
2020-07-21 20:09:38 +02:00
Henrik Grimler b474395f89 build-package.sh: spaces->tab 2020-07-21 10:22:55 +02:00
Henrik Grimler 4e902a41a5 building packages: create termux_step_get_source function
Calls termux_git_clone_src if TERMUX_PKG_SRCURL ends with .git, and
termux_download_src_archive and termux_extract_src_archive otherwise.
termux_step_extract_package has been split up into the latter two
functions.

termux_step_post_extract_package has been renamed to
termux_step_post_get_source to reflect these changes.
2020-07-21 10:14:00 +02:00
Leonid Pliushch 2f58b9f435 build-package.sh: add function for setting up GHC 2020-06-20 22:44:13 +03:00
Leonid Pliushch 64ee31c9dc build-package.sh: fix unbound variable error 2020-03-23 23:45:11 +02:00
Leonid Pliushch 2f5b36deb4 build-package.sh: attempt to fix gpg when no tty available 2020-03-21 23:03:43 +02:00
Leonid Pliushch d523f61477 build-package.sh: verify release files with gpg when doing autobuilds (#5062) 2020-03-21 16:30:02 +02:00
Henrik Grimler dbeeaef9c9 build-package: add new function termux_step_create_subpkg_debscripts (#4963)
Makes it possible to create postinst scripts and similar or subpackages as well
2020-02-23 21:30:44 +01:00
Henrik Grimler ab2a3b70f0 build-package.sh: add TERMUX_PKG_SERVICE_SCRIPT var and build script
termux_step_install_service_scripts is run after
termux_step_post_make_install and loops over the new array
TERMUX_PKG_SERVICE_SCRIPT to add service scripts for termux-services.

The service scripts are usually only a one-liner so we might just as
well define it in a variable like TERMUX_PKG_SERVICE_SCRIPT.

TERMUX_PKG_SERVICE_SCRIPT should be an array on the format
("daemon-name" 'script to execute' "another daemon" 'multi\n line\n script'),
i.e. it should be of even length with name + script where the script
part preferably is within single quotes (to avoid accidental expansion
of for example $HOME).
2020-01-05 15:00:15 +01:00
Leonid Pliushch 85462a0042 build-package.sh: use absolute paths when sourcing parts from scripts/build/
+ some fixes for shellcheck.
2019-08-16 23:19:11 +03:00