If `--tty` is not passed to `docker exec` because stdout is not available (`[ ! -t 1 ]`), like due to redirection to file (`&> build.log`) or if stdin is not available (`< /dev/null`), then docker does not forward kill signals to the process started and they remain running.
To fix the issue, the `DOCKER_EXEC_PID_FILE_PATH` env variable with the value `/tmp/docker-exec-pid-<timestamp>` is passed to the process called with `docke exec` and the process started stores its pid in the file path passed. Traps are set in `run-docker.sh` that runs the `docker exec` command to receive any kills signals, and if it does, it runs another `docker exec` command to read the pid of the process previously started from `DOCKER_EXEC_PID_FILE_PATH` and then kills it and all its children.
See Also:
https://github.com/docker/cli/issues/2607https://github.com/moby/moby/issues/9098https://github.com/moby/moby/pull/41548https://stackoverflow.com/questions/41097652/how-to-fix-ctrlc-inside-a-docker-container
Also passing `--init` to `docker run` to "Run an init inside the container that forwards signals and reaps processes", although it does not work for above cases, but may helpful in others. The `--init` flag changes will only engage on new container creation.
https://docs.docker.com/engine/reference/run/#specify-an-init-processhttps://docs.docker.com/engine/reference/commandline/run/
```
./scripts/run-docker.sh ./build-package.sh -f libjpeg-turbo &> build.log
^C
$ ./scripts/run-docker.sh ps -efww
Running container 'termux-package-builder' from image 'termux/package-builder'...
UID PID PPID C STIME TTY TIME CMD
builder 1 0 0 05:48 pts/0 00:00:00 bash
builder 9243 0 0 06:01 pts/1 00:00:00 bash
builder 28127 0 0 06:12 ? 00:00:00 /bin/bash ./build-package.sh -f libjpeg-turbo
builder 28141 28127 0 06:12 ? 00:00:00 /bin/bash ./build-package.sh -f libjpeg-turbo
builder 28449 28141 1 06:12 ? 00:00:00 ninja -w dupbuild=warn -j 8
builder 28656 28449 0 06:12 ? 00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28657 28656 79 06:12 ? 00:00:01 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28694 28449 0 06:12 ? 00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28695 28694 89 06:12 ? 00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28728 28449 0 06:12 ? 00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28729 28728 0 06:12 ? 00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28731 28449 0 06:12 ? 00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28734 28731 0 06:12 ? 00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28740 28449 0 06:12 ? 00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28741 28740 0 06:12 ? 00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28744 0 0 06:12 pts/2 00:00:00 ps -efww
builder 28748 28449 0 06:12 ? 00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28752 28748 0 06:12 ? 00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28753 28449 0 06:12 ? 00:00:00 /bin/sh -c /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28754 28753 0 06:12 ? 00:00:00 /home/builder/.termux-build/_cache/android-r23c-api-24-v0/bin/clang
builder 28755 28449 0 06:12 ? 00:00:00 ninja -w dupbuild=warn -j 8
$ ./scripts/run-docker.sh ./build-package.sh -f libjpeg-turbo &> build.log
$ ./scripts/run-docker.sh ./build-package.sh -f libjpeg-turbo
Running container 'termux-package-builder' from image 'termux/package-builder'...
ERROR: Another build is already running within same environment.
```
Absolute paths are still allowed in `DCMAKE_INSTALL_LIBDIR` as per https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html
Different packages have different way of handling `DCMAKE_INSTALL_LIBDIR`. The `libprotobuf` is appending an absolute path to `build` directory (#10068), while `libjpeg-turbo` is not appending a relative `lib` path to `DCMAKE_INSTALL_PREFIX` and instead appending to `build` directory and so all the lib files stay at `/home/builder/.termux-build/libjpeg-turbo/build/lib` and hence won't get added to the `deb`, which results in `openjdk-17` failing if `-i` is not passed to `build-package.sh`, since it can't find `libjpeg.so` with `-L${TERMUX_PREFIX}/lib` after compilation from source, unless `-L$TERMUX_TOPDIR/libjpeg-turbo/build/lib` is passed.
Considering that most packages would likely be considering an absolute path passed in `DCMAKE_INSTALL_LIBDIR` to actually be absolute, the default behaviour should be reverted, specially considering it is what's been working, otherwise lot of packages would need testing, like from https://github.com/termux/termux-packages/commit/9155acd040.
```
checking for which libjpeg to use... system
checking jpeglib.h usability... yes
configure: WARNING: jpeglib.h: accepted by the compiler, rejected by the preprocessor!
checking jpeglib.h presence... no
checking for jpeglib.h... yes
configure: WARNING: jpeglib.h: proceeding with the compiler's result
configure: error: --with-libjpeg=system specified, but no libjpeg found
checking for jpeg_CreateDecompress in -ljpeg... no
configure exiting with result code 1
```
```
[0/1] Install the project...
-- Install configuration: "Release"
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/libturbojpeg.so
-- Installing: /data/data/com.termux/files/usr/bin/tjbench
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/libturbojpeg.a
-- Installing: /data/data/com.termux/files/usr/include/turbojpeg.h
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/libjpeg.a
-- Installing: /data/data/com.termux/files/usr/bin/rdjpgcom
-- Installing: /data/data/com.termux/files/usr/bin/wrjpgcom
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/README.ijg
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/README.md
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/example.txt
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/tjexample.c
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/libjpeg.txt
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/structure.txt
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/usage.txt
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/wizard.txt
-- Installing: /data/data/com.termux/files/usr/share/doc/libjpeg-turbo/LICENSE.md
-- Installing: /data/data/com.termux/files/usr/share/man/man1/cjpeg.1
-- Installing: /data/data/com.termux/files/usr/share/man/man1/djpeg.1
-- Installing: /data/data/com.termux/files/usr/share/man/man1/jpegtran.1
-- Installing: /data/data/com.termux/files/usr/share/man/man1/rdjpgcom.1
-- Installing: /data/data/com.termux/files/usr/share/man/man1/wrjpgcom.1
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/pkgconfig/libjpeg.pc
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/pkgconfig/libturbojpeg.pc
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/cmake/libjpeg-turbo/libjpeg-turboConfig.cmake
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/cmake/libjpeg-turbo/libjpeg-turboConfigVersion.cmake
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/cmake/libjpeg-turbo/libjpeg-turboTargets.cmake
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/cmake/libjpeg-turbo/libjpeg-turboTargets-release.cmake
-- Installing: /data/data/com.termux/files/usr/include/jconfig.h
-- Installing: /data/data/com.termux/files/usr/include/jerror.h
-- Installing: /data/data/com.termux/files/usr/include/jmorecfg.h
-- Installing: /data/data/com.termux/files/usr/include/jpeglib.h
-- Installing: /home/builder/.termux-build/libjpeg-turbo/build/lib/libjpeg.so
-- Installing: /data/data/com.termux/files/usr/bin/cjpeg
-- Installing: /data/data/com.termux/files/usr/bin/djpeg
-- Installing: /data/data/com.termux/files/usr/bin/jpegtran
...
Most scripts are set up to just be sourced, but these ones can handle
being run as well. They have not had a shebang set though, use
/usr/bin/bash to ensure they work as intended.
Some of the IPs of fosshost's proxy do not work (see
termux/termux-packages#11007), so stick to using one that we know work
until that is fixed.
Suggested by agnostic-apollo.
Make changes as per new design implemented in termux/termux-app@b950efec and termux/termux-app#2740
The package build and termux-tools scripts use current package manager for custom logic. The `termux-tools/termux-setup-package-manager` script has been added that will now be used to provide backward compatibility for termux-app `< 0.119.0` (when its released) and validate the package manager. It will also ensure the variable in not unset to prevent `unbound variable` errors if `set -u` is being used by calling scripts.
Closes#10782
Currently, we assume that a package is not coupled with any specific
version of it's dependencies. Therefore, we update them individually
without any specific order. But this assumtion fails for package
families like lxqt which requires all it's family members to be of
specific version.
Although we would have to manually update dependencies in such
situation (if they can not be auto-updated), but we can atleast
decide order of updation for packages that can be auto-updated.
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
- add checks for 'newest-tag' as fallback, if 'latest-release-tag' is not found
- adhere to DRY principle
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Should speed up downloads and also reduce situations where downloads
simply have to wait for timeout to happen (I have no idea why this
happens, probably nginx trying to cooldown some requests to prevent
(D)DoS?)
- Now auto update will be disable until issue is closed.
- Assign to GITHUB_ACTOR. Myself if GITHUB_ACTOR is Termux bot.
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Dockerfile setup script doesn't know exactly which ndk/sdk version we
are using, so can't cd to the right folder now that the version is
part of the folder name.
- GitHub api only allows 65536 chars long body in one go.
- Now body beyond 65536 limit will be appended after a gap of 5 sec.
Appending (or patching) does not consider already present body, so we
can bypass this limit.
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
- now a new issue with output logs will be created and assigned to
$GITHUB_ACTOR, if auto update fails for a package
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
%ci:no-build
Above tag is for previous commit but CI checks head commit for
this, so applying here.
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
- this variable will store path of directory containing packages
`build.sh` file.
- this is a more standard approach than specifying `pkg_dir` each time
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
x11-repo uses distribution x11, and root-repo distribution root.
Store this information in the json file as well, and parse it to set
both REPOSITORY_NAME and REPOSITORY_DISTRIBUTION.
Also remove unnecessary `< repo.json` from jq command, jq accepts the
file as an argument.
This fixes upload issues to x11-repo and root-repo.
Earlier in 5b8b15e7aa, I had configured
our upload scripts to treat failure of `aptly_add_to_repo` as failure to publish
packages at all. However it won't work, since even if all debs are
uploaded, and aptly_add_to_repo is called, debs are moved from temporary
directory to the repository. The proxy may drop connection during this
moment due to stale connection. And just after that we call
aptly_delete_dir which will delete the temporary directory. On the
server side, aptly will however continue to move debs from temporary
directory. Repository is published, but since the temporary directory is
removed during the transaction, the transaction never completely
succeeds. So only some of the debs are uploaded practically. This is now
fixed by adding a sleep of 180s (should be more than enough for all
stuff)
Building packages that use libtool for linking libraries has issues
with ndk-r23 (https://github.com/android/ndk/issues/1614). This check
makes sure that the built libraries do not give an error like
CANNOT LINK EXECUTABLE "tesseract": cannot locate symbol "__extenddftf2" referenced by "/data/data/com.termux/files/usr/lib/libtesseract.so"...
once used on device. In https://github.com/android/ndk/issues/1614 it
seemed like arm gave false positives and needed special consideration,
but when I tested it again today the same false positives do not occur
in built libraries, so no "if arm then .." workaround has been added
for now.
- move epoch addition to termux_pkg_upgrade_version set.
- now termux_pkg_upgrade_version can be given retrived version/tag from
api, directly.
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Hmmm..., why do we even need to check whether package is unique or not if we have
TERMUX_PKG_AUTO_UPDATE=true?
It should be checked before enabling auto-update.
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
previously setting TERMUX_PKG_UPDATE_VERSION_REGEXP had no effect on version being wriiten to
build.sh. It was only used for version comparison.
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
