readme: add deprecation notice

new database url
store db on releases page currently
2021-07-10 21:02:40 +03:00 · 2021-06-26 18:31:54 +03:00 · 2021-02-11 21:00:25 +02:00 · 2021-02-09 13:40:00 +02:00 · 2020-11-22 19:14:42 +02:00 · 2020-11-22 17:14:21 +02:00
4 changed files with 310 additions and 61 deletions
--- a/README.md
+++ b/README.md
@ -1,3 +1,58 @@
+# Deprecation notice
+
+This project is deprecated in favor of `apt-file` utility. Package `whatprovides` and database file will be removed.
+
+You can install `apt-file` package with next command:
+```
+pkg install apt-file
+```
+
+Depending on Termux version (< 0.118), you may need to fix permission of one file in $PREFIX (known issue):
+```
+chmod +x $PREFIX/lib/apt/apt-helper
+```
+
+Usage example:
+```
+~ $ apt-file update
+~ $ apt-file list termux-tools
+termux-tools: /data/data/com.termux/files/usr/bin/chsh
+termux-tools: /data/data/com.termux/files/usr/bin/cmd
+termux-tools: /data/data/com.termux/files/usr/bin/dalvikvm
+termux-tools: /data/data/com.termux/files/usr/bin/df
+termux-tools: /data/data/com.termux/files/usr/bin/getprop
+termux-tools: /data/data/com.termux/files/usr/bin/ip
+termux-tools: /data/data/com.termux/files/usr/bin/logcat
+termux-tools: /data/data/com.termux/files/usr/bin/login
+termux-tools: /data/data/com.termux/files/usr/bin/mount
+termux-tools: /data/data/com.termux/files/usr/bin/ping
+termux-tools: /data/data/com.termux/files/usr/bin/ping6
+termux-tools: /data/data/com.termux/files/usr/bin/pkg
+termux-tools: /data/data/com.termux/files/usr/bin/pm
+termux-tools: /data/data/com.termux/files/usr/bin/settings
+termux-tools: /data/data/com.termux/files/usr/bin/su
+termux-tools: /data/data/com.termux/files/usr/bin/termux-change-repo
+termux-tools: /data/data/com.termux/files/usr/bin/termux-fix-shebang
+termux-tools: /data/data/com.termux/files/usr/bin/termux-info
+termux-tools: /data/data/com.termux/files/usr/bin/termux-open
+termux-tools: /data/data/com.termux/files/usr/bin/termux-open-url
+termux-tools: /data/data/com.termux/files/usr/bin/termux-reload-settings
+termux-tools: /data/data/com.termux/files/usr/bin/termux-reset
+termux-tools: /data/data/com.termux/files/usr/bin/termux-setup-storage
+termux-tools: /data/data/com.termux/files/usr/bin/termux-wake-lock
+termux-tools: /data/data/com.termux/files/usr/bin/termux-wake-unlock
+termux-tools: /data/data/com.termux/files/usr/bin/top
+termux-tools: /data/data/com.termux/files/usr/bin/umount
+termux-tools: /data/data/com.termux/files/usr/bin/xdg-open
+termux-tools: /data/data/com.termux/files/usr/etc/motd
+termux-tools: /data/data/com.termux/files/usr/etc/profile.d/init-termux-properties.sh
+termux-tools: /data/data/com.termux/files/usr/share/doc/termux-tools/LICENSE
+termux-tools: /data/data/com.termux/files/usr/share/examples/termux/termux.properties
+termux-tools: /data/data/com.termux/files/usr/share/man/man1/termux.1.gz
+```
+
+***
+
 # whatprovides
 yum --whatprovides utility for termux.

@ -15,10 +70,10 @@ pkg install whatprovides
 ### Manual Installation:

 ```
-pkg install git build-esssential
+pkg install git gawk gzip make sqlite
 git clone https://github.com/termux/whatprovides
 cd whatprovides
-make or make install 
+make install
 ```

 ### Usage:
@ -27,37 +82,22 @@ make or make install

 ```sh
 $ whatprovides -p aapt
-aapt: /.
-aapt: /data
-aapt: /data/data
-aapt: /data/data/com.termux
-aapt: /data/data/com.termux/files
-aapt: /data/data/com.termux/files/usr
-aapt: /data/data/com.termux/files/usr/share
-aapt: /data/data/com.termux/files/usr/share/aapt
-aapt: /data/data/com.termux/files/usr/share/aapt/android.jar
-aapt: /data/data/com.termux/files/usr/share/doc
-aapt: /data/data/com.termux/files/usr/share/doc/aapt
-aapt: /data/data/com.termux/files/usr/lib
-aapt: /data/data/com.termux/files/usr/lib/libandroid-cutils.so
-aapt: /data/data/com.termux/files/usr/lib/libandroid-ziparchive.so
-aapt: /data/data/com.termux/files/usr/lib/libandroid-base.so
-aapt: /data/data/com.termux/files/usr/lib/libandroid-fw.so
-aapt: /data/data/com.termux/files/usr/lib/libandroid-utils.so
-aapt: /data/data/com.termux/files/usr/bin
 aapt: /data/data/com.termux/files/usr/bin/aapt
 aapt: /data/data/com.termux/files/usr/bin/zipalign
+aapt: /data/data/com.termux/files/usr/lib/libandroid-base.so
+aapt: /data/data/com.termux/files/usr/lib/libandroid-cutils.so
+aapt: /data/data/com.termux/files/usr/lib/libandroid-fw.so
+aapt: /data/data/com.termux/files/usr/lib/libandroid-utils.so
+aapt: /data/data/com.termux/files/usr/lib/libandroid-ziparchive.so
+aapt: /data/data/com.termux/files/usr/share/aapt/android.jar
 aapt: /data/data/com.termux/files/usr/share/doc/aapt/LICENSE
 ```
-[![asciicast](https://cdn.jsdelivr.net/gh/termux/whatprovides@master/static/screenshot.jpg)](https://asciinema.org/a/373670)
-[Asciinema preview ](https://asciinema.org/a/373670)

 * search for files. Have you ever used `dpkg -S filepath`. 
 Here you need to use like `whatprovides filepath`. 
 You can search for any files which even not exist in your termux.
 it will show you which package provide that file.
 ```sh
-$ whatprovides $PREFIX/bin/aapt
-aapt: data/data/com.termux/files/usr/bin/aapt
-
+$ whatprovides $PREFIX/share/aapt
+aapt: /data/data/com.termux/files/usr/share/aapt
 ```
--- a/static/screenshot.jpg
+++ b/static/screenshot.jpg
--- a/update-whatprovides-db.sh
+++ b/update-whatprovides-db.sh
@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+##
+## Script that generates or updates a Whatprovides
+## database for Termux.
+##
+
+set -e -u
+
+list_files() {
+	FILES=$(dpkg-deb --fsys-tarfile "${1}" | tar -t | cut -b2- \
+		| xargs -rd\\n realpath -sm --relative-base="/data/data/com.termux/files/usr" -- \
+		| grep -vEx '[./]|/data(/data(/com\.termux(/files)?)?)?')
+	SORTKEYS='-k1,1'
+	MAXKEY=$(awk -F/ 'BEGIN { x=0 } { if (NF > x) x = NF } END { print x }' <<< "${FILES}")
+	for ((x=2; x<=$MAXKEY; x++)); do
+		SORTKEYS="$SORTKEYS -k$x,$x"
+	done
+	sort -t/ $SORTKEYS <<< "$FILES" \
+		| awk 'NR == 1 { p=$0; next } substr($0, 1, length(p) +1 ) != p"/" { print p } { p=$0 } END { print p }'
+}
+
+write_sql_script() {
+	local debfile_dir=$1
+	local debfile_path
+
+	# SQL statements should be executed within a single
+	# transaction, otherwise execution performance would
+	# be very slow.
+	echo "BEGIN TRANSACTION;" >> "$SQLSCRIPT_TMPFILE"
+
+	while read -r debfile_path; do
+		# .deb files are named package-name_version-revision_cpuarch.deb
+		local package_name
+		package_name=$(basename "$debfile_path" | cut -d_ -f1)
+
+		echo "[*] Processing '${debfile_path}'..." >&2
+
+		# Delete content of processed package to ensure there will not be any
+		# duplicates and stale entries.
+		echo "DELETE FROM whatprovides WHERE package_name == \"${package_name}\";" >> "$SQLSCRIPT_TMPFILE"
+
+		# Add row with matching package-name and file-name.
+		while read -r file_path; do
+			echo "${package_name}: adding '${file_path}'" >&2
+			echo "INSERT INTO whatprovides (package_name, owned_file) VALUES (\"${package_name}\", \"${file_path}\");" >> "$SQLSCRIPT_TMPFILE"
+		done < <(list_files "$debfile_path")
+	# Note: searching only for ARCH "all" (platform-independent) and "aarch64" which
+	# has 100% support by Termux packages.
+	done < <(find "$debfile_dir" -type f \( -name "*_all.deb" -o -name "*_aarch64.deb" \) -print)
+
+	echo "COMMIT;" >> "$SQLSCRIPT_TMPFILE"
+
+	# Databases which were not generated from scratch needs
+	# to be compacted.
+	echo "VACUUM;" >> "$SQLSCRIPT_TMPFILE"
+	echo "REINDEX;" >> "$SQLSCRIPT_TMPFILE"
+}
+
+if [ $# -ge 2 ]; then
+	SQLSCRIPT_TMPFILE=$(mktemp -t "whatprovides.XXXXXX.sql")
+
+	if [ ! -e "${1}" ]; then
+		echo "[*] Creating new database." >&2
+		sqlite3 "$1" "CREATE TABLE whatprovides ( id INTEGER PRIMARY KEY, package_name TEXT NOT NULL, owned_file TEXT NOT NULL);"
+	fi
+
+	echo "[*] Generating SQL script..." >&2
+	write_sql_script "$2" "$SQLSCRIPT_TMPFILE"
+
+	echo "[*] Rebuilding database..." >&2
+	sqlite3 "$1" < "$SQLSCRIPT_TMPFILE"
+
+	rm -f "${SQLSCRIPT_TMPFILE}"
+
+	echo "[*] Done." >&2
+else
+	echo
+	echo "Usage: update-whatprovides-db.sh [db path] [directory with .deb files]"
+	echo
+	echo "Regenerate a database for Termux Whatprovides utility."
+	echo
+	exit 1
+fi
--- a/198
+++ b/198
@ -3,59 +3,185 @@ set -e

 SCRIPT_NAME=$(basename "$(realpath "$0")")
 DB_PATH="/data/data/com.termux/files/usr/var/lib/whatprovides/whatprovides.db"
-DB_UPDATES_URL="https://dl.bintray.com/termux/metadata/whatprovides-db/whatprovides.db.gz"
+DB_UPDATES_URL="https://packages.termux.org/whatprovides/whatprovides.db.gz"

 show_usage () {
-	echo
-	echo "Usage: $SCRIPT_NAME [options] path/to/file"
-	echo
-	echo "Find out packages using specific files."
-	echo
-	echo "Options:"
-	echo
-	echo "  -h            Show this help."
-	echo "  -p [package]  List files of package."
-	echo "  -u            Update the database."
-	echo
+	{
+		echo
+		echo "Usage: $SCRIPT_NAME [-u|-U] path/to/file"
+		echo "       $SCRIPT_NAME -p [-u|-U] package"
+		echo
+		echo "Find out packages using specific files."
+		echo
+		echo "Options:"
+		echo
+		echo "  -h   Show this help."
+		echo
+		echo "  -p   Reverse mode. List all files owned by specified"
+		echo "       package."
+		echo
+		echo "  -u   Update the database."
+		echo
+		echo "  -U   Redownload the database even if it's not updated."
+		echo
+		echo "  -q   Quiet mode. Suppress informational messages."
+		echo
+	} >&2
 }

 update_database() {
 	if [ -e "${DB_PATH}.gz" ]; then
-		echo "[*] Cleaning up the remaining temporary files..."
+		if ! ${QUIET}; then
+			echo "[*] Cleaning up the remaining temporary files..." >&2
+		fi
 		rm -f "${DB_PATH}.gz"
 	fi

-	echo "[*] Downloading the new database..."
-	echo
-	curl --fail --retry 5 --retry-connrefused --retry-delay 5 --location \
-		--output "${DB_PATH}.gz" "${DB_UPDATES_URL}"
-	echo
+	if ! ${FORCE_UPDATE} && [ -e "${DB_PATH}" ]; then
+		NEWER=(--time-cond "${DB_PATH}")
+	else
+		NEWER=()
+	fi

-	echo "[*] Installing..."
-	rm -f "${DB_PATH}"
-	zcat "${DB_PATH}.gz" > "${DB_PATH}"
-	rm -f "${DB_PATH}.gz"
+	if ! ${QUIET}; then
+		echo "[*] Downloading the new database..." >&2
+		echo >&2

-	echo "[*] Finished."
+		curl --fail --retry 3 --retry-connrefused --retry-delay 1 --location \
+			"${NEWER[@]}" --output "${DB_PATH}.gz" "${DB_UPDATES_URL}"
+
+		echo >&2
+	else
+		curl --silent --fail --retry 3 --retry-connrefused --retry-delay 1 \
+			--location "${NEWER[@]}" --output "${DB_PATH}.gz" "${DB_UPDATES_URL}"
+	fi
+
+	if [ -e "${DB_PATH}.gz" ]; then
+		if ! ${QUIET}; then
+			echo "[*] Installing..." >&2
+		fi
+		zcat "${DB_PATH}.gz" > "${DB_PATH}"
+		rm -f "${DB_PATH}.gz"
+		if ! ${QUIET}; then
+			echo "[*] Finished." >&2
+		fi
+	elif ! ${QUIET}; then
+		echo "[*] No update done." >&2
+	fi
 }

-while getopts :hp:u option; do
-	case "$option" in
-		p)
-			echo "SELECT owned_file FROM 'whatprovides' WHERE package_name == '${OPTARG}' ORDER BY owned_file;" | \
-				sqlite3 "${DB_PATH}" | awk "{ print \"${OPTARG}: \"\$0 }"
-			exit 0
+check_database() {
+	if [ ! -e "${DB_PATH}" ]; then
+		{
+			echo "Error: database is not available."
+			echo "Please run '${SCRIPT_NAME} -u' to create it."
+		} >&2
+		return 1
+	fi
+}
+
+REVERSE_MODE=false
+DO_UPDATE=false
+FORCE_UPDATE=false
+QUIET=false
+while (($# > 0)); do
+	case "$1" in
+		-h) show_usage; exit 0;;
+		-p) REVERSE_MODE=true;;
+		-q) QUIET=true;;
+		-U) FORCE_UPDATE=true;& # fall-through
+		-u)
+			if ${DO_UPDATE}; then
+				echo "-u and -U are mutually exclusive." >&2
+				show_usage
+				exit 1
+			fi
+			DO_UPDATE=true
+			;;
+		-*)
+			echo >&2
+			echo "Unknown option '$1'." >&2
+			show_usage
+			exit 1
+			;;
+		*)
+			break
 			;;
-		u) update_database; exit 0;;
-		h) show_usage; exit 0;;
-		*) show_usage; exit 1;;
 	esac
+	shift 1
 done

-if [ $# -ge 1 ]; then
-	echo "SELECT package_name FROM 'whatprovides' WHERE owned_file == '${1}' ORDER BY package_name;" | \
-		sqlite3 "${DB_PATH}" | awk "{ print \$0\": ${1}\" }"
+if ${DO_UPDATE}; then
+	update_database
+	[ $# -lt 1 ] && exit 0
 else
-	show_usage
+	check_database
+fi
+
+if [ $# -lt 1 ]; then
+	{
+		echo
+		echo "Error: you have not specified the file or package."
+		echo
+	} >&2
 	exit 1
 fi
+
+if ${REVERSE_MODE}; then
+	if ! grep -qx '[a-z0-9_+-]\+' <<< "$1"; then
+		{
+			echo
+			echo "Error: package name '${1}' is not valid."
+			echo
+		} >&2
+		exit 1
+	fi
+
+	if ! sqlite3 "${DB_PATH}" \
+		"SELECT owned_file FROM 'whatprovides' WHERE package_name == '${1}' ORDER BY owned_file" \
+		| awk "{ if (substr(\$0, 1, 1) == \"/\") print \"${1}: \"\$0; else print \"${1}: ${PREFIX}/\"\$0 } END { if (NR == 0) exit 1 }"
+	then
+		{
+			echo
+			echo "Error: package '$1' is not found."
+			echo
+		} >&2
+		exit 1
+	fi
+else
+	NEWPREFIX=
+	FILE="$(realpath -sm "$1")"
+
+	if [[ "${PREFIX}/" = "${FILE}/"* || "${FILE}" = / ]]; then
+		# FILE is (((great...)grand)parent of) PREFIX, including root
+		if [ "${FILE}" = / ]; then
+			FILE=/.
+		fi
+
+		sqlite3 "${DB_PATH}" \
+			"SELECT DISTINCT package_name FROM 'whatprovides' ORDER BY package_name" \
+			| awk "{ print \$0\": ${FILE}\" }"
+		exit 0
+	fi
+
+	if [[ "${FILE}" = "${PREFIX}/"* ]]; then
+		# FILE is in PREFIX
+		FILE="${FILE##${PREFIX}/}"
+		NEWPREFIX="${PREFIX}/"
+	fi # else FILE is entirely separate from PREFIX (unlikely to be in DB)
+
+	FILE_ESC="${FILE//\'/\'\'}"
+	if ! sqlite3 "${DB_PATH}" \
+		"SELECT DISTINCT package_name FROM 'whatprovides' WHERE owned_file == '${FILE_ESC}' OR substr(owned_file, 1, ${#FILE} +1) == '${FILE_ESC}/' ORDER BY package_name" \
+		| awk "{ print \$0\": ${NEWPREFIX}${FILE}\" } END { if (NR == 0) exit 1 }"
+	then
+		{
+			echo
+			echo "Error: file '$1' is not found."
+			echo
+		} >&2
+		exit 1
+	fi
+fi
+
+exit 0
Author	SHA1	Message	Date
Leonid Pliushch	db3a7d3d7b	readme: add deprecation notice	2021-07-10 21:02:40 +03:00
Leonid Pliushch	99aa49cdfd	new database url	2021-06-26 18:31:54 +03:00
Leonid Pliushch	c0afb9d22d	store db on releases page currently https://github.com/termux/termux-packages/issues/6348	2021-02-11 21:00:25 +02:00
Lucy Phipps	5e163c1518	only download db if newer, -U to force (#15 )	2021-02-09 13:40:00 +02:00
Lucy Phipps	83d3c35496	hopefully fix sorting large packages	2020-11-22 19:14:42 +02:00
Leonid Pliushch	e141b80b6f	Merge pull request #13 from landfillbaby/patch-1 update usage examples	2020-11-22 17:14:21 +02:00
Lucy Phipps	c1a702eeb7	don't set $TERMUX_PREFIX, it's used once	2020-11-22 14:56:33 +00:00
Lucy Phipps	074df540d7	update usage examples removed screenshot and asciinema links temporarily	2020-11-22 12:51:44 +00:00
Lucy Phipps	8fdd508949	only bother with leaf nodes, smaller but slower? (#12 ) also error message when package not found also correctly prepend $PREFIX on reverse mode	2020-11-22 11:53:05 +05:30
Lucy Phipps	96a9fd100a	db builder: allow files outside of $PREFIX just in case!	2020-11-22 03:12:44 +02:00
Lucy Phipps	9afbbb72de	update-whatprovides-db.sh: avoid '.' entries in db (#10 )	2020-11-22 01:23:45 +02:00
Leonid Pliushch	78b19eef01	update-whatprovides-db.sh: define TERMUX_PREFIX	2020-11-22 00:09:22 +02:00
Lucy Phipps	c372df97f6	vastly reduce database size by not storing $PREFIX also escape filenames for sqlite, realpath -sm	2020-11-22 00:07:33 +02:00
Leonid Pliushch	f93bad1cfc	add script for generating or updating database	2020-11-21 20:39:46 +02:00
Krishna Kanhaiya	8078679e7a	Merge pull request #7 from landfillbaby/patch-1 a few improvements	2020-11-21 09:03:37 +05:30
Lucy Phipps	b9bd4408db	a few improvements	2020-11-21 03:27:21 +00:00
Krishna Kanhaiya	3d40821f10	Fix: files checks. Now it will be able to list those files which not exist in termux. #6	2020-11-21 08:33:34 +05:30
Leonid Pliushch	451422bc8f	few improvements - Send info/error messages to stderr. - Option to suppress all info messages except errors. - Allow to upgrade database before processing input. - Verify user input, show errors when file does not exist or package name is not valid. - Get rid of 'getopts' - don't like it.	2020-11-20 17:35:36 +02:00
Krishna Kanhaiya	2ebb0d3b2f	Update manual installation instructions	2020-11-20 19:07:32 +05:30
Leonid Pliushch	fdcf1f8417	check whether db exists before trying to open it	2020-11-20 03:03:10 +02:00
Leonid Pliushch	712f54797f	db update: reduce retry delay to 1 second and set amount to 3	2020-11-20 02:55:45 +02:00