libman

libman is a package manager... of sorts. It manages ebooks and sources of ebooks. It works based on json files made to a specific format. It does not, itself, house any files. You add a source and then can filter/search through the source data and install books as you see fit.

Building

You need a go compiler (>= 1.18 is probably best).

git clone https://tildegit.org/sloum/libman
cd libman
go install
libman -h

You can always use go build instead of go install. If you do so, change the last line to ./libman -h.

Usage

There are currently four known json files supporting our format (three scraped, one hand built):

• https://rawtext.club/~sloum/lib/standard-ebooks.json
• https://rawtext.club/~sloum/lib/global-grey.json
• https://rawtext.club/~sloum/lib/project-gutenberg.json
• https://hearthstories.org/hearth-stories.json

The PG file is quite large, both in file size and in number of books, and does slow down querying a bit.

To add a source

Let's add two. That way we get some variety. As things are currently built, each add needs to be a separate action (the program does not support multiple -add options in one command).

libman -add https://rawtext.club/~sloum/lib/standard-ebooks.json
libman -add https://rawtext.club/~sloum/lib/global-grey.json

To list your current sources

libman -sources
#
# Output:
#
# Global Grey: https://rawtext.club/~sloum/lib/global-grey.json
# Standard Ebooks: https://rawtext.club/~sloum/lib/standard-ebooks.json

To search

A basic search by author...

libman -author "fenimore cooper"
#
# Output:
#
# The Deerslayer              by James Fenimore Cooper    Global Grey       pdf, epub, mobi, azw3, html
# The Last of the Mohicans    by James Fenimore Cooper    Global Grey       pdf, epub, mobi
# The Last of the Mohicans    by James Fenimore Cooper    Standard Ebooks   epub, azw3, kepub
# The Pathfinder              by James Fenimore Cooper    Global Grey       pdf, epub, mobi, azw3, html

...you can use any part of the authors name (the whole thing is not usually required, and it is not case sensitive).

We can hone in on a specific source. You will notice in the example below that we only use part of the source name. Use as much of a name as is needed to get the specificity you want.

libman -author "fenimore cooper" -source "global"
#
# Output:
#
# The Deerslayer              by James Fenimore Cooper    Global Grey       pdf, epub, mobi, azw3, html
# The Last of the Mohicans    by James Fenimore Cooper    Global Grey       pdf, epub, mobi
# The Pathfinder              by James Fenimore Cooper    Global Grey       pdf, epub, mobi, azw3, html

...that still leaves us with three results. In this case all three books are part of a series (not something the cli tells you), so if we want to get them all we can add an -install flag to the above command. Or we could use -title to get a specific one. Such as: `-title "deer"

If we want to get more detailed information we can add a -l flag for long resuls:

libman -author "fenimore cooper" -source "global" -title "deer" -l
#
# Output:
#
# The Deerslayer by James Fenimore Cooper
# Source:        Global Grey
# Subjects:      adventure, fiction, historical fiction
# Files:
#   pdf   - https://www.globalgreyebooks.com/ebooks/james-fenimore-cooper_deerslayer.pdf
#   epub  - https://www.globalgreyebooks.com/ebooks/james-fenimore-cooper_deerslayer.epub
#   mobi  - https://www.globalgreyebooks.com/ebooks/james-fenimore-cooper_deerslayer.mobi
#   azw3  - https://www.globalgreyebooks.com/ebooks/james-fenimore-cooper_deerslayer.azw3
#   html  - https://www.globalgreyebooks.com/online-ebooks/james-fenimore-cooper_deerslayer_complete-text.html
# Description:
#   The Deerslayer, or The First War-Path, is a novel by American writer James Fenimore Cooper, first published in 1841. It is the final book in the Leatherstocking Tales, although chronologically it is the first and is considered as a prequel to the other novels. Set in early 18th-century New York on Otsego Lake, the book tells the story of Natty Bumppo, a young frontiersman who is the child of white parents, but who grew up among Delaware Indians. Whilst objecting to the practice of scalping, Bummpo gets involved in an adventure with two men (Henry March and Tom Hutter) who have no such reservations. The book has been adapted into many films, as well as a tv series, and a radio drama.
# 
# № 1 in the Leatherstocking Tales.
# 
# This book has 212,978 words, 320 pages in the PDF version, and was originally published in 1841.
# 
# Production notes:  This ebook of The Deerslayer was published by Global Grey on the 7th July 2022. The artwork used for the cover is 'Indian Hunter' by Albert Bierstadt.

To download/install

Okay. Let's download all three of these rather than just the one. Wait, what format do we want? The output (either long or short) lists the available formats for each. Let's get the epub by adding a -format option. Note that if you do not include -format you will get a query as to whether or not you want to proceed (y/n).

libman -author "fenimore cooper" -source "global grey" -format "epub" -install
#
# Output:
#
# GET https://www.globalgreyebooks.com/ebooks/james-fenimore-cooper_deerslayer.epub
# GET https://www.globalgreyebooks.com/ebooks1/james-fenimore-cooper/last-of-the-mohicans/last-of-the-mohicans.epub
# GET https://www.globalgreyebooks.com/ebooks/james-fenimore-cooper_pathfinder.epub

Great! Those got added to our book download folder, which defaults to ~/.local/share/libman, so we can read them with whatever reader we prefer. An important note is that when you use the -install option it will download all the books that you are currently filtered to. If you are not careful this could end up being a huge quantity of books. So narrow things down to the correct specificity before downloading.

To get updates from your sources

It is easy to update your book cache. Every now and again run the following:

libman -sync
#
# Output:
#
# Syncing Sources
# Retrieving: Global Grey...
# Retrieving: Standard Ebooks...

That will go through each source you have added and check to see if there is a newer version of the source's json file. To do so, it downloads the json, parses it, and compares the update date. If the version that was just downloaded is newer it replaces the older version in your local cache. Since syncing does require a full download and parse cycle, it can take a bit for some sources. Unless you are wanting a specific book you know a source added most users can probably sync once every few weeks or even months and have plenty of great books to read.

To update settings

To update configuration/settings we can do as follow:

libman -configure

...which will open the configuration file in your $EDITOR. The main use case for this would be to change your install location. Most other settings can be left alone, unless you really want to tinker in the file rather than use the program itself (such as for adding and removing sources).

If you prefer to just open the file in your editor directly you can do something like:

vim ~/.config/libman/settings.json

Other filters/options

There are other filters (for example by -subject... for which you can also use -subjects to see the available subjects).

libman -h
#
# Output:
#
# Usage of ./libman:
#   -a    Sort by author instead of title
#   -add string
#         Add a URL or filepath as a source location
#   -author string
#         Filter by author
#   -configure
#         Open the configuration file in your $EDITOR
#   -format string
#         Filter by format; ex. "epub"
#   -install
#         Installs the results of the query
#   -l    Show full details for each searched book
#   -remove string
#         Remove a source by name
#   -source string
#         Filter results by the given source
#   -sources
#         List sources
#   -subject string
#         Filter by subject
#   -subjects
#         List all subjects in cache
#   -sync
#         Sync sources
#   -title string
#         Filter by title

Why

I like command line tools. I like books. I doubt many folks will start offering up their libraries in this format, but it is a decent idea for book distribution (even if I didnt handle it as elegantly as some folks might like). Mostly this caters to the public domain books crowd, but there are lots of CC licensed works that cold be distributed this way as well. Any individual person can distribute their library as a json file that contains download links and any number of frontends could use this file type. If nothing else, it lets me search a few places that I would otherwise have to go to a few different web pages to search, so that is nice.