Initial commit
This commit is contained in:
sloum
commit
97ba4bc023
|
|
@ -0,0 +1 @@
|
|||
libman
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
# libman
|
||||
|
||||
libman is a package manager... of sorts. It manages ebooks and sources of ebooks. It works based on json files made to a specific format. It does not, itself, house any files. You add a _source_ and then can filter/search through the source data and install books as you see fit.
|
||||
|
||||
## Building
|
||||
|
||||
You need a [go](https://golang.org) compiler (>= 1.18 is probably best).
|
||||
|
||||
```sh
|
||||
git clone https://tildegit.org/sloum/libman
|
||||
cd libman
|
||||
go install
|
||||
libman -h
|
||||
```
|
||||
|
||||
You can always use `go build` instead of `go install`. If you do so, change the last line to `./libman -h`.
|
||||
|
||||
## Usage
|
||||
|
||||
There are currently three known json files supporting our format (two scraped, one hand built):
|
||||
|
||||
- https://rawtext.club/~sloum/lib/standard-ebooks.json
|
||||
- https://rawtext.club/~sloum/lib/global-grey.json
|
||||
- https://hearthstories.org/hearth-stories.json
|
||||
|
||||
### To add a source
|
||||
|
||||
```
|
||||
libman -add https://rawtext.club/~sloum/lib/standard-ebooks.json
|
||||
```
|
||||
|
||||
### To list your current sources
|
||||
|
||||
```
|
||||
libman -sources
|
||||
```
|
||||
|
||||
### To search
|
||||
|
||||
A basic search by author...
|
||||
|
||||
```
|
||||
libman -author "fenimore cooper"
|
||||
```
|
||||
|
||||
...you can use any part of the authors name (the whole thing is not usually required, and it is not case sensitive).
|
||||
|
||||
We can hone in on a specific source...
|
||||
|
||||
```
|
||||
libman -author "fenimore cooper" -source "global grey"
|
||||
```
|
||||
|
||||
...that still leaves us with three results. They are all part of a series, so we can skip doing a `-title` filter if we want the whole series.
|
||||
|
||||
If we want to get more detailed information we can add a `-l` flag for long resuls:
|
||||
|
||||
```
|
||||
libman -author "fenimore cooper" -source "global grey" -l
|
||||
```
|
||||
|
||||
### To download/install
|
||||
|
||||
Okay. Let's download these. Wait, what format do we want? The output (either long or short) lists the available formats for each. Let's get the `epub`.
|
||||
|
||||
```
|
||||
libman -author "fenimore cooper" -source "global grey" -format "epub" -install
|
||||
```
|
||||
|
||||
Great! Those got added to our book download folder, which defaults to `~/.local/share/libman`, so we can read them with whatever reader we prefer. An important note is that when you use the `-install` option it will download _all the books_ that you are currently filtered to. If you are not careful this could end up being a huge quantity of books. So narrow things down to the correct specificity before downloading.
|
||||
|
||||
To update configuration/settings we can do as follow:
|
||||
|
||||
```
|
||||
libman -configure
|
||||
```
|
||||
|
||||
...which will open the configuration file in your `$EDITOR`.
|
||||
|
||||
There are other filters (for example by `-subject`... for which you can also use `-subjects` to see the available subjects). To see the full list run: `libman -h`.
|
||||
|
||||
## Why
|
||||
|
||||
I like command line tools. I like books. I doubt many folks will start offering up their libraries in this format, but it is a decent idea for book distribution (even if I didnt handle it as elegantly as some folks might like). Mostly this caters to the public domain books crowd, but there are lots of CC licensed works as well that cold be distributed this way as well. Any individual person can distribute their library as a json file that contains download links and any number of frontends could use this file type. If nothing else, it lets me search a few places that I would otherwise have to go to a few different web pages to search, so that is nice.
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
# libman
|
||||
|
||||
_libman_ is a way to treat book sources the way a package manager
|
||||
would, to some degree.
|
||||
|
||||
## functions
|
||||
|
||||
1. add a source
|
||||
- sources are a single url with a json (?) file of the books it offers, plus other metadata
|
||||
2. delete a source
|
||||
3. sync
|
||||
- Will check for an update file for each source available to the system, will then provide
|
||||
updated info
|
||||
4. search
|
||||
- Searches for a given book via keyword(s), scope of search can be constrained to title,
|
||||
author, description. It can also be constrained by what source(s) to search, defaulting
|
||||
to "all"
|
||||
5. Library
|
||||
- View the current library
|
||||
- Load files in external viewers
|
||||
|
||||
## source format
|
||||
|
||||
```json
|
||||
{
|
||||
name: "", // the source name as it will appear to the user
|
||||
lastUpdate: "2024/03/16 16:23",
|
||||
documents: [
|
||||
{
|
||||
title: "My Book",
|
||||
files: [
|
||||
{
|
||||
url: "",
|
||||
format: "epub"
|
||||
},
|
||||
{
|
||||
url: "",
|
||||
format: "kepub"
|
||||
]
|
||||
author: "Jane Doe", // optional
|
||||
subjects: ["fiction", "adventure", "fantasy"], // optional
|
||||
description: "", // optional
|
||||
lastUpdate: "00/00/0000 00:00x", // optional
|
||||
license: "CC-BY-ND" // optional
|
||||
},
|
||||
{
|
||||
...
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## settings format
|
||||
|
||||
```json
|
||||
{
|
||||
downloadFolder: "~/books",
|
||||
viewer: {
|
||||
"epub": "/usr/local/bin/epr",
|
||||
"gpub": "~/bin/gpr"
|
||||
},
|
||||
cacheFolder: "~/.cache/libman"
|
||||
sources: {
|
||||
"source 1": "https://somesite.com/books/somesite.json"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -0,0 +1,151 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/user"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func ExpandedAbsFilepath(p string) string {
|
||||
if strings.HasPrefix(p, "~") {
|
||||
if p == "~" || strings.HasPrefix(p, "~/") {
|
||||
homedir, _ := os.UserHomeDir()
|
||||
if len(p) <= 2 {
|
||||
p = homedir
|
||||
} else if len(p) > 2 {
|
||||
p = filepath.Join(homedir, p[2:])
|
||||
}
|
||||
} else {
|
||||
i := strings.IndexRune(p, '/')
|
||||
var u string
|
||||
var remainder string
|
||||
if i < 0 {
|
||||
u = p[1:]
|
||||
remainder = ""
|
||||
} else {
|
||||
u = p[1:i]
|
||||
remainder = p[i:]
|
||||
}
|
||||
usr, err := user.Lookup(u)
|
||||
if err != nil {
|
||||
p = filepath.Join("/home", u, remainder)
|
||||
} else {
|
||||
p = filepath.Join(usr.HomeDir, remainder)
|
||||
}
|
||||
}
|
||||
} else if !strings.HasPrefix(p, "/") {
|
||||
wd, _ := os.Getwd()
|
||||
p = filepath.Join(wd, p)
|
||||
}
|
||||
|
||||
path, _ := filepath.Abs(p)
|
||||
return path
|
||||
}
|
||||
|
||||
func sourceNameToCachePath(name string) string {
|
||||
fn := strings.ToLower(name)
|
||||
fn = strings.TrimSpace(fn)
|
||||
fn = strings.ReplaceAll(fn, "/", "_")
|
||||
fn = strings.ReplaceAll(fn, " ", "-")
|
||||
fn = fn + ".json"
|
||||
return filepath.Join(CurrentSettings.CacheFolder, fn)
|
||||
}
|
||||
|
||||
func writeSourceFile(name string, data []byte) error {
|
||||
f, err := os.Create(sourceNameToCachePath(name))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
f.Write(data)
|
||||
return nil
|
||||
}
|
||||
|
||||
func escapeName(s string) string {
|
||||
s = strings.ReplaceAll(s, " ", "-")
|
||||
s = strings.ReplaceAll(s, "_", "-")
|
||||
s = strings.ReplaceAll(s, "/", "")
|
||||
s = strings.ReplaceAll(s, "\\", "")
|
||||
s = strings.ReplaceAll(s, "'", "")
|
||||
return s
|
||||
}
|
||||
|
||||
func makeFileName(d doc, format string) string {
|
||||
if format == "kepub" {
|
||||
format = format + ".epub"
|
||||
}
|
||||
return fmt.Sprintf(
|
||||
"%s_%s.%s",
|
||||
escapeName(d.Title),
|
||||
escapeName(d.Author),
|
||||
format,
|
||||
)
|
||||
}
|
||||
|
||||
func getUrl(u string) ([]byte, error) {
|
||||
r, err := http.Get(u)
|
||||
if err != nil {
|
||||
return []byte{}, err
|
||||
}
|
||||
body, err := io.ReadAll(r.Body)
|
||||
defer r.Body.Close()
|
||||
if err != nil || r.StatusCode > 299 {
|
||||
return body, fmt.Errorf("Invalid server response for %s\n", u)
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
|
||||
func writeDownload(data []byte, d doc, format string) error {
|
||||
fn := makeFileName(d, format)
|
||||
fmt.Printf("GET %s\n", fn)
|
||||
f, err := os.Create(filepath.Join(CurrentSettings.DownloadFolder, fn))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
f.Write(data)
|
||||
return nil
|
||||
}
|
||||
|
||||
func installBooks(d doc, format string) {
|
||||
urls := d.GetFileUrl(format)
|
||||
for i := range urls {
|
||||
data, err := getUrl(urls[i])
|
||||
if err != nil {
|
||||
fmt.Fprint(os.Stderr, err)
|
||||
continue
|
||||
}
|
||||
err = writeDownload(data, d, format)
|
||||
if err != nil {
|
||||
fmt.Fprint(os.Stderr, err)
|
||||
continue
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func similarSource(src string) string {
|
||||
for k := range CurrentSettings.Sources {
|
||||
if strings.Contains(strings.ToLower(k), strings.ToLower(src)) {
|
||||
return k
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func printSubjects() {
|
||||
l := make(map[string]bool)
|
||||
for i := range sourceList {
|
||||
l = sourceList[i].Subjects(l)
|
||||
}
|
||||
keys := make([]string, 0, len(l))
|
||||
for k := range l {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
fmt.Println(strings.Join(keys, "\n"))
|
||||
}
|
||||
|
|
@ -0,0 +1,144 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func editConfig(p string) error {
|
||||
editor := os.Getenv("EDITOR")
|
||||
var err error
|
||||
if editor == "" {
|
||||
editor, err = exec.LookPath("vi")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
cmd := exec.Command(editor, p)
|
||||
cmd.Stdin = os.Stdin
|
||||
cmd.Stdout = os.Stdout
|
||||
cmd.Stderr = os.Stderr
|
||||
return cmd.Run()
|
||||
}
|
||||
|
||||
func main() {
|
||||
addSource := flag.String("add", "", "Add a URL or filepath as a source location")
|
||||
removeSource := flag.String("remove", "", "Remove a source by name")
|
||||
listSources := flag.Bool("sources", false, "List sources")
|
||||
filterSource := flag.String("source", "", "Filter results by the given source")
|
||||
longList := flag.Bool("l", false, "Show full details for each searched book")
|
||||
syncAll := flag.Bool("sync", false, "Sync sources")
|
||||
filterAuthor := flag.String("author", "", "Filter by author")
|
||||
filterSubject := flag.String("subject", "", "Filter by subject")
|
||||
listSubjects := flag.Bool("subjects", false, "List all subjects in cache")
|
||||
filterTitle := flag.String("title", "", "Filter by title")
|
||||
filterFormat := flag.String("format", "", "Filter by format; ex. \"epub\"")
|
||||
installResults := flag.Bool("install", false, "Installs the results of the query")
|
||||
configure := flag.Bool("configure", false, "Open the configuration file in your $EDITOR")
|
||||
flag.Parse()
|
||||
var err error
|
||||
settingsPath = ExpandedAbsFilepath(`~/.config/libman/settings.json`)
|
||||
err = loadSettings()
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
if *configure {
|
||||
err = editConfig(settingsPath)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if *removeSource != "" {
|
||||
err = CurrentSettings.RemoveSource(*removeSource)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "! %s\n", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
if *addSource != "" {
|
||||
err = CurrentSettings.AddSource(*addSource)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "! %s\n", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
if *listSources {
|
||||
fmt.Printf("\033[1mCurrent Sources\033[0m:\n%s", CurrentSettings.ListSources())
|
||||
}
|
||||
|
||||
if *addSource != "" || *removeSource != "" || *listSources {
|
||||
return
|
||||
}
|
||||
|
||||
if *syncAll {
|
||||
syncSources()
|
||||
return
|
||||
}
|
||||
|
||||
sourceList = loadSources(*filterSource)
|
||||
|
||||
if *listSubjects {
|
||||
printSubjects()
|
||||
return
|
||||
}
|
||||
|
||||
if *filterAuthor == "" && *filterTitle == "" && *filterFormat == "" && *filterSubject == "" {
|
||||
fmt.Println("Full catalog listing is ill advised. Please provide a filter. See: `libman -h`")
|
||||
return
|
||||
}
|
||||
|
||||
if *filterFormat == "" && *installResults {
|
||||
var confirm rune
|
||||
fmt.Print("Are you sure you want to install without filtering for format (this will install all formats of each book)? [y/n] ")
|
||||
_, err := fmt.Scanf("%c", &confirm)
|
||||
if err != nil || (confirm != 'y' && confirm != 'Y') {
|
||||
fmt.Fprint(os.Stderr, "Aborting install\n")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
out := make([]string, 0, 20)
|
||||
|
||||
for _, s := range sourceList {
|
||||
for _, d := range s.Documents {
|
||||
if *filterAuthor != "" && !d.MatchTerm(*filterAuthor, authorEnum) {
|
||||
continue
|
||||
}
|
||||
if *filterTitle != "" && !d.MatchTerm(*filterTitle, titleEnum) {
|
||||
continue
|
||||
}
|
||||
if *filterFormat != "" && !d.HasFileType(*filterFormat) {
|
||||
continue
|
||||
}
|
||||
if *filterSubject != "" && !d.HasSubject(*filterSubject) {
|
||||
continue
|
||||
}
|
||||
if *installResults {
|
||||
installBooks(d, *filterFormat)
|
||||
} else if *longList {
|
||||
out = append(out, d.StringLong(s.Name))
|
||||
} else {
|
||||
out = append(out, d.String(s.Name))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if *installResults {
|
||||
return
|
||||
}
|
||||
|
||||
sort.Strings(out)
|
||||
|
||||
if *longList {
|
||||
fmt.Println(strings.TrimSpace(strings.Join(out, "\n\n")))
|
||||
} else {
|
||||
fmt.Println(strings.TrimSpace(strings.Join(out, "\n")))
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -0,0 +1,143 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type Settings struct {
|
||||
DownloadFolder string
|
||||
CacheFolder string
|
||||
Viewer map[string]string
|
||||
Sources map[string]string
|
||||
}
|
||||
|
||||
var settingsPath string
|
||||
|
||||
var CurrentSettings Settings
|
||||
|
||||
func (s Settings) StoreSettings() error {
|
||||
b, err := json.MarshalIndent(CurrentSettings, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
f, err := os.Create(settingsPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
f.Write(b)
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *Settings) UpdateSource(name, u string) {
|
||||
s.Sources[name] = u
|
||||
s.StoreSettings()
|
||||
}
|
||||
|
||||
func (s *Settings) ListSources() string {
|
||||
var o strings.Builder
|
||||
for k, v := range s.Sources {
|
||||
o.WriteString("\033[4m")
|
||||
o.WriteString(k)
|
||||
o.WriteString("\033[0m: ")
|
||||
o.WriteString(v)
|
||||
o.WriteString("\n")
|
||||
}
|
||||
if len(s.Sources) == 0 {
|
||||
return "\033[3mNone\033[0m\n"
|
||||
}
|
||||
return o.String()
|
||||
}
|
||||
|
||||
func (s *Settings) AddSource(u string) error {
|
||||
var body []byte
|
||||
var err error
|
||||
if !strings.Contains(u, "://") {
|
||||
body, err = os.ReadFile(ExpandedAbsFilepath(u))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
u = strings.TrimSpace(u)
|
||||
r, err := http.Get(u)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body, err = io.ReadAll(r.Body)
|
||||
r.Body.Close()
|
||||
if err != nil || r.StatusCode > 299 {
|
||||
return err
|
||||
}
|
||||
}
|
||||
var sd sourceData
|
||||
err = json.Unmarshal(body, &sd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
sourceList[sd.Name] = sd
|
||||
s.Sources[sd.Name] = u
|
||||
s.StoreSettings()
|
||||
return writeSourceFile(sd.Name, body)
|
||||
}
|
||||
|
||||
func (s *Settings) RemoveSource(name string) error {
|
||||
if _, ok := s.Sources[name]; ok {
|
||||
delete(s.Sources, name)
|
||||
os.Remove(sourceNameToCachePath(name))
|
||||
s.StoreSettings()
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("Could not find a source named %q", name)
|
||||
}
|
||||
|
||||
func createFolders(s, c, d string) error {
|
||||
err := os.MkdirAll(s, 0755)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = os.MkdirAll(c, 0755)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = os.MkdirAll(d, 0755)
|
||||
return err
|
||||
}
|
||||
|
||||
func loadSettings() error {
|
||||
CurrentSettings = Settings{
|
||||
ExpandedAbsFilepath("~/.local/share/libman"),
|
||||
ExpandedAbsFilepath("~/.cache/libman"),
|
||||
make(map[string]string),
|
||||
make(map[string]string),
|
||||
}
|
||||
|
||||
if _, err := os.Stat(settingsPath); errors.Is(err, os.ErrNotExist) {
|
||||
err = createFolders(filepath.Dir(settingsPath), CurrentSettings.CacheFolder, CurrentSettings.DownloadFolder)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = CurrentSettings.StoreSettings()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
b, err := os.ReadFile(settingsPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = json.Unmarshal(b, &CurrentSettings)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
|
@ -0,0 +1,192 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
authorEnum int = iota
|
||||
titleEnum
|
||||
)
|
||||
|
||||
type fileSource struct {
|
||||
Url string
|
||||
Format string
|
||||
}
|
||||
|
||||
func (fs fileSource) String() string {
|
||||
return strings.ToLower(fs.Format)
|
||||
}
|
||||
|
||||
type doc struct {
|
||||
Title string
|
||||
Files []fileSource
|
||||
Author string
|
||||
Subjects []string
|
||||
Description string
|
||||
LastUpdate string
|
||||
License string
|
||||
}
|
||||
|
||||
func (d doc) GetFileUrl(format string) []string {
|
||||
format = strings.ToLower(format)
|
||||
out := make([]string, 0, len(d.Files))
|
||||
for _, f := range d.Files {
|
||||
if format == "" || format == f.Format {
|
||||
out = append(out, f.Url)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (d doc) HasFileType(s string) bool {
|
||||
s = strings.ToLower(s)
|
||||
for _, f := range d.Files {
|
||||
if strings.ToLower(f.Format) == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (d doc) HasSubject(s string) bool {
|
||||
s = strings.ToLower(s)
|
||||
for _, sub := range d.Subjects {
|
||||
if strings.Contains(strings.ToLower(sub), s) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (d doc) MatchTerm(term string, kind int) bool {
|
||||
term = strings.ToLower(term)
|
||||
switch kind {
|
||||
case authorEnum:
|
||||
return strings.Contains(strings.ToLower(d.Author), term)
|
||||
case titleEnum:
|
||||
return strings.Contains(strings.ToLower(d.Title), term)
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (d doc) FileTypes() string {
|
||||
var o strings.Builder
|
||||
for i, f := range d.Files {
|
||||
o.WriteString(f.String())
|
||||
if i != len(d.Files)-1 {
|
||||
o.WriteString(", ")
|
||||
}
|
||||
}
|
||||
return o.String()
|
||||
}
|
||||
|
||||
func (d doc) FilesString() string {
|
||||
var o strings.Builder
|
||||
for _, f := range d.Files {
|
||||
o.WriteString(" ")
|
||||
o.WriteString(fmt.Sprintf("%-5s", f.String()))
|
||||
o.WriteString(" - ")
|
||||
o.WriteString(f.Url)
|
||||
o.WriteRune('\n')
|
||||
}
|
||||
return o.String()
|
||||
}
|
||||
|
||||
func (d doc) String(s string) string {
|
||||
return fmt.Sprintf("\033[3m%-35s\033[0m \033[2mby\033[0m %-20s \033[2m(\033[0m%s\033[2m) %s\033[0m", d.Title, d.Author, d.FileTypes(), s)
|
||||
}
|
||||
|
||||
func (d doc) StringLong(s string) string {
|
||||
return fmt.Sprintf("\033[7;1m%s\033[0m \033[2mby\033[0m %s\nSource: %s\nSubjects: %s\nFiles:\n%sDescription:\n %s\n\n", d.Title, d.Author, s, strings.Join(d.Subjects, ", "), d.FilesString(), strings.TrimSpace(strings.ReplaceAll(d.Description, "\\n", "\n")))
|
||||
}
|
||||
|
||||
type sourceData struct {
|
||||
Name string
|
||||
LastUpdate string
|
||||
Url string
|
||||
Documents []doc
|
||||
}
|
||||
|
||||
func (sd sourceData) Subjects(s map[string]bool) map[string]bool {
|
||||
for _, d := range sd.Documents {
|
||||
for _, subs := range d.Subjects {
|
||||
s[strings.ToLower(subs)] = true
|
||||
}
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
var sourceList = make(map[string]sourceData)
|
||||
|
||||
func loadSources(src string) map[string]sourceData {
|
||||
out := make(map[string]sourceData)
|
||||
var sourceFiles []string
|
||||
if src == "" {
|
||||
sourceFiles, _ = filepath.Glob(filepath.Join(CurrentSettings.CacheFolder, "*"))
|
||||
} else {
|
||||
searchSource := similarSource(src)
|
||||
if searchSource == "" {
|
||||
fmt.Fprintf(os.Stderr, "Could not read source %q\n", src)
|
||||
return out
|
||||
}
|
||||
sourceFiles = []string{sourceNameToCachePath(searchSource)}
|
||||
}
|
||||
for _, p := range sourceFiles {
|
||||
b, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Could not read source %s\n", p)
|
||||
continue
|
||||
}
|
||||
var s sourceData
|
||||
err = json.Unmarshal(b, &s)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Could not unmarshal json for %q\n", p)
|
||||
continue
|
||||
}
|
||||
out[s.Name] = s
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func syncSources() {
|
||||
fmt.Println("\033[1mSyncing Sources\033[0m")
|
||||
for name, u := range CurrentSettings.Sources {
|
||||
fmt.Printf("Retrieving: %s...\n", name)
|
||||
r, err := http.Get(u)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "! Could not GET %s, aborting sync for %s\n", u, name)
|
||||
continue
|
||||
}
|
||||
body, err := io.ReadAll(r.Body)
|
||||
r.Body.Close()
|
||||
if err != nil || r.StatusCode > 299 {
|
||||
fmt.Fprintf(os.Stderr, "! Invalid server response for %s, aborting sync for %s\n", u, name)
|
||||
continue
|
||||
}
|
||||
var sd sourceData
|
||||
err = json.Unmarshal(body, &sd)
|
||||
if err != nil || sd.Name == "" {
|
||||
fmt.Fprintf(os.Stderr, "! Corrupt source file for %s, aborting sync for %s\n", u, name)
|
||||
continue
|
||||
}
|
||||
if oldSd, ok := sourceList[name]; ok {
|
||||
if sd.LastUpdate == oldSd.LastUpdate {
|
||||
continue
|
||||
}
|
||||
}
|
||||
err = writeSourceFile(sd.Name, body)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "! Could not write source file, aborting sync for %s\n", name)
|
||||
continue
|
||||
}
|
||||
}
|
||||
sourceList = loadSources("")
|
||||
}
|
||||
|
||||
Loading…
Reference in New Issue