Merge branch 'develop' into fix-path-issues

README.md

@@ -9,9 +9,11 @@ These instructions will get a copy of the project up and running on your local m
 
 ### Prerequisites
 
-If building from source, you will need to have [Go](https://golang.org/) version >= 1.12.
+If building from source, you will need to have [Go](https://golang.org/) version >= 1.11.
 
-While Bombadillo has one external dependency of [Mailcap](https://tildegit.org/sloum/mailcap), no action is typically required to download this, as it is handled automatically during the build process.
+#### Optional
+
+[Lynx](https://lynx.invisible-island.net/), the text based web browser, can be used as a parsing engine for http/https. This is a totally optional item and Lynx is in no way required in order to compile or run Bombadillo. Having it available on the system can help create a richer experience by allowing users to navigate directly to web content in Bombadillo. Many users may wish to avoid this entirely, and the default configuration does not have this behavior turned on. To turn it on from within Bombadillo enter the command `set lynxmode true`.
 
 ### Installing
 

bombadillo.1

@@ -1,4 +1,4 @@
-.TH "bombadillo" 1 "12 OCT 2019" "" "General Opperation Manual"
+.TH "bombadillo" 1 "27 OCT 2019" "" "General Operation Manual"
 .SH NAME
 \fBbombadillo \fP- a non-web client
 .SH SYNOPSIS
@@ -28,11 +28,11 @@ Gopher is the default protocol for \fBbombadillo\fP. Any textual item types will
 .TP
 .B
 gemini
-Gemini is supported, but as a new protocol with an incomplete specification, features may change over time. At present Bombadillo supports TLS with a trust on first use certificate pinning system (similar to SSH). Client certificates are also supported via option configuration. Gemini maps and other text types are rendered in the client and non-text types will trigger a prompt to the user to download or open in an appropriate program. 
+Gemini is supported, but as a new protocol with an incomplete specification, features may change over time. At present Bombadillo supports TLS with a trust on first use certificate pinning system (similar to SSH). Client certificates are also supported via option configuration. Gemini maps and other text types are rendered in the client and non-text types will be downloaded.
 .TP
 .B
 finger
-Basic support is provided for the finger protocol. The format is: \fIfinger://[[username@]][hostname]\fP. Many servers still support finger and it can be fun to see if friends are online or read about the users who's phlogs you follow.
+Basic support is provided for the finger protocol. The format is: \fIfinger://[[username@]][hostname]\fP. Many servers still support finger and it can be fun to see if friends are online or read about the users whose phlogs you follow.
 .TP
 .B
 local
@@ -44,7 +44,11 @@ Telnet is not supported directly, but addresses will be followed and opened as a
 .TP
 .B
 http, https
-Neither of the world wide web protocols are supported directly. However, bombadillo is capable of attempting to open web links in a user's default web browser. This feature is opt-in only and controlled in a user's settings. This feature does not function properly in a terminal only environment at present.
+Neither of the world wide web protocols are supported directly. However, \fBbombadillo\fP can open web links in a user's default web browser, or display web content directly in the client via the lynx web browser. Opening http links is opt-in only, controlled by the \fIopenhttp\fP setting.
+.IP
+Opening links in a default web browser only works if a GUI environment is available.
+.IP
+Opening web content directly in the client requires the lynx web browser, and is enabled using the \fIlynxmode\fP setting. Web content is processed using lynx, and then displayed in the client.
 .SH COMMANDS
 .SS  KEY COMMANDS
 These commands work as a single keypress anytime \fBbombadillo\fP is not taking in a line based command. This is the default command mode of \fBbombadillo\fP.
@@ -216,8 +220,13 @@ homeurl
 The url that \fBbombadillo\fP navigates to when the program loads or when the \fIhome\fP or \fIh\fP LINE COMMAND is issued. This should be a valid url. If a scheme/protocol is not included, gopher will be assumed.
 .TP
 .B
+lynxmode
+Will use lynx as a rendering engine for http/https requests if lynx is installed and \fIopenhttp\fP is set to \fItrue\fP. Valid values are \fItrue\fP and \fIfalse\fP.
+.TP
+.B
 openhttp
-Tells the client whether or not to try to follow web (http/https) links. If set to \fItrue\fP, \fBbombadillo\fP will try to open a user's default web browser to the link in question. Any value other than \fItrue\fP is considered false.
+Tells the client whether or not to try to follow web (http/https) links. If set to \fItrue\fP, \fBbombadillo\fP will try to open a user's default web browser to the link in question. Valid values are \fItrue\fP and \fIfalse\fP.
+
 .TP
 .B
 savelocation
@@ -233,11 +242,11 @@ Tells the client what command to use to start a telnet session. Should be a vali
 .TP
 .B
 terminalonly
-Sets whether or not to try to open non-text files served via gemini in gui programs or not. If set to \fItrue\fP, bombdaillo will only attempt to use terminal programs to open files. If set to anything else, \fBbombadillo\fP may choose from the appropriate programs installed on the system, if one is present.
+Sets whether or not to try to open non-text files served via gemini in GUI programs or not. If set to \fItrue\fP \fBbombadillo\fP will only attempt to use terminal programs to open files. If set to \fIfalse\fP \fBbombadillo\fP may choose from the appropriate programs installed on the system, including graphical ones.
 .TP
 .B
 theme
-Can toggle between visual modes. Valid values are \fInormal\fP and \fIinverse\fP. When set to ivnerse, the terminal color mode is inversed.
+Can toggle between visual modes. Valid values are \fInormal\fP and \fIinverse\fP. When set to inverse, the terminal color mode is inverted.
 .TP
 .B
 tlscertificate
@@ -245,7 +254,7 @@ A path to a tls certificate file on a user's local filesystem. Defaults to NULL.
 .TP
 .B
 tlskey
-A path the a tls key that pairs with the tlscertificate setting, on a user's local filesystem. Defaults to NULL. Both \fItlskey\fP and \fItlscertificate\fP must be set for client certificates to work in gemini.
+A path to a tls key that pairs with the tlscertificate setting, on a user's local filesystem. Defaults to NULL. Both \fItlskey\fP and \fItlscertificate\fP must be set for client certificates to work in gemini.
 .SH BUGS
 There are very likely bugs. Many known bugs can be found in the issues section of \fBbombadillo\fP's software repository (see \fIlinks\fP).
 .SH LINKS

defaults.go

@@ -21,12 +21,12 @@ var defaultOptions = map[string]string{
 	"savelocation": userinfo.HomeDir,
 	"searchengine": "gopher://gopher.floodgap.com:70/7/v2/vs",
 	"openhttp":     "false",
-	"httpbrowser":  "",
 	"telnetcommand": "telnet",
 	"configlocation": userinfo.HomeDir,
 	"theme": "normal", // "normal", "inverted"
 	"terminalonly": "true",
 	"tlscertificate": "",
 	"tlskey": "",
+	"lynxmode": "false",
 }
 

go.mod

@@ -1,5 +1,3 @@
 module tildegit.org/sloum/bombadillo
 
-go 1.12
-
-require tildegit.org/sloum/mailcap v0.0.0-20190706214029-b787a49e9db2
+go 1.11

go.sum

@@ -1,2 +0,0 @@
-tildegit.org/sloum/mailcap v0.0.0-20190706214029-b787a49e9db2 h1:tAPIFBpXwOq1Ytxk8aGsDjCutnwUC01BVkK77QS1bdU=
-tildegit.org/sloum/mailcap v0.0.0-20190706214029-b787a49e9db2/go.mod h1:m4etAw9DbXsdanDUNS8oERhL+7y4II82ZLHWzw2yibg=

http/lynx_mode.go

@@ -0,0 +1,85 @@
+package http
+
+import (
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os/exec"
+	"strings"
+)
+
+type page struct {
+	Content string
+	Links   []string
+}
+
+func Visit(url string, width int) (page, error) {
+	if width > 80 {
+		width = 80
+	}
+	w := fmt.Sprintf("-width=%d", width)
+	c, err := exec.Command("lynx", "-dump", w, url).Output()
+	if err != nil {
+		return page{}, err
+	}
+	return parseLinks(string(c)), nil
+}
+
+// Returns false on err or non-text type
+// Else returns true
+func IsTextFile(url string) bool {
+	c, err := exec.Command("lynx", "-dump", "-head", url).Output()
+	if err != nil {
+		return false
+	}
+	content := string(c)
+	content = strings.ToLower(content)
+	headers := strings.Split(content, "\n")
+	for _, header := range headers {
+		if strings.Contains(header, "content-type:") && strings.Contains(header, "text") {
+			return true
+		}
+	}
+	return false
+}
+
+func parseLinks(c string) page {
+	var out page
+	contentUntil := strings.LastIndex(c, "References")
+	if contentUntil >= 1 {
+		out.Content = c[:contentUntil]
+	} else {
+		out.Content = c
+		out.Links = make([]string, 0)
+		return out
+	}
+	links := c[contentUntil+11:]
+	links = strings.TrimSpace(links)
+	linkSlice := strings.Split(links, "\n")
+	out.Links = make([]string, 0, len(linkSlice))
+	for _, link := range linkSlice {
+		ls := strings.SplitN(link, ".", 2)
+		if len(ls) < 2 {
+			continue
+		}
+		out.Links = append(out.Links, strings.TrimSpace(ls[1]))
+	}
+	return out
+
+}
+
+func Fetch(url string) ([]byte, error) {
+	resp, err := http.Get(url)
+	if err != nil {
+		return []byte{}, err
+	}
+	defer resp.Body.Close()
+
+	bodyBytes, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return []byte{}, err
+	}
+
+	return bodyBytes, nil
+}
+

main.go

@@ -1,6 +1,7 @@
 package main
 
-// Bombadillo is a gopher and gemini client for the terminal of unix or unix-like systems.
+// Bombadillo is an internet client for the terminal of unix or
+// unix-like systems.
 //
 // Copyright (C) 2019 Brian Evans
 //
@@ -30,7 +31,6 @@ import (
 	"tildegit.org/sloum/bombadillo/config"
 	"tildegit.org/sloum/bombadillo/cui"
 	_ "tildegit.org/sloum/bombadillo/gemini"
-	"tildegit.org/sloum/mailcap"
 )
 
 var version string
@@ -39,7 +39,6 @@ var build string
 var bombadillo *client
 var helplocation string = "gopher://bombadillo.colorfield.space:70/1/user-guide.map"
 var settings config.Config
-var mc *mailcap.Mailcap
 
 func saveConfig() error {
 	var opts strings.Builder
@@ -70,6 +69,7 @@ func validateOpt(opt, val string) bool {
 		"openhttp":     []string{"true", "false"},
 		"theme":        []string{"normal", "inverse"},
 		"terminalonly": []string{"true", "false"},
+		"lynxmode":     []string{"true", "false"},
 	}
 
 	opt = strings.ToLower(opt)
@@ -193,10 +193,6 @@ func main() {
 	}
 	args := flag.Args()
 
-	// Build the mailcap db
-	// So that we can open files from gemini
-	mc = mailcap.NewMailcap()
-
 	cui.InitTerm()
 	defer cui.Exit()
 	err := initClient()