README updates

This commit is contained in:
Netscape Navigator 2019-10-22 20:45:58 -05:00
parent 1f09fe6c07
commit 643062253b
1 changed files with 57 additions and 35 deletions

View File

@ -4,15 +4,34 @@
A synchronizing peer-to-peer messaging protocol that is:
* replicated among peers
* decentralized (peer-to-peer)
* replicated
* tamper-resistant
* delay tolerant (assumes sneakernet is the default transport layer)
* delay tolerant
* built for [sneakernet](https://en.wikipedia.org/wiki/Sneakernet) from the ground up
The document below describes a protocol as it _should be_ rather than as it is. This document does not describe a working protocol. It is a planning document for a protocol and the first software packages that will implement the protocol.
# Why?
Pigeon can serve a number of use cases. Below are some examples:
* Systems with low connectivity or uptime (remote sensor logging, maritime systems, solar systems with intermittent power, IoT systems with poor network connectivity)
* Store-and-forward message gateways, such as a [data mule](https://en.wikipedia.org/wiki/Data_mule).
* Censorship resistant communications
* [Delay tolerant networking](https://en.wikipedia.org/wiki/Delay-tolerant_networking)
* Applications that require a high level of systems
* delay-tolerant peer-to-peer social networks, games, file sharing etc...
* Time series data storage
* Communications software
# What Does It Do?
Each node in a swarm of peers has a local "log". The log is an append only feed of messages written in an ASCII-based serialization format. Messages are signed with a secret key to validate a message's integrity and to prevent tampering by untrusty peers. Nodes in the swarm "follow" other logs from peers of interest. Nodes always replicate the logs of their peers and "gossip" information about peers across the swarm. Gossip information is packaged into "bundles" which contain backups of peer logs in an efficient binary format that can be easily transmitted via sneakernet, direct serial connection, or any other high-latency, high throughput medium.
# How It Works
You synchronize with peers via sneakernet or any other high-delay, high-throughput medium.
Sneakernet is the main use case for Pigeon messages to be transmitted. Transmission of SD Cards via postal mail offer an excellent medium for transmission of Pigeon messages, although any data transfer medium is theoretically possible.
![](sync.png)
@ -22,24 +41,6 @@ This is an exploration of ideas set forth by the Secure Scuttlebutt protocol. It
I've also been inspired by the compactness and minimalism of [SQLite, which should serve as a role model for all of us](https://www.sqlite.org/talks/wroclaw-20090310.pdf).
# What is Possible?
Below are some possible use cases to illustrate real-world applications. Once protocol implementations exist, the ideas below should be possible.
* Sync over sneakernet using common Unix utilities and default features.
* A GUI database browser for developers that wish to use the protocol for log storage or as a time series DB.
* Import and export your data to and from Secure Scuttlebutt.
* Play-by-mail [patchwork clone](https://github.com/ssbc/patchwork)
* Offer developers the opportunity to use the protocol in the language of their choice. One of the languages will be WASM compatible for portability.
* A newsgroup analog.
* A play-by-mail (or bluetooth) file sharing app.
* FireFox log viewer extension, similar to [Patchfox](https://github.com/soapdog/patchfox).
* A turn-based board game.
* A messenger app
* Sync a feed over email (via external tool)
* Sync a feed over bluetooth (via external tool)
* Sync a feed over actual pigeons, possibly soliciting help from world famous boxer and pigeon racing enthusiast Mike Tyson.
# Hard Constraints
* Have [near] zero config. We will allow a limit of 10 configuration options for all eternity. These are simple key/value pairs. No nesting, no namespacing, no dots, no dashes, no nested config names, no arrays, none of that crap. Seriously, I'm watching you.
@ -76,12 +77,27 @@ An **application** is software that uses a Pigeon Protocol implementation, possi
* Assume CPU resources and memory are limited.
* Assume block storage is plentiful (storage space measured in GBs)
# What is Possible?
Below are some possible use cases to illustrate real-world applications. Once protocol implementations exist, the ideas below should be possible.
* Play-by-mail [patchwork clone](https://github.com/ssbc/patchwork)
* A GUI database browser for developers that wish to use the protocol for log storage or as a time series DB.
* A messenger app
* Secure Scuttlebutt import / export / gateway tool.
* A newsgroup / NNTP analog.
* A play-by-mail (or bluetooth) file sharing app.
* A turn-based board game.
* An IoT data logger
* Sync a feed over email (via external tool)
* Sync a feed over bluetooth (via external tool)
* Sync a feed over actual pigeons, possibly soliciting help from world famous boxer and pigeon racing enthusiast Mike Tyson.
# Unanswered Questions
* Ephemeral key exchange
* Merkle tree vs. hash chain
* Standardized general purpose message schemas (follow, unfollow, redirect, etc.)
# Concepts
* Base64: The protocol will use Base 64 Encoding with URL and Filename Safe Alphabet as specified in [RFC 4648](https://tools.ietf.org/html/rfc4648). The protocol always uses the standard "=" character for padding. Deviations from this will be explicitly noted.
@ -91,7 +107,7 @@ An **application** is software that uses a Pigeon Protocol implementation, possi
* Blob: Arbitrary binary data, with a current max size of 1 MB.
* Blob Hash: A base64'ed SHA256 of a Blob, starting with `&` and ending with `.sha256`.
* Pair: One `string` and on of the following: `Blob Hash|Signature|Identity|String`, joined with a `:` character between the two. See "key" and "value" below.
* Header: A reserved pair of information that is required by the protocol for internal reasons. Not to be confused by a `Pair`, which is user definable. The only Headers the protocol currently uses are `author`, `depth`, `kind`, `prev`. Headers do not have "string quotes" around the key, and the value is delimited by a space (` `) character. Example: `depth 46`.
* Header: A reserved pair of information that is required by the protocol for internal reasons. Not to be confused by a `Pair`, which is user definable. The only Headers the protocol currently uses are `author`, `depth`, `kind`, `prev`. Headers do not have "string quotes" around the key, and the value is delimited by a space (` `) character. Example: `sequence 46`.
* Key: The value to the left of a `:` in a pair. Always a string.
* Value: The value to the right of a `:` in a pair. It is always one of the following: `Blob Hash, Signature, Identity, String`.
* Message: A document with an author, prev, depth, kind, an arbitrary set of attribute pairs and a footer.
@ -106,9 +122,10 @@ An **application** is software that uses a Pigeon Protocol implementation, possi
```
author @ajgdylxeifojlxpbmen3exlnsbx8buspsjh37b/ipvi=.ed25519
depth 23
sequence 23
kind "example"
prev %85738f8f9a7f1b04b5329c590ebcb9e425925c6d0984089c43a022de4f19c281.sha256
previous %85738f8f9a7f1b04b5329c590ebcb9e425925c6d0984089c43a022de4f19c281.sha256
timestamp 23123123123
"foo":&3f79bb7b435b05321651daefd374cdc681dc06faa65e374e38337b88ca046dea.sha256
"baz":"bar"
@ -155,9 +172,10 @@ pigeon message new weather_report
pigeon message current # Show active log entry.
# => author @ajgdylxeifojlxpbmen3exlnsbx8buspsjh37b/ipvi=.ed25519
# => depth 1
# => sequence 1
# => kind weather_report
# => prev %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => previous %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => timestamp 23123123123
# =>
# =>
@ -165,34 +183,38 @@ pigeon blob get 2e7a0bc3 | pigeon message append funy_cat_video
pigeon message save
# => author @ajgdylxeifojlxpbmen3exlnsbx8buspsjh37b/ipvi=.ed25519
# => depth 1
# => sequence 1
# => kind &82244417f956ac7c599f191593f7e441a4fafa20a4158fd52e154f1dc4c8ed92.sha256
# => prev %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => previous %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => timestamp 23123123123
# =>
# => current_mood:&2e7a0bc31f3c4fe6114051c3a56c8ed8a030b3b394df7d29d37648e9b8cbf54b.sha256
# =>
pigeon message find %g0Fs9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => author @ajgdylxeifojlxpbmen3exlnsbx8buspsjh37b/ipvi=.ed25519
# => depth 1
# => sequence 1
# => kind &82244417f956ac7c599f191593f7e441a4fafa20a4158fd52e154f1dc4c8ed92.sha256
# => prev %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => previous %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => timestamp 23123123123
# =>
# => user_profile:&2e7a0bc31f3c4fe6114051c3a56c8ed8a030b3b394df7d29d37648e9b8cbf54b.sha256
# =>
pigeon message find-all --author=@ajgdylxeifojlxpbmen3exlnsbx8buspsjh37b/ipvi=.ed25519 --since=1
# => author @ajgdylxeifojlxpbmen3exlnsbx8buspsjh37b/ipvi=.ed25519
# => depth 1
# => sequence 1
# => kind &82244417f956ac7c599f191593f7e441a4fafa20a4158fd52e154f1dc4c8ed92.sha256
# => prev %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => previous %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => timestamp 23123123123
# =>
# => like:%2e7a0bc31f3c4fe6114051c3a56c8ed8a030b3b394df7d29d37648e9b8cbf54b.ed25519
# =>
# => author @ajgdylxeifojlxpbmen3exlnsbx8buspsjh37b/ipvi=.ed25519
# => depth 2
# => sequence 2
# => kind &82244417f956ac7c599f191593f7e441a4fafa20a4158fd52e154f1dc4c8ed92.sha256
# => prev %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => previous %jvKh9yoiEJaePzoWCF1nnqpIlPgTk9FHEtqczQbvzGM=.sha256
# => timestamp 23123123123
# =>
# => favorite_song:&2e7a0bc31f3c4fe6114051c3a56c8ed8a030b3b394df7d29d37648e9b8cbf54b.sha256
# =>