2019-10-10 00:32:54 +00:00
2019-09-22 02:30:03 +00:00
# Pigeon Ruby
2020-04-26 02:08:11 +00:00
A [Pigeon Protocol ](https://tildegit.org/PigeonProtocolConsortium/protocol_spec ) client written in Ruby.
# Features
* CLI and API available (link to quick start for both here)
* Minimal dependencies - only outside deps are `thor` (for CLI) and `ed25519` (for signatures).
* Thoroughly unit tested.
2019-09-22 02:30:03 +00:00
2020-04-19 21:27:09 +00:00
# Caveats
2020-04-26 02:08:11 +00:00
* Current windows support is unknown (and unlikely to work in current state). Please report bugs.
2020-04-19 21:27:09 +00:00
* Not published to RubyGems yet (see installation instructions below)
2020-04-26 02:08:11 +00:00
* Single threaded use is assumed and is intended for a single user per OS process. It makes many design tradeoffs around that use case.
2020-04-26 13:51:14 +00:00
* Bundling operations need performance tuning. Optimizations are planned and help is welcome.
2020-04-19 21:27:09 +00:00
2020-04-26 02:08:11 +00:00
# Build From Source
2019-09-22 02:30:03 +00:00
2020-04-18 18:18:00 +00:00
We are not yet on Rubygems. The gem will be released after we are fully compliant with the spec and have high test coverage stats.
2019-09-22 02:30:03 +00:00
2020-04-18 16:05:19 +00:00
In the meantime:
```
git clone https://tildegit.org/PigeonProtocolConsortium/pigeon_ruby.git
cd pigeon_ruby
2020-04-18 18:18:00 +00:00
gem build pigeon.gemspec
2020-04-26 02:08:11 +00:00
gem install pigeon-0.1.1.gem
2020-04-18 18:18:00 +00:00
pigeon-cli identity new # Should work. Raise issue if not.
pigeon-cli status
pigeon-cli help
2020-04-18 16:05:19 +00:00
```
2020-04-18 15:51:08 +00:00
# Usage: CLI
2020-04-18 18:18:00 +00:00
See `pigeon-cli help` for documentation.
See `kitchen_sink.sh` examples.
2019-09-22 02:30:03 +00:00
2020-04-18 15:51:08 +00:00
# Usage: Ruby Lib
2020-04-18 16:05:19 +00:00
TODO
2019-09-22 02:30:03 +00:00
# Current Status
2020-04-26 02:08:11 +00:00
- [ ] BUG: Keys that start with a carriage return (`\n`) freeze tokenizer.
- [ ] Update README.md
- [ ] Write tutorial.rb (user manual for `Pigeon::Database` ).
- [ ] Convert literals to constants, remove unused locals, reduce duplication, run linter.
- [ ] 100% class / module documentation
2020-04-22 13:21:55 +00:00
- [ ] Update spec document CLI usage examples to reflect API changes in 2020.
- [ ] Publish to RubyGems
# Optimizations
2020-04-26 02:08:11 +00:00
2020-03-15 18:12:26 +00:00
- [ ] add parsers and validators for all CLI inputs
2020-04-22 13:21:55 +00:00
- [ ] Make the switch to LevelDB, RocksDB, [UNQLite ](https://unqlite.org/features.html ) or similar (currently using Ruby PStore).
2020-04-06 12:52:32 +00:00
- [ ] Reduce whole darn repo into single module to aide portability. `::Helpers` module is OK.
- [ ] Update the bundles.md document once `bundle consume` works.
2020-04-03 10:35:48 +00:00
- [ ] Performance benchmarks (Do this second to last!)
- [ ] Performance tuning (Do this last!)
2020-04-08 13:54:28 +00:00
2020-04-26 02:08:11 +00:00
# New Features / Road Map
2020-04-08 13:54:28 +00:00
2020-04-26 02:08:11 +00:00
- [ ] Support partial verification via `lipmaa` property.
- [ ] Add `--since=` /`--until=` args to `bundle create` for sending partial / "slice" bundles.
2020-04-20 02:32:37 +00:00
- [ ] Interest and Disinterest Signalling for document routing: Create a `$blob_status` message to express `have` , `want` signalling. This can steer bundle creation and an eventual `--for` flag at bundle creation time to customize a bundle to a particular user.
- [ ] Add a schema for `$peer_status` . Eg: `block` , `unblock` , `follow` , `unfollow` .
2020-03-08 16:18:05 +00:00
# Idea Bin
2020-04-18 15:51:08 +00:00
2020-04-26 13:51:14 +00:00
- [ ] Ability to add map/reduce plugins to support custom indices?
2020-03-08 16:18:05 +00:00
- [ ] Ability to add a blob in one swoop using File objects and `Message#[]=` , maybe?
2020-04-20 13:37:08 +00:00
- [ ] Bundling via [Optar ](http://ronja.twibright.com/optar/ ) or [Colorsafe ](https://github.com/colorsafe/colorsafe )
2020-04-04 22:35:07 +00:00
2020-04-05 20:31:44 +00:00
# New Bundle Format
2020-04-04 22:35:07 +00:00
2020-04-05 20:31:44 +00:00
We have a bundle format that works, but it only exports messages.
2020-04-04 22:35:07 +00:00
2020-04-05 20:31:44 +00:00
We need a bundle format that may optionally include blobs as well.
2020-04-04 22:35:07 +00:00
2020-04-05 20:31:44 +00:00
Here's how we will support that:
2020-04-04 22:35:07 +00:00
1. Create a `bundle_X/` directory. The name is arbitrary and can be defined by the user.
2. In the root directory of `bundle_x/` , a single `messages.pgn` file contains all messages.
* All messages are expected to be sorted by depth
2020-04-05 20:31:44 +00:00
* Messages from multiple authors may be included in a single bundle, but the messages must appear in the correct order with regards to the `depth` field.
2020-04-04 22:35:07 +00:00
3. Blobs are stored in a very specific hierarchy to maintain FAT compatibility:
2020-04-22 13:21:55 +00:00
* `blobs/bundle/7Z2CSZK/MB1RE5G/6SKXRZ6/3ZGCNP8/VVEM3K0/XFMYKET/RDQSM5W.BSG`
2020-04-04 22:35:07 +00:00
2020-04-05 20:31:44 +00:00
Additional notes:
2020-04-04 22:35:07 +00:00
2020-04-06 12:18:04 +00:00
* It is recommended to compress bundles (ex: *.zip files) but these concerns are not handled by the protocol currently.
# Unanswered Questions
* PEER MESSAGES: I want to add a `--depth` option to bundle exports that would only return messages after the `nth` sequence number. It would not make sense to apply `--depth` to all peer messages in the bundle. It would not be practical to expect the user to provide a `--depth` for every peer every time a bundle is generated.
2020-04-19 16:16:04 +00:00
* Create a new `received_on` index that records the local user's `depth` at the time of ingestion?
