Update README, specs, kitche sync script
This commit is contained in:
parent
3d3c22019f
commit
f2e9bcc03b
88
README.md
88
README.md
|
|
@ -2,17 +2,22 @@
|
|||
|
||||
# Pigeon Ruby
|
||||
|
||||
This is a WIP [Pigeon Protocol](https://tildegit.org/PigeonProtocolConsortium/protocol_spec) client written in Ruby.
|
||||
A [Pigeon Protocol](https://tildegit.org/PigeonProtocolConsortium/protocol_spec) client written in Ruby.
|
||||
|
||||
# Features
|
||||
|
||||
* CLI and API available (link to quick start for both here)
|
||||
* Minimal dependencies - only outside deps are `thor` (for CLI) and `ed25519` (for signatures).
|
||||
* Thoroughly unit tested.
|
||||
|
||||
# Caveats
|
||||
|
||||
* Probably does not work on Windows, but testers are welcome to try it out. Please let me know how it goes.
|
||||
* Blobs are not included in bundles yet.
|
||||
* Current windows support is unknown (and unlikely to work in current state). Please report bugs.
|
||||
* Not published to RubyGems yet (see installation instructions below)
|
||||
* Not thread safe, never will be. This implementation is intended for a single user and makes many design tradeoffs around that use case.
|
||||
* Bundle works, but is inefficient. Will optimize after proof of concept.
|
||||
* Single threaded use is assumed and is intended for a single user per OS process. It makes many design tradeoffs around that use case.
|
||||
* Bundling operations need performance tuning. Optimizations are planned, help is welcome.
|
||||
|
||||
# Installation
|
||||
# Build From Source
|
||||
|
||||
We are not yet on Rubygems. The gem will be released after we are fully compliant with the spec and have high test coverage stats.
|
||||
|
||||
|
|
@ -22,7 +27,7 @@ In the meantime:
|
|||
git clone https://tildegit.org/PigeonProtocolConsortium/pigeon_ruby.git
|
||||
cd pigeon_ruby
|
||||
gem build pigeon.gemspec
|
||||
gem install pigeon-0.0.7.gem
|
||||
gem install pigeon-0.1.1.gem
|
||||
pigeon-cli identity new # Should work. Raise issue if not.
|
||||
pigeon-cli status
|
||||
pigeon-cli help
|
||||
|
|
@ -39,78 +44,27 @@ TODO
|
|||
|
||||
# Current Status
|
||||
|
||||
- [X] pigeon identity new
|
||||
- [X] pigeon identity show
|
||||
- [X] pigeon status
|
||||
- [X] pigeon blob set
|
||||
- [X] pigeon blob get
|
||||
- [X] pigeon peer add
|
||||
- [X] pigeon peer remove
|
||||
- [X] pigeon peer block
|
||||
- [X] pigeon peer all
|
||||
- [X] 100% coverage
|
||||
- [X] Convert `".sig.ed25519"` literals to constants
|
||||
- [X] Rename numerous "pigeon message ..." commands to "pigeon draft ..."
|
||||
- [X] pigeon draft create
|
||||
- [X] pigeon draft append
|
||||
- [X] pigeon draft current
|
||||
- [X] pigeon draft save
|
||||
- [X] pigeon bundle create
|
||||
- [X] Use JSON.stringify() for string keys (instead of `inspect`)
|
||||
- [X] Move literals into `Pigeon` module as constants, again.
|
||||
- [X] pigeon message find
|
||||
- [X] pigeon message find-all for local feed.
|
||||
- [X] pigeon bundle consume (We are minimally feature complete at this point)
|
||||
- [X] Fix the diagram in the spec document
|
||||
- [X] Validate inputs for `Draft#[]=`.
|
||||
- [X] Put all the [HEADER, string, FOOTER].join("") nonsense into Pigeon::Helpers
|
||||
- [X] Use raw SHA256 hashes for blob multihashes, not hex.
|
||||
- [X] Change all the `{40,90}` values in ::Lexer to real length values
|
||||
- [X] Don't double-ingest messages. It will screw up indexes.
|
||||
- [X] 100% test coverage
|
||||
- [X] Implement pigeon message find-all for peer feed. I will need to add index for `author => message_count`
|
||||
- [X] Switch to Crockford base32- Simplifies support for legacy systems. Easy to implement.
|
||||
- [X] Fix `scratchpad.sh` to use Base32
|
||||
- [X] Rename (RemoteIdentity|LocalIdentity)#public_key to #multihash for consistency with other types.
|
||||
- [X] Fix diagram in spec doc
|
||||
- [X] refactor `Bundle.create` to use `message find-all`.
|
||||
- [X] Rename `message find` to `message read`, since other finders return a multihash.
|
||||
- [X] Message.ingest should be the only code path to message authoring.
|
||||
- [X] Don't allow any type of whitespace in `kind` or `string` keys. Write a test for this.
|
||||
- [X] Run Flog / Flay and friends to find duplications. Will aid in port to other languages.
|
||||
- [X] Make all methods private except those required for the CLI.
|
||||
- [X] Add Lipmaa links like the Bamboo folks do.
|
||||
- [X] Set a max message size.
|
||||
- [X] Clean up all singletons / .current hacks
|
||||
- [X] Reduce cross cutting where collaborating objects need access to `@db`
|
||||
- [X] Ensure all disks writes perform verification!
|
||||
- [X] Make CLI names consistent with API names. Eg: find vs. read.
|
||||
- [X] `find-all` should....find all. Currently finds your messages or maybe peers, but not all.
|
||||
- [X] Add log count to `pigeon-cli status`
|
||||
- [X] Delete `Draft#put` entirely.
|
||||
- [X] Check block list before ingesting bundles.
|
||||
- [X] Need a way of importing / exporting a feeds blobs. (see "Bundle Brainstorming" below)
|
||||
- [X] Need a way of adding peers messages / gossip to bundles. (see "Bundle Brainstorming" below)
|
||||
- [X] Rename `who_am_i` as `get_who_am_i` to follow VERB + NOUN convention.
|
||||
- [ ] Find that non-deterministic runaway loop in the test suite.
|
||||
- [ ] Update README.md / tutorial.rb (user manual for `Pigeon::Database`).
|
||||
- [ ] BUG: Keys that start with a carriage return (`\n`) freeze tokenizer.
|
||||
- [ ] Update README.md
|
||||
- [ ] Write tutorial.rb (user manual for `Pigeon::Database`).
|
||||
- [ ] Convert literals to constants, remove unused locals, reduce duplication, run linter.
|
||||
- [ ] 100% class / module documentation
|
||||
- [ ] Update spec document CLI usage examples to reflect API changes in 2020.
|
||||
- [ ] Publish to RubyGems
|
||||
|
||||
# Optimizations
|
||||
|
||||
- [ ] add parsers and validators for all CLI inputs
|
||||
- [ ] Make the switch to LevelDB, RocksDB, [UNQLite](https://unqlite.org/features.html) or similar (currently using Ruby PStore).
|
||||
- [ ] Convert literals to constants, remove unused locals, reduce duplication, run linter.
|
||||
- [ ] Reduce whole darn repo into single module to aide portability. `::Helpers` module is OK.
|
||||
- [ ] Update the bundles.md document once `bundle consume` works.
|
||||
- [ ] 100% documentation
|
||||
- [ ] Performance benchmarks (Do this second to last!)
|
||||
- [ ] Performance tuning (Do this last!)
|
||||
|
||||
# After v0.0.1
|
||||
# New Features / Road Map
|
||||
|
||||
- [ ] (later, not now) Support partial verification via `lipmaa` property.
|
||||
- [ ] Add mandatory `--since=` arg to `bundle create
|
||||
- [ ] Support partial verification via `lipmaa` property.
|
||||
- [ ] Add `--since=`/`--until=` args to `bundle create` for sending partial / "slice" bundles.
|
||||
- [ ] Interest and Disinterest Signalling for document routing: Create a `$blob_status` message to express `have`, `want` signalling. This can steer bundle creation and an eventual `--for` flag at bundle creation time to customize a bundle to a particular user.
|
||||
- [ ] Add a schema for `$peer_status`. Eg: `block`, `unblock`, `follow`, `unfollow`.
|
||||
|
||||
|
|
|
|||
|
|
@ -82,7 +82,7 @@ echo "=== Dump the bundle"
|
|||
./bin/pigeon-cli bundle create
|
||||
cat bundle/gossip.pgn
|
||||
echo "=== end bundle dump. Ingesting bundle..."
|
||||
./bin/pigeon-cli bundle ingest bundle/gossip.pgn
|
||||
./bin/pigeon-cli bundle ingest bundle
|
||||
|
||||
echo "=== find a message"
|
||||
./bin/pigeon-cli message find $(./bin/pigeon-cli message last)
|
||||
|
|
|
|||
|
|
@ -212,8 +212,6 @@ module Pigeon
|
|||
FileUtils.makedirs(path)
|
||||
else
|
||||
end
|
||||
rescue => x
|
||||
binding.pry
|
||||
end
|
||||
|
||||
def self.write_to_disk(base_path, mhash, data)
|
||||
|
|
|
|||
|
|
@ -155,8 +155,6 @@ module Pigeon
|
|||
[b, c].map do |d|
|
||||
set.add(d) if Helpers.blob_multihash?(d)
|
||||
end
|
||||
end.map do |multihash|
|
||||
binding.pry unless store.have_blob?(multihash)
|
||||
end
|
||||
Pigeon::Parser.parse(self, tokens)
|
||||
end
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
module Pigeon
|
||||
VERSION = "0.1.0"
|
||||
VERSION = "0.1.1"
|
||||
end
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
require "spec_helper"
|
||||
require "timeout"
|
||||
|
||||
RSpec.describe Pigeon::Message do
|
||||
def reset_draft(params)
|
||||
|
|
@ -162,6 +163,19 @@ RSpec.describe Pigeon::Message do
|
|||
end
|
||||
end
|
||||
|
||||
# This was originally a bug nooted during development
|
||||
# That caused a runaway loop in the tokenizer.
|
||||
it "handles this key: '\\nVUx0hC3'" do
|
||||
db.reset_draft
|
||||
db.new_draft(kind: "unit_test")
|
||||
db.update_draft("\nVUx0hC3", "n")
|
||||
db.update_draft("n", "\nVUx0hC3")
|
||||
Timeout::timeout(1) do
|
||||
boom = -> { Pigeon::Lexer.tokenize(db.publish_draft.render) }
|
||||
expect(boom).to raise_error(Pigeon::Lexer::LexError)
|
||||
end
|
||||
end
|
||||
|
||||
it "does not allow whitespace in key names" do
|
||||
WHITESPACE.map do |n|
|
||||
db.reset_draft
|
||||
|
|
|
|||
Loading…
Reference in New Issue