Finish various TODO items.

README.md

@@ -375,14 +375,6 @@ Check out the following links:
 
 TODO:
 
- * Specify how headers are sorted (%TQbh/Z7Wpsa8rhsvyYwPWpdQ4HMrw3Ka7b3sbHBRmTk=.sha256)
- * Clarification: Is a string character always one character, regardless of bytes? (Kas)
- * Specify where `depth` starts (Cinnamon)
- * Specify if `depth` allows gaps (Cinnamon)
- * Specify if body strings not allowed to contain characters like newlines, null bytes, UTF-8 stuff? Can they contain an escaped double quote \"?
- * Specify how many keys can a message have
- * Specify if the signature includes the headers also, or just the body.
- * Explain what free listening is for SSB outsiders.
  * Explain what forking is and why it is not supported.
+ * Explain how verification happens (Soapdog)
  * Question: "How do apps find what they need in the Pigeon feeds and/or accumulate state forward? Is this the responsibility of the Pigeon libraries, or do apps do it themselves?"
- * Explain how verification happens (Soapdog)

dev_docs.md

@@ -1,7 +1,31 @@
+# Fundamental Concept: Free Listening
+
+The concept of "Free Listening" is mentioned often when developing the protocol. The original concept is borrowed from the Secure Scuttlebutt protocol and is well summarized in [a 2018 work by Andre Staltz](https://staltz.com/reinvent-the-social-web.html):
+
+```
+Pull is the opposite of push. Instead of realtime push notifications, you choose when you want to get updates about new content. Instead of free speech, as in the right to broadcast, we value free listening, which is the right not to be shouted at. Most social networks allow people to basically insert—push—themselves into a community. But Scuttlebutt invites are the other way around: someone in the community has to pull you in.
+
+We are not proponents of total self-reliance. With full independence, you’d just be a lonely node in your social graph. The real value of the social graph is not the amount of nodes, it’s the amount of edges. They represent dependence on each other.
+
+Your friend is your backup, literally. If you happen to lose all your data because your computer explodes, all you need is your crypto key pair, your identity, which is a small file. Then you can re-download everything you said and liked and did from a friend nearby. Nothing will be lost. This happened to someone in the community, and we were pleasantly surprised how well it worked.
+```
+
+Pigeon has many of the same goals as Secure Scuttlebutt, including the user's right to Free Listening. Keep this in mind as you navigate the various design decisions and compromises of the protocol.
+
+# Fundamental Concept: Content Addressing
+
+Pigeon Protocol relies heavily on content addressing and the SHA-512 hashing algorithm. This means that instead of assigning arbitrary or user-generated names to resources (as is the case with web addresses), names are assigned using the SHA-512 hash algorithm.
+
+Please watch [this video] if you are unfamiliar with hashing algorithms.
+
+# Fundamental Concept: Digital Signatures
+
+Pigeon uses [ED25519](https://ed25519.cr.yp.to/) to generate and verify message signatures. An understanding of digital signing algorithms is assumed. If you need help understanding this concept, please contact us on the mailing list. Due to time constraints, I will defer to existing resources that exist online.
+
 # Messages: The Basic Building Block
 
 The most important protocol concept is that of the "message".
-In their most simple form, Pigeon protocol messages are just ASCII text documents. They are human readable and can even be created by hand in a text editor, though most clients will provide better means of authoring messages.
+In their most simple form, Pigeon protocol messages are just text documents. Text encoding is determined by the client, but UTF-8 is highly encouraged for maximum interoperability. They are human readable and can even be created by hand in a text editor, though most clients will provide better means of authoring messages.
 
 Below is an example of such a message:
 

message_format.md

@@ -25,7 +25,7 @@ Pigeon has 4 data types:
  * Blob multihash: Used to identify arbitrary binary files mentioned in a feed. Blob multihashes start with the word `FILE.`, followed by a Base32 hash (SHA256).
  * Message multihash: Used to reference a message in someone's feed. Starts with the word `TEXT.` followed by a Base32 SHA256 checksum.
  * User multihash: Used to reference a particular feed. Starts with the word `USER.` followed by a base32 encoded ED25519 public key.
- * String: Information shorter than 128 characters can be wrapped in "double quotes" and placed directly into messages. Larger strings must be converted to blobs.
+ * String: Information shorter than 128 characters can be wrapped in "double quotes" and placed directly into messages. Larger strings must be converted to blobs. The byte length of a character is determined by the client's chosen text encoding scheme. Pigeon has no official text encoding, but UTF-8 is highly encouraged in scenarios where interoperability is important. One character always one character, even if it is multi-byte. Strings cannot contain whitespace characters (encoding dependent). The protocol does not implement character escaping (eg: `"\n"`). Please start a discussion on the mailing list if your application requires this feature.
  * None: The word "NONE" is used to indicate the absence of data, similarly to `null` or `nil` seen in some programming languages.
 
 ### Parts of a Message
@@ -80,6 +80,8 @@ Pigeon messages exist in a linear sequence which only moves forward and never "f
 Every message has a `depth` field to indicate its "place in line".
 Because every message has an ever-increasing integer that never duplicates, every message in a Pigeon feed will have a unique hash. This is true even if messages have identical body content.
 
+The `depth` count always starts at 0 and icreases by 1 every time a new message is added to the feeds. Feeds that assign `depth` values in a non-sequential order are invalid. Eg: No gaps, no skipping, etc..
+
 ### Line 3: `Kind`
 
 EXAMPLE:
@@ -175,7 +177,8 @@ Some notes about body entries:
 
  * The body of a message starts and ends with an empty line (`\n`).
  * Every body entry is a key value pair. Keys and values are separated by a `:` character (no spaces).
- * A key must be 1-90 characters in length
+ * A message may not exceed 128 key/value pairs.
+ * A key must be 1-90 characters in length.
  * A key cannot contain whitespace or control characters
  * A key may contain any of the following characters:
     * alphanumeric characters (a-z, A-Z, 0-9)
@@ -239,6 +242,6 @@ signature JSPJJQJRVBVGV52K2058AR2KFQCWSZ8M8W6Q6PB93R2T3SJ031AYX1X74KCW06HHVQ9Y6N
 ```
 
 A signature starts with the word `signature` followed by a space.
-After that, the body (including the trailing `\n`) is signed using the author's ED25519 key.
+After that, the body and header (including the trailing `\n`) are signed using the author's ED25519 key.
 The signature is encoded with Crockford base 32.
 An empty carriage return is added after the signature line.