CMS ETL BBQ! Herein lie scripts I use to go from Day One exports to fully functional web blog and gemini capsule. FAQ 1. Are they good scripts? No. 2. Do they run without a bunch of external dependencies? Also no. 3. Should other people use this? Thrice no.
Go to file
Joe Mulvaney 14458f80d2 tweak gemini index in preparation for submission to antenna 2023-08-17 22:21:57 +00:00
assets tweak html index template for better doc structure 2023-08-17 22:21:35 +00:00
src tweak gemini index in preparation for submission to antenna 2023-08-17 22:21:57 +00:00
.gitignore ignore temp vim files 2023-07-09 16:54:17 +00:00
DEPENDENCIES update dependencies -- don't need ffmpeg rn 2023-07-17 16:24:08 +00:00
README check_email script finished, and very thoroughly documented 2023-08-07 16:54:53 +00:00
check_email.py check_email script finished, and very thoroughly documented 2023-08-07 16:54:53 +00:00
package-lock.json WIP: nearly working, just need to figure out why the comments link in gemini is broken 2023-07-20 04:07:54 +00:00
package.json WIP: nearly working, just need to figure out why the comments link in gemini is broken 2023-07-20 04:07:54 +00:00

README

CMS ETL BBQ

# What is this?
I made this to take zip files exported from Day One (the only writing app I've ever used semi-consistently) and makes them into a journal that works over gemini, gopher, and http. Featuring an index page. Tag pages. Soon, comments, maybe. 

The HTTP part has been tested to work on three major classes of web browser
* Modern ones. I test on Firefox and Safari. Probably Chromium browsers work too.
* Ones that run on OS X 10.4.12, which is Tiger + a bunch of patches that includes emoji. Testing is done on OmniWeb 5.11. The main effect of supporting this is that
  ** I'm using `float`s instead of `flexbox`es in my CSS, and 
  ** requesting HTML fragments with `XMLHttpRequest` for comments, instead of JSON with `fetch`.
* Text mode browsers. Tested with `lynx` and `links`. This means all my fallbacks have to work and my DOM has to be logically arranged. 

# What does it do?
Given a set of entries from Day One on mac, that is uploaded as a zip file to ~/dayone/ and an empty dir ~/markdown:

$ node processDayone.js --inputZip ~/dayone/<zip> --root ~/markdown

This will:-
1. Run day-one-to-markdown script to get a set of directories each containing one front matter format markdown file
2. Load the frontmatter markdowns, and create gemtext and html
3. Process the images to be ready for distribution (strip EXIF, ensure rotation, smallify)

Note also:
* The DEPENDENCIES file discusses what's needed in the environment that the script is run
* The TODO file is my super-advanced issue tracking system
* There's no license yet


# Autoimport notes
There is a script, `check_email.py` that will check a maildir-format local mailbox for files from specified people that have a zip attachment. It's designed to be run in the local user's crontab. I'm sure this is very secure. I have it set up with all the *s (ie it runs every minute). I have an env var file that i source first before invoking the script. That env var defines the following things:

export PATH=<the necessary path for python, whatever, etc, stuff>
export JOURNAL_EMAIL_WHITELIST=<comma separated list of allowed email senders>
export JOURNAL_DIR=<the directory that contains the journal files, e.g. ~/journal>
export MAIL_DIR=<the mail dir, e.g. ~/.mail>
export ZIP_ROOT_DIR=<the place the zip files should go, e.g. ~/dayone>