Commit Graph

8721 Commits

Author SHA1 Message Date
Kartik K. Agaram 8aa407a974 . 2021-08-11 10:37:12 -07:00
Kartik K. Agaram b3decbb57f . 2021-08-11 10:29:34 -07:00
Kartik K. Agaram 4eda8c599c debugging helper: heap remaining 2021-08-10 22:47:30 -07:00
Kartik K. Agaram 34d3cb48e9 . 2021-08-10 22:46:51 -07:00
Kartik K. Agaram 76eceb2ca3 slack: first page rendering of real FoC data
https://futureofcoding.org/community
2021-08-10 22:44:49 -07:00
Kartik K. Agaram c407d842a4 slack: first rendering of test data 2021-08-10 22:17:05 -07:00
Kartik K. Agaram 0d905ca59e slack: parse items 2021-08-10 21:05:02 -07:00
Kartik K. Agaram 3c88966683 . 2021-08-10 20:52:18 -07:00
Kartik K. Agaram 6ff54c16f9 . 2021-08-10 20:50:34 -07:00
Kartik K. Agaram 2c07b35861 . 2021-08-10 20:50:16 -07:00
Kartik K. Agaram e6d6a9e4df . 2021-08-10 20:49:50 -07:00
Kartik K. Agaram 622d9e4016 . 2021-08-10 19:55:52 -07:00
Kartik K. Agaram 7175cf056a . 2021-08-10 15:35:13 -07:00
Kartik K. Agaram 6477c46465 slack: parse users 2021-08-10 13:58:14 -07:00
Kartik K. Agaram ca356b9618 slack: import some test data 2021-08-10 10:03:32 -07:00
Kartik K. Agaram c41f17170d . 2021-08-10 09:13:59 -07:00
Kartik K. Agaram 350a255134 new tool: creating a data disk 2021-08-10 08:59:00 -07:00
Kartik K. Agaram b0a83af0e5 slack: fix abort after load
I forgot about the null bytes that come from loading sectors.
2021-08-10 06:29:11 -07:00
Kartik K. Agaram 179db5aeb6 slack: takes almost 3 minutes to load 55MB
This is with KVM on a pretty fast Thinkpad.

If I exclude images, 14MB takes 45s to load.

There's also a bug in parsing at the moment; it aborts after loading all
records.
2021-08-10 06:17:50 -07:00
Kartik K. Agaram e72e5ad297 . 2021-08-10 05:52:48 -07:00
Kartik K. Agaram ed068e859d . 2021-08-10 05:26:10 -07:00
Kartik K. Agaram 74dad4c484 slack: emit comment parent indices in converter
They're easier to process when loading the data disk.

In the process we lose a few more items because they're comments to
items we were dropping earlier.
2021-08-10 05:14:59 -07:00
Kartik K. Agaram 49352e1616 . 2021-08-10 04:55:19 -07:00
Kartik K. Agaram 7f972dfc7c undo my stupid format for post/comment ids
Now items just have a field for parent id.
2021-08-10 04:51:02 -07:00
Kartik K. Agaram c7d2682aa7 . 2021-08-10 04:45:27 -07:00
Kartik K. Agaram 5456a1edcb . 2021-08-10 04:44:43 -07:00
Kartik K. Agaram 98b11ea645 . 2021-08-10 04:34:58 -07:00
Kartik K. Agaram 34aba8e6c6 . 2021-08-10 04:34:11 -07:00
Kartik K. Agaram f37689f932 . 2021-08-10 04:32:37 -07:00
Kartik K. Agaram 49dacb600e . 2021-08-10 04:28:44 -07:00
Kartik K. Agaram 639f89ca42 .
Vestige from when we were scanning all files in the current directory
rather than channels.json.
2021-08-10 04:25:44 -07:00
Kartik K. Agaram f09ee2aa19 . 2021-08-10 04:22:29 -07:00
Kartik K. Agaram 58c62538ea . 2021-08-10 04:20:10 -07:00
Kartik K. Agaram 4fd5514a59 . 2021-08-10 04:19:54 -07:00
Kartik K. Agaram 33b77b31b4 . 2021-08-10 04:18:34 -07:00
Kartik K. Agaram 12d37bf5b9 . 2021-08-10 04:16:36 -07:00
Kartik K. Agaram d94eb48b92 . 2021-08-10 04:16:22 -07:00
Kartik K. Agaram cb38fc348b . 2021-08-10 04:15:40 -07:00
Kartik K. Agaram 0c8f865861 . 2021-08-10 04:14:32 -07:00
Kartik K. Agaram 4f236749a2 . 2021-08-10 04:11:51 -07:00
Kartik K. Agaram dd625f6caa . 2021-08-10 04:08:15 -07:00
Kartik K. Agaram 7784855c18 . 2021-08-08 11:15:22 -07:00
Kartik K. Agaram b67270e171 . 2021-08-07 22:57:15 -07:00
Kartik K. Agaram fbc1f309d8 . 2021-08-07 22:57:15 -07:00
Kartik K. Agaram 3e6da6e25c . 2021-08-06 21:01:38 -07:00
Kartik K. Agaram cbfaf23ae3 . 2021-08-06 20:54:34 -07:00
Kartik K. Agaram 52f1c1fb2f beginnings of a Slack archive reader
I'm hackily depending on Python (3.something) to prototype the disk
image creator. But no non-std libs.

Once the disk image is created, I've validated that it can be loaded
from disk without too much latency (assuming KVM).
2021-08-06 20:46:52 -07:00
Kartik K. Agaram 19d2e6423a . 2021-08-02 10:27:58 -07:00
Kartik K. Agaram 4692c48f40 better error handling on stray commas 2021-08-01 18:26:35 -07:00
Kartik K. Agaram 964becd404 . 2021-08-01 14:47:30 -07:00