5 Commits

Author	SHA1	Message	Date
Matt Arnold	4f5efa818b	Add uglydump subcommand to web2text.	2021-04-03 23:38:41 -04:00
Matt Arnold	f0caf13470	* Set a proper user agent, and add a safety valve so infinate recursions don't happen	2021-01-01 00:27:17 -05:00
Matt Arnold	0f6bb97425	Redesign on a more mark/sweep pattern	2020-12-31 19:18:30 -05:00
Matt Arnold	5a4425b644	Add crawler api code	2020-12-31 15:57:08 -05:00
Matt Arnold	43ce9c5e1d	Refactor in preperation building the crawler	2020-12-30 21:46:12 -05:00