4 Commits

Author	SHA1	Message	Date
James Mills	fe18a79dd2	Add support for robots.txt user agent control from web crawlers (#10)	2018-04-03 14:32:43 -07:00
James Mills	74783dccd7	Minor refactor/cleanup	2017-10-15 13:16:35 -07:00
James Mills	76f337705e	Set sensible defaults	2017-06-26 01:54:33 -07:00
James Mills	c51b6d8887	Migrated to a Library structure with gopherproxy cmd for reuse by gopherclient	2016-09-30 19:40:24 -07:00