openproxyherder/README.md

openproxyherder

A Python webapp that "herds" a list of open proxies, with an API for external programs to add and update entries.

scripts/ contains some example scripts that might be useful.

There's a todo list for issue and feature tracking.

Setup openproxyherder

This is tested with Python 3.8 and 3.9, but should work with Python 3.5 and above.

You'll need a postgresql database available via TCP, with the following table:

create table proxies (
    id serial primary key,
    ip inet,
    port int,
    exit_ip inet,
    exit_ipv6 inet,
    proxy_type text,
    status text default 'unscanned',
    last_seen timestamptz default 'epoch',
    last_checked timestamptz default 'epoch',
    submitted_at timestamptz,
    comment text,
    num_failures int default 0,
    recheck bool default false
);

Copy config.example.yaml to config.yaml, edit config.yaml as needed.

Install the packages in requirements.txt (probably a good idea to create a venv).

Run python openproxyherder.py

Scripts

See README in scripts/ for information about the scripts