proxy scraper checker

HTTP, SOCKS4, SOCKS5 proxies scraper and checker with support of determining exit-node's geolocation for each proxy.

monosans
118
38
Python

proxy-scraper-checker

CI

Screenshot

HTTP, SOCKS4, SOCKS5 proxies scraper and checker.

  • Written in Rust.
  • Can determine if the proxy is anonymous.
  • Supports determining the geolocation of the proxy exit node.
  • Can sort proxies by speed.
  • Uses regex to find proxies of format protocol://username:password@host:port on a web page or in a local file, allowing proxies to be extracted even from json without code changes.
  • Supports proxies with authentication.
  • It is possible to specify the URL to which to send a request to check the proxy.
  • Supports saving to plain text and json.
  • Asynchronous.

You can get proxies obtained using this project in monosans/proxy-list.

Installation and usage

Binary

[!NOTE]
There is a separate section for Termux.

  1. Download the archive for your platform from nightly.link. If you are not sure which archive you need, use the table.
  2. Unpack the archive into a separate folder.
  3. Edit config.toml to your preference.
  4. Run the executable.

Docker

[!NOTE]
Only a simple user interface in the form of logs is implemented for Docker.

  1. Install Docker Compose.

  2. Download the archive for your platform from nightly.link. Look for artifacts named proxy-scraper-checker-docker.

  3. Unpack the archive into a separate folder.

  4. Edit config.toml to your preference.

  5. Run the following commands:

    Windows:

    docker compose build
docker compose up --no-log-prefix --remove-orphans

    Linux/macOS:

    docker compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g)
docker compose up --no-log-prefix --remove-orphans

Termux

  1. Download Termux from F-Droid. Don’t download it from Google Play.

  2. Run the following command. It will automatically download and install proxy-scraper-checker.

    bash <(curl -fsSL 'https://raw.githubusercontent.com/monosans/proxy-scraper-checker/main/termux.sh')

  3. Edit ~/proxy-scraper-checker/config.toml to your preference using a text editor (vim/nano).

  4. To run proxy-scraper-checker use the following command:

    cd ~/proxy-scraper-checker && ./proxy-scraper-checker

License

MIT

This product includes GeoLite2 Data created by MaxMind, available from https://www.maxmind.com.