proxy-scraper-checker

HTTP, SOCKS4, SOCKS5 proxies scraper and checker.

• Can determine if the proxy is anonymous.
• Supports determining the geolocation of the proxy exit node.
• Can sort proxies by speed.
• Uses regex to find proxies of format protocol://username:password@host:port on a web page or in a local file, allowing proxies to be extracted even from json without code changes.
• Supports proxies with authentication.
• It is possible to specify the URL to which to send a request to check the proxy.
• Supports saving to plain text and json.
• Asynchronous.

You can get proxies obtained using this project in monosans/proxy-list.

Installation and usage

Standalone executable

This is the easiest way, but it is only available for x86-64 Windows, x86-64/arm64 macOS and x86-64 Linux. Just download the archive for your OS from nightly.link, unzip it, edit config.toml and run the proxy_scraper_checker executable.

If Windows antivirus detects the executable file as a virus, please read this.

Docker

• Install Docker Compose.
• Download and unpack the archive with the program.
• Edit config.toml to your preference.
• Run the following commands:

  docker compose build --pull
  docker compose up --no-log-prefix --remove-orphans
  

Running from source code

Desktop

• Install Python. Supported versions are 3.9 to 3.12. The recommended version is 3.12.
• Download and unpack the archive with the program.
• Edit config.toml to your preference.
• Run the script that installs dependencies and starts proxy-scraper-checker:
  • On Windows run start.cmd
  • On Unix-like operating systems run start.sh

Termux

To use proxy-scraper-checker in Termux, knowledge of the Unix command-line interface is required.

• Download Termux from F-Droid. Don’t download it from Google Play.
• Run the following command (it will automatically update Termux packages, install Python, and download and install proxy-scraper-checker):

  bash <(curl -fsSL 'https://raw.githubusercontent.com/monosans/proxy-scraper-checker/main/install-termux.sh')
  

• Edit ~/proxy-scraper-checker/config.toml to your preference using a text editor (vim/nano).
• To run proxy-scraper-checker use the following command:

  cd ~/proxy-scraper-checker && sh start-termux.sh
  

Something else?

All other info is available in config.toml file.

License

MIT

This product includes GeoLite2 Data created by MaxMind, available from https://www.maxmind.com.