Spider4j is an open source web crawler expand from webmagic for Java which provides a simple interface for crawling the Web. Using it, you can setup a multi-threaded web crawler in few minutes.