A next-generation crawling and spidering framework.