Web Crawling and Scraping using Python, Selenium, and PhantomJS

Tonight I gave a demonstration of basic web crawling and scraping in Python, using PhantomJS and Selenium. The slides are brief, and are in PPTX and PDF.

Note: These pages are still being written, and so the examples are currently incomplete.

Example #1: Web scrape Yahoo search results

Want to see what search terms get your site on the first page of Yahoo results? Here's a basic example for doing this.

Example #2: Find links and crawl

Want to crawl the web by finding and following links on web pages? Here's a simple web crawler example.

Related Tutorials

How to Fix a Transparent Background in a PhantomJS Screenshot (Python 3, Selenium)


Leave a comment

What color are green eyes? (spam prevention)
Code under MIT License unless otherwise indicated.
© 2020, Downranked, LLC.