One problem I've encountered in the past with automated screen scrapers is having the IP address of my scraper blocked. How do you handle IP bans? Conceivably, your scrapers exist in a cloud infrastructure with the ability to create new IP addresses on the fly, but that could prove costly and ineffective long term. Thanks.
Please sign in to leave a comment.