Most websites that require pagination will update the URL which makes extraction easy. For websites that do not update the URL as you paginate, you can build an interactive extractor with pagination to extract data on each page.
An interactive extractor with pagination allows you to record the steps required to go to the next page, and the extractor will treat each page as an input for you to train.
Caution: Your account will be charged per page paginated using this method. For example, running a paginated extractor with 1 input with 10 pages and screenshots enabled should consume 20 queries (10 pages + 10 screenshots).
To add pagination to your interactive extractor, simply click the Add Pagination button.
Once you've added pagination to your interactive extractor, record the interaction steps required to navigate to the next page. For the example below, we're extracting reviews from https://www.walmart.com/reviews/product/14301816. We simply record a mouse click on the Next Page button and add a small delay for the page to load.
Since there are 6 total pages, the Next Page button needs to be clicked 5 times. By setting the pages variable to 5, the extractor will paginate 5 times and extract the data on each page.
If you're dealing with a link that changes every time it's being clicked, you may want to modify the selector to handle different variations of the link.
Once the pagination is done, you can train the extractor like you would normally. You can modify the number of pages to paginate by modifying the value in the settings for the extractor.