Crawl Pilot Quickstart: Extract Structured Web Data in 5 Minutes (No Code)

This guide uses Books to Scrape — a site built specifically for scraping practice. No login, no anti-bot measures. It's the best place to learn the tool before pointing it at your real targets.

By the end you'll have a clean CSV of book titles, prices, and ratings from all 50 pages (1,000 books total). The whole thing takes under 5 minutes once the extension is loaded.

Step 1: Install the Extension

CrawlPilot is a Chrome extension loaded in developer mode.

1. 02
   Download or clone the CrawlPilot repository.
2. 04
   Build it:

   bash

   npm install npm run build

   This creates a dist/ folder.
3. 06
   In Chrome, go to chrome://extensions.
4. 08
   Toggle Developer mode on (top-right corner).
5. 10
   Click Load unpacked → select the dist/ folder.
6. 12
   Click the puzzle icon in Chrome's toolbar → find CrawlPilot → click the pin icon.

The CrawlPilot icon now appears in your toolbar.

Step 2: Open the Target Page

Go to https://books.toscrape.com. You'll see a grid of 20 books with titles, star ratings, and prices.

Leave the page open and don't navigate anywhere yet.

Step 3: Open CrawlPilot and Connect the Tab

Click the CrawlPilot icon in your toolbar. The extension opens as a side panel on the right side of Chrome.

At the top of the panel you'll see a connection banner. If it shows "Extension not connected to this tab", click Connect. The banner clears and the side panel is now linked to the books page.

Step 4: Open List Extractor

In the left sidebar of the panel, click List Extractor. This is the tool for pulling repeated items from a page — product grids, article feeds, job listings, search results. It runs as a simple three-step wizard.

Step 5: Wizard Step 1 — Pick a Repeating Item

You don't hunt for containers or map out columns. You click one book and CrawlPilot figures out the rest.

1. 02
   Click Select on Page. The panel switches to "Listening for Click…" and the page becomes selectable.
2. 04
   Hover over one book card and click it.
3. 06
   CrawlPilot detects the repeating pattern and shows a green "Repeating Pattern Found · 20 matching items on page" card — 20 being the books on the first page. It also auto-extracts the fields it found on that card: title, price, rating, availability, and the link.

From here you can:

• Preview Sample Data (the database icon) — opens a sample table of the detected rows so you can confirm the data looks right before the full run.
• Highlight on page (the eye icon) — flashes every matched item so you can verify all 20 were caught.
• Add another field (the +) — click it, then click an element on the card to capture an extra column.
• Remove any field you don't need.

Once the preview looks right, click Next.

Step 6: Wizard Step 2 — Choose How to Load More Items

The wizard asks "How should we load more items?" with three options:

• Numbered Pagination — page numbers or a Next button
• "Load More" Button — a button that appends more items
• Infinite Scroll / Single Page — items load as you scroll, or everything's on one page

Books to Scrape uses page numbers, so choose Numbered Pagination. The wizard asks you to point to the "Next" button — click the "next →" link at the bottom of the books grid so CrawlPilot learns the selector. Click Next.

Step 7: Wizard Step 3 — Review & Run

The final step shows a Summary (fields selected, items per page, page limit) and a Crawl Speed setting: Careful, Balanced, or Aggressive. Balanced is fine for this practice site.

Open Advanced Settings and set Max Pages to Extract to 50 (all pages). Then click Start Extraction. The status bar shows:

• A live item count climbing as each page is processed
• A Stop button if you need to pause early

For 50 pages at Balanced speed, expect 2–4 minutes.

Step 8: Open the Results Table and Export

When it finishes, click Open Results Table — this opens the full data table in a new tab.

You'll see 1,000 rows, one per book, with all your columns.

Click Export and choose CSV (JSON and Markdown are also available, plus Google Sheets, Webhook, Airtable, and Zapier). Open the CSV in Google Sheets or Excel — it's ready to use immediately.

Your output will look like this:

Troubleshooting

"The side panel shows the extension isn't connected" Click Connect in the banner. If it doesn't clear, refresh the page and reopen the panel.

"Nothing highlights when I click Select on Page" The extension needs permission to inject scripts. Go to chrome://extensions → CrawlPilot → Details → Site access → change to On all sites. Then refresh the page.

"Only 15 items detected instead of 20" The pattern was picked from an unusual card. Click Select on Page again and pick a cleaner, typical book card, then use Highlight on page and Preview Sample Data to confirm all 20 are caught before running.

"Pagination stops after a few pages" Try the Careful crawl speed. Some sites throttle rapid sequential page loads.

What to Try Next

Once you have the basics, try the Page Extractor — paste a list of the individual book URLs from your CSV and extract the full description, UPC, and dimensions from each detail page. That's a two-step pipeline: List Extractor to get the URLs, Page Extractor to drill into each one.