Most Puppeteer guides for Amazon follow the same pattern: launch a browser, navigate to a product page, extract with CSS selectors, done. Then you run it and get a CAPTCHA on the third request.
This guide is different. It's built around how Amazon's detection system actually works — which means the code here is substantially more complex than a 30-line tutorial, but it's code that functions in the real world.
Before writing a line: if you need to collect Amazon data at scale (millions of pages, daily refresh), you will eventually need residential proxies and a proxy management layer not covered here. This guide focuses on the scraping logic itself. See our proxy industry deep dive for infrastructure options.
node --version # v18+ required
npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth cheerio
Why puppeteer-extra with stealth? Because Puppeteer out of the box leaks headless signals that Amazon detects immediately — navigator.webdriver === true, missing Chrome-specific properties, zero installed plugins. The stealth plugin patches these. It's not perfect, but without it you're caught before the page loads.
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
const cheerio = require('cheerio');
const fs = require('fs');
puppeteer.use(StealthPlugin());
async function launchBrowser() {
const browser = await puppeteer.launch({
headless: 'new',
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-blink-features=AutomationControlled',
'--disable-dev-shm-usage',
'--disable-accelerated-2d-canvas',
'--window-size=1920,1080',
],
defaultViewport: { width: 1920, height: 1080 },
});
return browser;
}
The --disable-blink-features=AutomationControlled flag is critical — it suppresses the navigator.webdriver property that Chromium exposes in automation mode. Without it, Amazon's scripts see webdriver: true and your session risk score spikes immediately.
This is the step that most tutorials skip and the most common reason scrapers fail.
Real Amazon users don't navigate directly to product URLs. They land on the homepage, search for something, click a result. This navigation chain generates the session cookie sequence (session-id, session-id-time, ubid-main) that Amazon expects. A scraper that lands directly on amazon.com/dp/ASIN with no prior cookies is an anomaly.
const COOKIE_FILE = './amazon_session.json';
async function warmSession(page) {
// Set a realistic User-Agent
await page.setUserAgent(
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
);
// Set headers that match a real browsing context
await page.setExtraHTTPHeaders({
'Accept-Language': 'en-US,en;q=0.9',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
'Accept-Encoding': 'gzip, deflate, br',
'Upgrade-Insecure-Requests': '1',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'none',
'Sec-Fetch-User': '?1',
});
// Load saved cookies if they exist (reuse session)
if (fs.existsSync(COOKIE_FILE)) {
const cookies = JSON.parse(fs.readFileSync(COOKIE_FILE, 'utf8'));
await page.setCookie(...cookies);
console.log(`Loaded ${cookies.length} saved cookies`);
return;
}
// No saved session — warm up by visiting homepage first
console.log('Warming new session via homepage...');
await page.goto('https://www.amazon.com', { waitUntil: 'domcontentloaded', timeout: 30000 });
await randomDelay(2000, 4000);
// Simulate brief homepage interaction
await page.evaluate(() => {
window.scrollBy(0, Math.floor(Math.random() * 300) + 100);
});
await randomDelay(1000, 2000);
// Save the session cookies for reuse
const cookies = await page.cookies();
fs.writeFileSync(COOKIE_FILE, JSON.stringify(cookies, null, 2));
console.log(`Saved ${cookies.length} session cookies`);
}
Fixed delays are detectable. Real users don't wait exactly 2000ms between every action — they spend varying amounts of time reading, accidentally hovering over things, pausing to check their phone.
function randomDelay(min, max) {
// Gaussian-ish distribution: most delays cluster in the middle,
// with occasional long pauses
const u1 = Math.random();
const u2 = Math.random();
const gaussian = Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2);
const mean = (min + max) / 2;
const std = (max - min) / 4;
const delay = Math.round(mean + gaussian * std);
return new Promise(resolve => setTimeout(resolve, Math.max(min, Math.min(max, delay))));
}
async function humanScroll(page) {
// Scroll in a human-like pattern — pause at content, scroll in chunks
const scrollHeight = await page.evaluate(() => document.body.scrollHeight);
const viewportHeight = page.viewport().height;
let currentPosition = 0;
while (currentPosition < scrollHeight * 0.7) {
const scrollAmount = Math.floor(Math.random() * 200) + 100;
await page.evaluate((amount) => window.scrollBy(0, amount), scrollAmount);
currentPosition += scrollAmount;
await randomDelay(300, 800);
}
}
Amazon's class names change with every deploy. The most stable extraction strategy uses multiple fallback approaches, with JSON-LD as the primary source.
async function extractProductData(page, url) {
// Set referrer to simulate coming from search results
await page.goto(url, {
waitUntil: 'domcontentloaded',
timeout: 30000,
referer: 'https://www.google.com/search?q=amazon',
});
// Check if we hit a CAPTCHA or block page
const pageTitle = await page.title();
if (pageTitle.includes('Robot Check') || pageTitle.includes('Sorry') || pageTitle.includes('captcha')) {
console.warn(`Block detected on ${url} — session may need rotation`);
return null;
}
await humanScroll(page);
await randomDelay(1000, 2000);
const data = await page.evaluate(() => {
const result = {
asin: null,
title: null,
price: null,
currency: null,
rating: null,
reviewCount: null,
availability: null,
brand: null,
extractedAt: new Date().toISOString(),
source: null,
};
// Strategy 1: JSON-LD structured data (most stable)
const ldScripts = document.querySelectorAll('script[type="application/ld+json"]');
for (const script of ldScripts) {
try {
const ld = JSON.parse(script.textContent);
if (ld['@type'] === 'Product') {
result.title = ld.name || null;
result.brand = ld.brand?.name || null;
result.rating = ld.aggregateRating?.ratingValue || null;
result.reviewCount = ld.aggregateRating?.reviewCount || null;
if (ld.offers) {
result.price = parseFloat(ld.offers.price) || null;
result.currency = ld.offers.priceCurrency || 'USD';
result.availability = ld.offers.availability?.split('/').pop() || null;
}
result.source = 'json-ld';
break;
}
} catch (_) {}
}
// Strategy 2: ASIN from URL (always stable)
const asinMatch = window.location.pathname.match(/\/dp\/([A-Z0-9]{10})/);
result.asin = asinMatch ? asinMatch[1] : null;
// Strategy 3: Title fallback via #productTitle (stable ID)
if (!result.title) {
const titleEl = document.querySelector('#productTitle');
result.title = titleEl ? titleEl.textContent.trim() : null;
if (result.title) result.source = 'dom-id';
}
// Strategy 4: Price fallback via data attributes on add-to-cart
if (!result.price) {
const priceInput = document.querySelector('input[name="displayedPrice"]');
if (priceInput) {
result.price = parseFloat(priceInput.value) || null;
result.source = result.source || 'data-attr';
}
}
// Strategy 5: Price via .a-price > .a-offscreen (reasonably stable)
if (!result.price) {
const priceEl = document.querySelector('.a-price .a-offscreen');
if (priceEl) {
const priceText = priceEl.textContent.replace(/[^0-9.]/g, '');
result.price = parseFloat(priceText) || null;
result.source = result.source || 'css-class';
}
}
return result;
});
return data;
}
Three extraction strategies with fallbacks means you still get data even when one layer of Amazon's DOM changes. JSON-LD is the primary target because it's structured for machine consumption — Amazon puts it there for Google's benefit, and you benefit from it too.
Your session degrades if you never update the cookies. Refresh them after each scraping run:
async function saveSession(page) {
const cookies = await page.cookies('https://www.amazon.com');
fs.writeFileSync(COOKIE_FILE, JSON.stringify(cookies, null, 2));
}
async function scrapeProducts(asins) {
const browser = await launchBrowser();
const page = await browser.newPage();
const results = [];
try {
await warmSession(page);
for (let i = 0; i < asins.length; i++) {
const asin = asins[i];
const url = `https://www.amazon.com/dp/${asin}`;
console.log(`[${i + 1}/${asins.length}] Scraping ASIN: ${asin}`);
const data = await extractProductData(page, url);
if (data === null) {
// Got blocked — save session and stop this batch
console.error('Detection triggered. Stopping batch to avoid further flagging.');
await saveSession(page);
break;
}
results.push(data);
await saveSession(page);
// Variable delay between products — longer if we've done many in a row
const baseDelay = i > 10 ? 5000 : 3000;
await randomDelay(baseDelay, baseDelay + 3000);
}
} finally {
await browser.close();
}
return results;
}
async function main() {
const asins = [
'B09G9FPHY6', // Example: AirPods Pro
'B0BDHWDR12', // Example: Kindle
'B09B8YWXDF', // Example: Echo Dot
];
const products = await scrapeProducts(asins);
// Write output
fs.writeFileSync('output.json', JSON.stringify(products, null, 2));
console.log(`\nExtracted ${products.length} products`);
console.log(JSON.stringify(products[0], null, 2));
}
main().catch(console.error);
Example output:
{
"asin": "B09G9FPHY6",
"title": "Apple AirPods Pro (2nd Generation) Wireless Earbuds",
"price": 189.99,
"currency": "USD",
"rating": "4.7",
"reviewCount": "82134",
"availability": "InStock",
"brand": "Apple",
"extractedAt": "2025-03-18T14:32:11.000Z",
"source": "json-ld"
}
Be honest with yourself about the limits:
At 10–50 ASINs per day from a single residential IP: This setup works reliably with a warmed session. You'll rarely see a CAPTCHA.
At 100–500 ASINs per day: Occasional blocks. The session recovery logic will handle most of them. Run in the off-peak hours (2–6am EST) when Amazon's detection thresholds are slightly more lenient.
At 1,000+ ASINs per day: You need proxy rotation. A single residential IP cannot sustain this volume without frequent blocks. You'll need to split the load across multiple IPs and sessions.
At 10,000+ ASINs per day: This requires a full proxy management layer, multiple browser instances, session pooling, and likely a proxy provider with a large, clean residential pool. The architecture becomes as complex as the scraping logic itself.
Logged-in scraping: Don't use your real account. If you want the benefits of a logged-in session (better prices, higher detection threshold), use a throwaway account with some organic activity on it — purchases, reviews, wishlist items. Expect eventual account suspension.
CSS selectors in this codebase may break within weeks. Amazon's frontend is continuously deployed. The only future-proof extraction is JSON-LD (Amazon controls this for SEO reasons and changes it rarely) and the ASIN-from-URL pattern.
If your source field starts returning css-class frequently instead of json-ld, Amazon has either changed their JSON-LD output for that product type or something is wrong with page rendering. Investigate before assuming the CSS fallback is reliable long-term.
Monitor your output data quality as closely as you monitor your block rate. Silent failures (returning stale data) are worse than noisy failures (returning null).
