How to scrape PerimeterX-protected sites

PerimeterX, now part of HUMAN, protects some of the most scraped sites on the web, including Zillow, Walmart and StockX. When it decides a visitor is automated, it serves a "Press & Hold" challenge or a short "Robot or human?" page instead of the content. StealthASF Ultra mode gets the real page. In our production tests on 8 October 2026, Zillow, Walmart and StockX all returned HTTP 200 with their real page titles in Ultra mode. This guide uses a Walmart category page as the example.

1. Use Ultra mode

The API field engine selects the request mode. Set it to ultra for PerimeterX sites. Ultra is our strongest mode, built for the hardest targets, and it is the mode that passed all three PerimeterX sites in our tests. It costs 50 base credits. Ultra renders the page and supports extraction. It does not run browser steps such as clicks, so plan to collect data from page URLs directly: category pages, search result pages and product pages each have their own URL.

2. Send a first request with curl

Verify your email, create an API key in the dashboard and set STEALTHASF_API_KEY in your shell. Keep the key out of source control. The request asks for text extraction so you can see right away whether the page contains products or a challenge.

curl --max-time 600 "https://stealthasf.com/v1/scrape" \
  -H "x-api-key: $STEALTHASF_API_KEY" \
  -H "content-type: application/json" \
  --data-raw '{"url":"https://www.walmart.com/browse/electronics/3944","engine":"ultra","extract":"text"}'

The endpoint accepts JSON and authenticates with the x-api-key header. Use the 600-second client timeout from the example, since a rendered page takes longer than a plain request.

3. Read the response

The API's HTTP status and the target's status are separate. A good result has API status 200, target status 200, the site's normal title in html, and product names and prices in the extracted text. If you see "Press & Hold" or "Robot or human?" in the text, you have the challenge, not the page. When StealthASF detects a block, it returns HTTP 422 with a job_id and charges nothing.

Keep engine and credits_charged from every response. They show which mode produced the page and exactly what it cost.

4. Move the request into Python

The Python example uses the standard library only. An API error stops the script, so an error body never ends up in your dataset. Add your own checks after the request succeeds: for a category page, that the page lists products; for a product page, that it has a name and a price.

import json
import os
from urllib.error import HTTPError
from urllib.request import Request, urlopen

payload = {
    "url": "https://www.walmart.com/browse/electronics/3944",
    "engine": "ultra",
    "extract": "text"
}
request = Request(
    "https://stealthasf.com/v1/scrape",
    data=json.dumps(payload).encode("utf-8"),
    headers={
        "x-api-key": os.environ["STEALTHASF_API_KEY"],
        "content-type": "application/json",
    },
    method="POST",
)
try:
    with urlopen(request, timeout=600) as response:
        result = json.load(response)
except HTTPError as error:
    detail = error.read().decode("utf-8")
    raise SystemExit(f"API error {error.code}: {detail}")

print("Target status:", result["status"])
print("Credits:", result["credits_charged"])
print(result.get("data"))

For structured output, switch extract to links to get every link with its text, which is the fastest way to collect product URLs from a category page. Use meta for the title, description and Open Graph tags. The full rendered HTML is always in html, so you can read embedded JSON-LD or run your own parser. When the page pulls its data from a JSON endpoint, the response can include discovered_api.

5. Budget the credits

An Ultra request costs 50 credits and includes the first 1 MB of transfer. Each additional MB adds 10 credits, rounded up to a whole credit, where MB means 1,048,576 bytes. A 2 MB Ultra request costs 60 credits. A solved CAPTCHA adds 25 credits. Blocked requests are never charged. At the base rate, the Hobby plan's 70,000 credits cover 1,400 Ultra requests, Pro covers 5,000 and Scale covers 20,000. The 200 free trial credits cover four, enough to check your target before you buy.

6. Scale up

Start with a dozen URLs of each page type you need and confirm the fields you care about are present in every response. Record the URL, engine, target status, job ID and credits. A product page with no price may be out of stock rather than broken, so keep a few examples of each state.

Then increase concurrency gradually within your plan limit. On a 429 response, follow Retry-After and cap your retries. If one URL keeps returning 422, send support the job ID and a short description of what you expected. Track credits per complete record over time. A sudden drop in complete records usually means the page layout changed. For site-specific walkthroughs see the Zillow, Walmart and StockX guides, and the API reference for every field.