What environment variables does the scrape skill require?

It requires BRIGHTDATA_API_KEY and BRIGHTDATA_UNLOCKER_ZONE to be set in the environment before the skill can make requests.

What format does the scrape skill return?

It returns the webpage content as clean markdown.

Can it handle sites that block bots or show CAPTCHAs?

Yes. It uses Bright Data Web Unlocker API to bypass bot detection and CAPTCHA challenges.

Does the scrape skill work on any webpage?

It is designed to scrape any webpage, including those that block bots or present CAPTCHAs, by routing through Bright Data Web Unlocker API.

Bright Data - Web Scraper

SkillWeb & browsing

The scrape skill lets an AI agent scrape any webpage as clean markdown via Bright Data Web Unlocker API. It handles pages that block bots or present CAPTCHAs, returning readable text instead of raw HTML. Use it when an agent needs page content in a form it can read and quote.

Use Bright Data - Web Scraper in Claude, ChatGPT or Ahel Desktop

Free. Sign in, add Bright Data - Web Scraper and connect your AI. About a minute.

Claude ChatGPT Ahel Desktop

Also: Claude Code · Cursor · Codex

Then ask your AI: use the Bright Data - Web Scraper skill

Details

Instructions available. Your AI can read the instructions. Execution depends on the setup they require.

Account requirements not reviewed. Check the skill instructions before use; ahel provides instructions and does not run this skill.

No other account needed.

Obtain a Bright Data API key and an Unlocker zone name.

Bright Data - Web ScraperStart free

What your AI can do with it

Fetch a webpage and return its content as clean markdown
Bypass bot detection on sites that block automated requests
Handle CAPTCHA challenges during page retrieval
Use Bright Data Web Unlocker API for the request
Read BRIGHTDATA_API_KEY and BRIGHTDATA_UNLOCKER_ZONE from the environment

Getting started

Obtain a Bright Data API key and an Unlocker zone name.
Set the BRIGHTDATA_API_KEY environment variable to your API key.
Set the BRIGHTDATA_UNLOCKER_ZONE environment variable to your zone name.
Add the scrape skill to your agent's available skills.
Ask the agent to scrape a URL and confirm it returns markdown.

What this skill tells your AI

The instructions your AI receives, as published by davila7/claude-code-templates in cli-tool/components/skills/web-data/scrape/SKILL.md and read by ahel’s review.

Scrape any webpage and get clean markdown content using Bright Data's Web Unlocker API. Automatically bypasses bot detection and CAPTCHA.

Setup

1. Get your API Key: Get a key from Bright Data Dashboard.

2. Create a Web Unlocker zone: Create a zone at brightdata.com/cp by clicking "Add" (top-right), selecting "Unlocker zone".

3. Set environment variables:

export BRIGHTDATA_API_KEY="your-api-key"
export BRIGHTDATA_UNLOCKER_ZONE="your-zone-name"

Usage

bash scripts/scrape.sh "url"

Parameters:

url (required): The webpage URL to scrape

Examples:

# Scrape a news article
bash scripts/scrape.sh "https://example.com/article"

# Scrape a product page
bash scripts/scrape.sh "https://shop.example.com/product/123"

Output Format

Returns clean markdown content extracted from the webpage:

# Page Title

Main content of the page converted to markdown format...

## Section Heading

More content...

Features

Bot Detection Bypass: Automatically handles anti-bot measures
CAPTCHA Solving: Bypasses CAPTCHA challenges
Clean Markdown: Returns well-formatted markdown content
JavaScript Rendering: Handles JavaScript-heavy pages

Dependencies

curl - For API requests

Signals

GitHub stars: 32k
Forks: 4k
Last commit: Oct 2026

Questions

What environment variables does the scrape skill require?: It requires BRIGHTDATA_API_KEY and BRIGHTDATA_UNLOCKER_ZONE to be set in the environment before the skill can make requests.
What format does the scrape skill return?: It returns the webpage content as clean markdown.
Can it handle sites that block bots or show CAPTCHAs?: Yes. It uses Bright Data Web Unlocker API to bypass bot detection and CAPTCHA challenges.
Does the scrape skill work on any webpage?: It is designed to scrape any webpage, including those that block bots or present CAPTCHAs, by routing through Bright Data Web Unlocker API.

Advanced

Item type: skill
Key: scrape
Source: github.com/davila7/claude-code-templates

github.com/davila7/claude-code-templates