Pages that resist ordinary browsing
Retrieve a public page as clean Markdown or capture a truthful screenshot when JavaScript, layout, or anti-bot friction gets in the way.
scrape_as_markdown · screenshotResearch infrastructure · MCP
Give your agent the web it can’t reliably reach.
Decodo turns difficult public web pages into structured inputs an AI agent can actually use—then exposes the workflow through an official MCP server.
Partner link. It supports Superbash Learn at no extra cost to you.
Beginner course
From first scrape to a reviewed data workflow.
Thirteen illustrated tutorials explain when scraping belongs in an agent workflow, then cover every current MCP toolset plus direct API rendering, browser actions, async jobs, batches, multi-format output, n8n, LangChain, Agent Skills, and operations.
- 01
Why agents need scraping
Know when built-in search is enough and when repeatable research needs a scraping layer.
Open tutorial - 02
Choose your Decodo route
Know when to use MCP, the Web Scraping API, Site Unblocker, or direct proxies.
Open tutorial - 03
Connect MCP
Set up Basic authentication, hosted or local MCP, narrow toolsets, and a smoke test.
Open tutorial - 04
Pages and screenshots
Collect Markdown or a rendered PNG with controlled geography, device, and output size.
Open tutorial - 05
Dynamic pages
Use JavaScript rendering, browser actions, waits, selectors, and XHR capture.
Open tutorial - 06
Search and AI answers
Research Google, Bing, Ads, Lens, hotels, ChatGPT, Perplexity, and Google AI Mode.
Open tutorial - 07
Ecommerce research
Compare Amazon, Walmart, Target, and TikTok Shop without flattening price evidence.
Open tutorial - 08
Reddit and TikTok
Collect public community observations while keeping claims and verification separate.
Open tutorial - 09
YouTube research
Search videos, inventory channels, collect metadata and subtitles, and understand downloads.
Open tutorial - 10
Sync, async, and batch
Scale a proven job with task IDs, callbacks, batches, and multi-format results.
Open tutorial - 11
Cost and quality control
Set budgets, validate records, retry by cause, monitor cost, and enforce safe boundaries.
Open tutorial - 12
n8n, LangChain, and docs
Choose visual automation, a TypeScript agent tool, or live Decodo documentation over MCP.
Open tutorial - 13
Agent Skills routing
Teach coding agents when to select Decodo without putting credentials inside a Skill.
Open tutorial
Why it earns a place
Search finds the page. Decodo helps the agent finish the job.
Use it when your workflow needs complete page content, repeatable fields, a visual capture, or a target-specific scraper—not just a list of search snippets. The useful pattern is simple: let Decodo retrieve the evidence, let the agent interpret it, and make a human review the final result.
Best uses
Three jobs where it changes the outcome
Marketplace research
Compare products, prices, sellers, and availability across structured ecommerce targets without asking the model to guess from snippets.
Amazon · Walmart · TargetSearch and social evidence
Collect current results from search engines, Reddit, YouTube, or TikTok, then make the agent cite and review what it found.
Search · Reddit · YouTube · TikTokFastest safe setup
Connect narrowly. Test cheaply. Scale only what works.
Decodo’s current MCP documentation supports a hosted endpoint as well as its official npm server. The exact client configuration differs, but the operating discipline should not.
- 01
Get the right token
Create a Decodo Web Scraping API account and copy its Basic authentication token. Keep it in your client’s secret or environment configuration—never in a prompt or repository.
- 02
Connect the official MCP server
Use the local
@decodo/mcp-serverpackage or the hosted endpoint athttps://mcp.decodo.com/mcp. Follow your agent client’s MCP configuration format. - 03
Expose only the tools you need
Start with
webplus one job-specific toolset such assearch,ecommerce, orsocial_media. A smaller catalog makes tool choice easier to audit. - 04
Run one bounded test
Name the target, required fields, failure behavior, and maximum request count. Verify the source URLs and output before turning the prompt into a repeatable workflow.
A better first prompt
Make the budget and evidence rules explicit.
Use Decodo's scrape_as_markdown tool on [PUBLIC URL].
Return: title, source URL, date, and five evidence-backed findings.
If the page fails, stop after one retry and report the error.
Cap the entire run at 2 Decodo requests.Go deeper
Watch the original Hermes + Decodo build
The video companion preserves Ron's real Amazon comparison, request-cost discussion, failures, corrections, and reusable Hermes Skill. Use the new course above for the current official MCP and API path.