MCP CheckScraping & crawling MCP servers
233 listed servers in Browser automation. Each row shows the automated checks, the secrets asked for and an install command.
No keys needed, every check passed
Scraping & crawling servers 233
Now showing are unlisted. These entries didn't pass our listing filter: no source repo, a very short or duplicated description, or a publisher with 100+ servers. Same automated checks, shown as-is. How we check
| Type | Secrets | Version | Actions | ||||
|---|---|---|---|---|---|---|---|
★ 107 io.github.Sriram-PR/doc-scraper Crawl documentation sites into local corpora agents can search, read, and diff fully offline. | 3/41 warning | 49health of 100 | MCPBRuns locally | Noneno keys asked | 2.24.1 | ||
io.github.jin-1119/plato-mcp PLATO (Pusan National University LMS) MCP server via Moodle Web Service API + ubboard scraping | 4/4All passed | 49health of 100 | RemoteHosted | Noneno keys asked | 0.1.0 | ||
★ 15 io.github.Mingye-Lu/acrawl Autonomous web crawler. 17 browser tools + goal-driven run_goal agent. Single binary, stealth. | 5/61 warning | 52health of 100 | MCPBRuns locally | Noneno keys asked | 0.17.0 | ||
 ★ 7.7k io.github.firecrawl/firecrawl-mcp-server Official Firecrawl Firecrawl, the context API for AI agents: search, scrape, interact with the web, plus paper search. | 8/8All passed | 76health of 100 | npmLocal or hosted | 1secret asked | 3.36.0 | ||
io.github.CodyWatters/riveter MCP server for Riveter's enrichment, scraping, and monitoring API | 8/8All passed | 53health of 100 | npmLocal or hosted | 1secret asked | 0.3.5 | ||
 ★ 1 io.github.nolindnaidoo/scrape-le Analyse robots.txt content and report whether a path may be crawled. | 7/7All passed | 63health of 100 | npmRuns locally | Noneno keys asked | 2.6.0 | ||
★ 11 io.github.eurafaeldecarvalho/browser-scraper Stealth Chrome for AI agents: bot-protected sites, Turnstile, closed Shadow DOM, clean Markdown. | 7/7All passed | 58health of 100 | npmRuns locally | Noneno keys asked | 0.8.3 | ||
★ 17 io.browserless/mcp Official Browserless MCP server: browser automation, scraping, screenshots and PDFs for AI agents. | 7/81 warning | 58health of 100 | npmLocal or hosted | 1secret asked | 1.40.2 | ||
★ 1 io.github.kopachlager/prerenderbuddy-mcp Crawler diagnostics, AI visibility evidence and article drafts for review in Prerender Buddy. | 7/7All passed | 54health of 100 | npmRuns locally | Noneno keys asked | 0.3.3 | ||
 ★ 14k com.apify/apify-mcp-server Official Apify, hosted Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡ | 3/3All passed | 65health of 100 | RemoteHosted | Noneno keys asked | 0.19.5 | ||
★ 192 io.github.xberg-io/crawlberg Scrape, crawl, and map websites to Markdown or JSON via local CLI. | 4/4All passed | 71health of 100 | DockerRuns locally | Noneno keys asked | 1.12.3 | ||
 com.geonode/mcp Web data for agents. Extract web pages, crawl websites, discover URLs, and search the web. | 4/4All passed | 48health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 | ||
io.github.Screenshot-API/screenshot-mcp Website screenshots, URL/HTML to PDF, scrolling videos, and web scraping via ScreenshotAPI. | 7/7All passed | 56health of 100 | npmRuns locally | 1secret asked | 1.0.2 | ||
★ 21 io.github.ZenRows/zenrows-mcp Zenrows MCP server: Fetch, Extract, Batch, and Browser Sessions for AI agents | 8/8All passed | 61health of 100 | npmLocal or hosted | 1secret asked | 2.17.0 | ||
★ 1 net.crawlora/crawlora-mcp Hosted MCP: 3926 structured web-data tools across 488 platform groups. | 4/4All passed | 55health of 100 | RemoteHosted | Noneno keys asked | 1.17.16 | ||
com.soflution/deepcrawl Clone websites, scrape pages to markdown and crawl whole sites. A free Firecrawl alternative. | 4/62 warnings | 35health of 100 | MCPBRuns locally | Noneno keys asked | 1.1.0 | ||
★ 3 io.github.NovadaLabs/novada-mcp One MCP server for all web data. Scrape, search, crawl, and AI research in one install. | 8/8All passed | 57health of 100 | npmLocal or hosted | 1secret asked | 0.9.44 | ||
io.github.crawlbrulee/mcp EU-native web scraping for AI agents. scrape pages, map sites, run background jobs, check usage. | 7/7All passed | 52health of 100 | npmRuns locally | 1secret asked | 1.3.3 | ||
com.seoperception/mcp Search Console, crawl, and SEO to-dos in Cursor and Claude. OAuth or token; close to-dos optionally. | 3/3All passed | 48health of 100 | RemoteHosted | Noneno keys asked | 2.1.0 | ||
 ★ 6 ai.deepscout/deepscout Scrape any e-shop and match products across shops without EAN. API & MCP for AI agents. | 3/3All passed | 52health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 | ||
 ★ 1 com.geekflare/mcp Web search, Google SERP, scraping, screenshots, PDFs, brand data, and website diagnostics. | 8/8All passed | 52health of 100 | npmLocal or hosted | 1secret asked | 0.5.0 | ||
★ 4 io.github.Wopee-io/wopee-mcp AI testing agents for web apps — dispatch test runs, crawls, fetch artifacts and status. | 7/7All passed | 57health of 100 | npmRuns locally | 1secret asked | 1.32.0 | ||
★ 13k io.github.seleniumbase/seleniumbase Stealthy browser automation, testing, and web-scraping via CDP Mode. | 6/71 warning | 85health of 100 | PyPIRuns locally | Noneno keys asked | 4.56.4 | ||
★ 3 io.github.crawlerpeek/crawlerpeek Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only. | 7/7All passed | 57health of 100 | npmRuns locally | Noneno keys asked | 0.2.0 | ||
★ 1 io.github.abhinav162/poltertab Drive and scrape your existing Chrome profile from any AI assistant — no headless browser. | 6/71 warning | 62health of 100 | npmRuns locally | Noneno keys asked | 1.12.0 | ||
io.github.Grubbomatic/crawl-readiness Audit a site for 50+ AI crawlers, check if a page permits an AI use, generate llms.txt and schema. | 7/7All passed | 53health of 100 | npmRuns locally | 1secret asked | 0.2.6 | ||
★ 86 io.github.houtini-ai/seo-audit-console Technical SEO in Claude: GSC + crawl + DataForSEO - audit, portfolio, market sizing, content recon. | 7/81 warning | 65health of 100 | npmRuns locally | 3secrets asked | 0.17.0 | ||
io.github.mark1russell7/polite-crawler A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes. | 7/7All passed | 54health of 100 | npmRuns locally | Noneno keys asked | 0.2.4 | ||
 ★ 1 com.atlas-compiler/atlas Deterministic Zero-LLM Web Compiler, Scraper, Crawler, and Site Mapper for AI Agents. | 8/8All passed | 54health of 100 | npmLocal or hosted | 1secret asked | 1.0.5 | ||
io.github.catalyst-labs-dev/career-site-jobs-scraper Live jobs from companies you name, 17 ATS, one schema; misses never charged. | 3/3All passed | 43health of 100 | RemoteHosted | Noneno keys asked | 1.0.2 | ||
com.queryrev/ai-readiness Checks a site for AI crawler access, robots.txt and llms.txt. Runs locally, no telemetry. | 7/7All passed | 53health of 100 | npmRuns locally | Noneno keys asked | 1.0.0 | ||
★ 18 io.github.Role1776/mcp-retrieval Local Go MCP server: DuckDuckGo/Bing web + image search, readability scraping to Markdown. | 6/6All passed | 63health of 100 | DockerRuns locally | 2secrets asked | 0.1.10 | ||
eu.citedby/citedby Check a page, AI-crawler access, llms.txt and backlinks of a site. Free, read-only. | 4/4All passed | 43health of 100 | RemoteHosted | Noneno keys asked | 1.0.0 | ||
★ 5 io.github.pournasserian/mcp-my-site Search and fetch tools for one website, served from an index you build with the mcpmysite crawler. | 6/6All passed | 65health of 100 | DockerLocal or hosted | 1secret asked | 0.3.0 | ||
★ 75 io.github.maximilianfeix/proxy-scraper Free proxies that actually work: checked HTTP/SOCKS proxies, and pages fetched through them | 7/7All passed | 66health of 100 | PyPIRuns locally | Noneno keys asked | 1.31.0 | ||
io.github.CrawlKit560/crawlkit-scrapers Scrape Booking, Airbnb, Amazon, Zillow, Indeed, Yelp, Reddit, Google Shopping/Jobs via Apify MCP | 3/3All passed | 48health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 | ||
 ★ 63 io.github.supadata-ai/mcp Video transcripts, metadata and structured data from YouTube, TikTok, Instagram, X; web scraping. | 3/3All passed | 51health of 100 | RemoteHosted | Noneno keys asked | 1.3.6 | ||
io.github.jordan-belfort-ai/workday-jobs-scraper Every open job on any myworkdayjobs.com careers site as JSON. Runs on Apify, $0.001 per job. | 3/3All passed | 43health of 100 | RemoteHosted | Noneno keys asked | 1.0.0 | ||
io.github.projectworks007/ai-agent-web-tools Web tools for AI agents: URL to clean Markdown, site crawl, screenshots, PDF/Office to text, OCR. | 3/3All passed | 52health of 100 | RemoteHosted | Noneno keys asked | 1.0.2 | ||
★ 6 io.github.mysleekdesigns/crawlforge-mcp-server Web scraping, crawling, deep research & autonomous extraction — 31 MCP tools, clean Markdown/JSON | 6/82 warnings | 51health of 100 | npmRuns locally | 3secrets asked | 6.24.0 | ||
 ★ 2 io.github.securecoders/opengraph-io-mcp Fetch web/social data, screenshots, scrape/extract content, generate images, audit SEO & links. | 8/8All passed | 50health of 100 | npmLocal or hosted | 1secret asked | 3.2.1 | ||
io.github.atakanelik34/clawifi-mcp Agent-native internet gateway: typed search, fetch, scrape, crawl, and extract via the Clawifi API. | 8/8All passed | 56health of 100 | PyPILocal or hosted | 1secret asked | 0.4.0 | ||
★ 3 io.github.scrapeeverything/mcp Public data from Instagram, TikTok, X, YouTube, Facebook, Telegram, Spotify, Pinterest and Twitch. | 7/7All passed | 53health of 100 | npmRuns locally | 1secret asked | 0.1.0 | ||
app.openkrill/site-check Check public pages for SEO tags, AI crawler rules, AI citation signals and landing page conversion. | 4/4All passed | 50health of 100 | RemoteHosted | Noneno keys asked | 1.8.0 | ||
io.github.datadmon/luma-events-scraper Luma (lu.ma) events as JSON by city, category or calendar: date, venue, hosts, tickets. $1.50/1k | 3/3All passed | 50health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 | ||
★ 1.2k io.github.fastcrw/crw Open-source web scraper for AI agents with scrape, crawl, and map tools | 3/41 warning | 70health of 100 | DockerRuns locally | Noneno keys asked | 0.39.0 | ||
io.github.datadmon/tickertape-stocks-scraper NSE/BSE stock data via Tickertape as JSON: index lists, screener, 10-year fundamentals. | 3/3All passed | 50health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 | ||
io.github.datadmon/oracle-hcm-jobs-scraper Oracle Recruiting Cloud (HCM) career-site jobs as JSON for any company: full JD, dates. $2/1k | 3/3All passed | 50health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 | ||
io.github.datadmon/instahyre-jobs-scraper Instahyre India startup tech jobs as JSON: full JD, skills, company size. $0.80/1k | 3/3All passed | 50health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 | ||
io.github.datadmon/iimjobs-jobs-scraper IIMJobs India MBA, finance & management jobs as JSON: full JD, salary, applicants. $2/1k | 3/3All passed | 50health of 100 | RemoteHosted | Noneno keys asked | 1.0.1 |
Showing 1 to 50 of 233