Skip to content
anyai.fyi

AI tools for “crawl”

87 results (tools & skills) · 191 ms

CrawlQ Studio logo
CrawlQ Studio crawlq.ai

For agencies producing AI-assisted content: score every output against each client's brand documents, require a named reviewer's approval, and keep a tamper-evident audit trail. EU-hosted.

Paid Writing API Open source Online · checked 1d ago
Match 10.0
Scrapewise logo
Scrapewise scrapewise.ai

Scrape, clean, match: ready data APIs and AI scrapers that end in a competitor price comparison. Pay as you go, 5 free requests. REST API, CSV, MCP.

Paid Writing API Online · checked 1d ago
Match 2.9
Parallel logo
Parallel parallel.ai

Web infrastructure for AI to search, extract, monitor, and reason over the world's information. Introducing Parallel Decisions API: the first decision engine with world knowledge built in. Learn more.[Learn…

Coding API Online · checked 1d ago
Match 1.8
The Aggregate logo
The Aggregate theaggregate.ai

Every public LLM benchmark we can scrape, folded into one daily ranking with uncertainty, plus saturation forecasts per benchmark and a changelog of what moved.

Coding Online · checked 1d ago
Match 2.3
SummaryTube logo
SummaryTube summarytube.com

SummaryTube is the best AI YouTube video summarizer. Get instant AI summaries of any YouTube video for free — extract key insights, moments & transcripts in seconds.

Paid Education Online · checked 1d ago
Match 1.9
PDF Splitter logo
PDF Splitter mdpresso.com

Free PDF splitter to extract pages, split by page range, or save every page as a separate PDF. 100% in your browser — files never leave your device.

Coding API Online · checked 5d ago
Match 1.8
Skill stardust adobe/skills

…existing website through a four-phase pipeline — extract (crawl and capture the current site), direct (set a visual direction), prototype (generate redesigned HTML), and migrate (emit a deployable static site). Tracks…

Design & Media curated Worth a look
Match 7.7
Skill prepare-migration adobe/skills

…by orchestrating the prep cascade — a full-inventory crawl (extract --prep), page-type and module-catalog confirmation (direct --prep), archetype prototypes plus design canon (prototype --prep), and asset…

Design & Media curated No risky patterns found
Match 7.7
Skill extract adobe/skills

Crawl an existing website (capped, multi-page) and seed stardust/current/ with PRODUCT.md, DESIGN.md, DESIGN.json, a per-page inventory, and the consolidated brand surface — the captured design system, palette,…

Design & Media curated Worth a look
Match 11.6
Skill link-rot-scanner adobe/skills

Crawl and validate all internal and external links across an AEM Edge Delivery Services site. Uses the query index or sitemap to discover pages, extracts links from .plain.html renditions, checks HTTP status codes, and…

Research & Web curated No risky patterns found
Match 7.7
Skill managing-data-catalog aws/agent-toolkit-for-aws

…an asset, create a data product, run the data source crawl, delete a catalog asset, attach a conventions document to a table as a catalog skill asset, which assets in my project are missing descriptions or owners…

Data & Databases curated No risky patterns found
Match 7.7
Skill gaia-bot-curate gaia-research/gaia-skill-tree

Curate Gaia bot crawler branches into real-skill catalog entries and clean up processed bot/* branches. Use when the user says: "process bot branches", "curate the crawl output", "clean up bot proposals", "run bot…

Research & Web No risky patterns found
Match 11.6
Skill firecrawl-search firecrawl/cli

Find web sources with query-relevant page excerpts and optional full-page content, and discover workflows, data APIs, and indexes. Use for web research or finding structured records, listings, transcripts, and datasets.…

Research & Web 85k installs No risky patterns found
Match 3.2
Skill firecrawl-scrape firecrawl/cli

Read a known webpage or execute a discovered workflow or data-provider capability. Use for page content or structured results once the URL or tool is selected.

Research & Web 84k installs No risky patterns found
Match 3.2
Skill firecrawl firecrawl/cli

Any live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or…

Development 105k installs Worth a look
Match 3.3
Skill just-scrape scrapegraphai/just-scrape

Search, scrape, crawl, extract structured data, and monitor web pages via the ScrapeGraph AI CLI. Use when the user asks to search the web, scrape a webpage, grab content from a URL, extract JSON from a site, crawl…

Research & Web 245k installs Worth a look
Match 13.1
Skill seo-audit coreyhaines31/marketingskills

…"Google update hit me," "page speed," "core web vitals," "crawl errors," "indexing issues," "stuck on page 2," "striking distance keywords," "title tag rewrite," "low CTR," "local SEO," "Google Business Profile," or…

Security 256k installs No risky patterns found
Match 9.2
XCrawl logo
XCrawl xcrawl.ai

Extract structured JSON, Markdown, and SERP data from any website with XCrawl, the AI-ready web scraping API with built-in proxies, crawling, and automation.

Freemium GEO & SEO API Online · checked 1d ago
Match 6.6
Crawlforge logo
Crawlforge crawlforge.dev

MCP server with 31 web scraping tools for AI agents. Turn any URL into clean markdown or JSON. Start free with 1,000 credits, no credit card required.

Coding API Open source Online · checked 7d ago
Match 10.2
Salience logo
Salience salience.com

Search and AI crawler intelligence, verified from your server logs. Salience shows what Googlebot, GPTBot, ClaudeBot and 200+ crawlers requested, what they got back and where they wasted crawl budget.

GEO & SEO API Online · checked 1d ago
Match 8.3