Skip to content

Releases: capt-marbles/firecrawl

Firecrawl v1.0.0

30 Jan 16:06
@capt-marbles capt-marbles
v1.0.0
e064df6

Choose a tag to compare

View all tags
Firecrawl v1.0.0 Latest
Latest

Web Scraping and Crawling with Firecrawl API 🕷️

Powerful web scraping, crawling, and data extraction using the Firecrawl API.

Features

📄 Markdown Conversion

  • Convert any webpage to clean markdown
  • Handles JavaScript-rendered content
  • Skip navigation and footer (--main-only)
  • Perfect for content analysis

📸 Screenshots

  • Full-page screenshots of any URL
  • High-quality PNG output
  • Capture dynamic content

🔍 Structured Data Extraction

  • Define JSON schemas for extraction
  • Pull specific fields from pages
  • AI-powered extraction with prompts
  • Perfect for product data, prices, etc.

🌐 Web Search

  • Search the web and scrape results
  • Get content from multiple sources
  • Configurable result limits

🕷️ Documentation Crawling

  • Crawl entire documentation sites
  • Perfect for training LLMs on frameworks
  • Save to local directory
  • Set crawl limits to control costs

🗺️ Site Mapping

  • Discover all URLs on a website
  • Search within URLs
  • Plan scraping strategy

Usage Examples

Markdown Conversion

python3 fc.py markdown "https://blog.example.com/post"

Screenshots

python3 fc.py screenshot "https://example.com" -o screenshot.png

Data Extraction

# Define schema
echo '{"type": "object", "properties": {"title": {"type": "string"}, "price": {"type": "number"}}}' > schema.json

# Extract data
python3 fc.py extract "https://example.com/product" --schema schema.json

Documentation Crawling

python3 fc.py crawl "https://docs.example.com" --limit 30 --output ./docs

Site Mapping

python3 fc.py map "https://example.com" --limit 100

Perfect For

  • Content research and aggregation
  • Price monitoring and tracking
  • Structured data extraction
  • LLM training data (RAG)
  • Competitive analysis
  • Documentation archiving
  • Website monitoring

Requirements

  • Python 3.6+
  • Firecrawl API key from firecrawl.dev
  • pip3 install firecrawl

Pricing

  • Free Tier: 500 credits
  • Cost: 1 credit per page/screenshot/query
  • Set limits to control costs

Perfect for OpenClaw/Clawdbot web automation and data gathering!

Assets 2
Loading