Overview
FireCrawl is a web scraping tool developed by MBO that converts website data into a clean markdown format suitable for large language models.
Role in this knowledge base
It is used to efficiently ingest and structure website content for RAG systems, reducing noise and preparing metadata.
Key facts
- It can scrape single URLs, crawl entire domains, or search across the web, outputting clean markdown with titles and images.