Firecrawl Simple is a stripped down and stable version of firecrawl optimized for self-hosting and ease of contribution. Billing logic and AI features

Search code, repositories, users, issues, pull requests...

submited by
Style Pass
2024-11-05 23:30:13

Firecrawl Simple is a stripped down and stable version of firecrawl optimized for self-hosting and ease of contribution. Billing logic and AI features are completely removed.

playwright is replaced with puppeteer-cluster and puppeteer-extra's stealth plugins such that fire-engine and scrapingbee are not required for guarded pages. Further, a 2captcha token and proxy credentials may be included as ENV's for maximum stealthiness. In the near-term future, the goal is to switch over to hero since puppeteer-extra is no longer maintained

Only the v1 /scrape, /crawl/{id}, and /crawl routes are supprted in firecrawl simple, see the openapi spec here. Also, creditsUsed has been removed from the API response on the /crawl/{id} route.

This is a lot to maintain by ourselves and we are actively looking for others who would like to help. There are paid part-time maintainer positions available. We currently have bounties on a couple of issues, but would like someone intersted in being an active maintainer longer-term.

This repository is in development, and we're still integrating custom modules into the mono repo. It's not fully ready for self-hosted deployment yet, but you can run it locally.

Leave a Comment