Firecrawl is a web crawler and scrape, made for working with LLMs

Firecrawl can be self hosted using Docker. See the self-hosting Guide at:
https://docs.firecrawl.dev/contributing/self-host

It comes with an Integration for LangChain: https://docs.langchain.com/oss/javascript/integrations/document_loaders/web_loaders/firecrawl and for Python: https://docs.langchain.com/oss/python/integrations/providers/firecrawl