portfolio
DeepCrawl-Chat: An Intelligent Web Crawler and RAG System
- FastAPI
- LangChain
- FAISS
- Docker
- Uvicorn
- NVIDIA AI
- Hugging Face
- Groq
- SQLAlchemy
- Redis
Engineered ‘DeepCrawl-Chat,’ a comprehensive system for advanced web crawling and conversational search. The system allows users to crawl specified websites, extracts and processes the content, and builds a queryable knowledge base. Users can then interactively query the crawled content using natural language through a RAG pipeline. This project involved designing a scalable crawling architecture and integrating it with modern LLMs for accurate, context-aware responses.