NicoScraper is a comprehensive .NET-based utility designed for scraping inventory and stock prices from various websites, initially focusing on QuitMed. It leverages a Blazor WebApp for data presentation and utilizes an actor system for periodic data scraping, ensuring up-to-date information is always available.
Note: This project was last verified to work on June 5, 2024. Due to the dynamic nature of web scraping, changes to target websites may require updates to the scraping logic.
- Data Scraping: Automated scraping of inventory and stock prices from designated websites.
- Blazor WebApp: A user-friendly interface for data visualization.
- Actor System: Utilizes Akka.NET to manage scraping processes efficiently.
- Docker Support: Includes Docker Compose for easy local setup.
- Azure Ready: Prepared Bicep files for potential Azure deployment.
- .NET 8.0
- Blazor
- Entity Framework
- Akka.NET
- Docker Compose
- Bicep
- .NET 8.0 SDK
- Docker and Docker Compose
- An IDE like Visual Studio or VS Code
- Clone the repository to your local machine.
- Navigate to the project directory.
- Use Docker Compose to build and run the application:
docker-compose -f infra/compose.yaml up --build
- Access the Blazor WebApp through your browser at the specified port.
- src/Quitmed-Scraper.Console/: Console application for background scraping tasks.
- src/Quitmed-Scraper.Database/: Entity Framework Core database project.
- src/Quitmed-Scraper.Library/: Core library containing the scraping logic.
- src/Quitmed-Scraper.WebApp/: Blazor WebApp for data presentation.
- infra/: Infrastructure as Code (IaC) files for Docker and Azure deployment.