Meliscraper

Table of Contents

• About
• Requirements
• Getting Started
  • Configuring
    • .env
• Usage
  • Starting
  • Routes

About

Meliscraper is a web scraping API designed to monitor and track Mercado Libre's best-selling products. It captures a complete picture of the best sellers across all categories, storing them in a structured database.

Key features:

• Flexible Snapshot Retrieval: Get today's full snapshot or list all available snapshots summary.
• Category-based Navigation: List all available categories and retrieve products filtered by category slug.
• Historical Data Tracking: Query snapshots by specific dates.
• OpenAPI Documentation: Full Swagger UI documentation with examples and error responses.

Requirements:

For Docker (Recommended):

• Docker & Docker Compose

For Local Development:

• Java 21+
• Maven 3.9+
• PostgreSQL

Getting Started

Configuring

.env

Using Docker, .env file is optional since default values exist in Docker Compose file. For local development without Docker, you must create the PostgreSQL database manually and set POSTGRES_USER and POSTGRES_PASSWORD environment variables for PostgreSQL authentication. Other parameters have sensible defaults.

Rename .env.example to .env and modify variables according to your needs.

Variable	For Docker	For Local Development	Description
PORT	Optional (Default: "8080")	Optional (Default: "8080")	Server port
POSTGRES_USER	Optional (Default: "admin")	Required	PostgreSQL username
POSTGRES_PASSWORD	Optional (Default: "password")	Required	PostgreSQL password
POSTGRES_DB	Optional (Default: "meliscraper")	Optional (Default: "meliscraper")	PostgreSQL database

Usage

Starting

For the fastest setup, it is recommended to use Docker Compose to start the app and its services:

# Run docker compose command to start all services
$ docker compose up -d --build

Access the application at http://localhost:8080/docs (or the port you configured).

Routes

Route	HTTP Method	Params	Description	Auth Method
/docs	GET	-	Swagger documentation	None
/api/v1/snapshots	GET	Query Parameters: • page - Page number (default: 0) • size - Page size (default: 10)	Get all snapshots summary sorted by date in ascending order	None
/api/v1/snapshots/today	GET	-	Get today's snapshot	None
/api/v1/snapshots/:date	GET	Path Parameters: • date - Snapshot date in yyyy-MM-dd format	Find snapshot by specific date	None
/api/v1/snapshots/:date	DELETE	Path Parameters: • date - Snapshot date in yyyy-MM-dd format	Delete snapshot by date	None
/api/v1/categories	GET	Query Parameters: • page - Page number (default: 0) • size - Page size (default: 10)	Get list of categories (name and slug only) sorted by name in ascending order	None
/api/v1/categories/:slug/products	GET	Path Parameters: • slug - Category slug identifier Query Parameters: • page - Page number (default: 0) • size - Page size (default: 10) • orderBy - Sort field (default: "name") • direction - Sort direction: ASC/DESC (default: "ASC")	List products by category slug	None

⬆ Back to the top