Skip to content

CATEGORIES_IMPORT_FEATURE.md

Latest commit

 

History

History
270 lines (201 loc) · 6.34 KB

CATEGORIES_IMPORT_FEATURE.md

File metadata and controls

270 lines (201 loc) · 6.34 KB

Categories CSV Import Feature

Date: October 26, 2025 Status: ✅ Complete

Overview

Added a complete CSV import system for categories, allowing users to bulk import categories, groups, and types from a CSV file.

Features

1. Smart Import System

  • Auto-creates categories_type and categories_group if they don't exist
  • Skips duplicates - won't create duplicate categories
  • Validates CSV format and provides clear error messages
  • Session-based - automatically associates with logged-in user

2. User-Friendly Interface

  • Drag and drop file upload
  • Browse button for traditional file selection
  • Progress indicator during upload
  • Detailed results showing:
    • Categories created
    • Duplicates skipped
    • Groups processed
    • Types processed

3. CSV Format Support

Expected format:

categories,categories_group,categories_type
Groceries,Food & Dining,Expense
Salary,Income,Income
Gas & Fuel,Auto & Transportation,Expense

Files Created/Modified

Backend (API)

api/categories/controllers.py

Added:

  • New endpoint: /api/categories/csv_import (POST)
  • Imports: os, csv, session, secure_filename, Config
  • Full CSV processing logic with validation

Features:

  • Validates CSV headers
  • Creates types and groups automatically
  • Checks for duplicate categories
  • Returns detailed import statistics
  • Cleans up uploaded file after processing

Frontend (Web)

app/categories/controllers.py

Added:

  • New route: /categories/import (GET)
  • Renders the import page

app/templates/categories/import.html

Created new file:

  • Beautiful upload interface with drag & drop
  • Clear instructions and CSV format examples
  • Progress bar during upload
  • Detailed success/error messages
  • Statistics display after import

app/static/js/categories/import.js

Created new file:

  • Drag and drop file handling
  • File validation (CSV only)
  • AJAX upload to API
  • Progress tracking
  • Result display with statistics

app/templates/categories/index.html

Modified:

  • Added import button (upload icon) next to add category button
  • Links to /categories/import

Usage

For Users

  1. Navigate to Categories

    • Go to /categories
    • Click the upload icon button

  2. Upload CSV

    • Drag and drop your CSV file OR
    • Click "Browse File" to select

  3. Review Results

    • See how many categories were created
    • See how many duplicates were skipped
    • See groups and types processed

For Developers

API Endpoint:

POST /api/categories/csv_import
Content-Type: multipart/form-data

# Form data:
file: <CSV file>

Response:

{
  "message": "Categories imported successfully",
  "categories_created": 131,
  "categories_skipped": 0,
  "types_processed": 3,
  "groups_processed": 25
}

CSV Sample Data

The repository includes a sample file at /data/categories_data.csv with:

  • 131 categories
  • 25 groups (Food & Dining, Auto & Transportation, Bills & Utilities, etc.)
  • 3 types (Income, Expense, Transfer)

Technical Details

Import Process Flow

  1. Validate user is authenticated
  2. Check file is provided and is CSV
  3. Save file to uploads folder temporarily
  4. Parse CSV and validate headers
  5. For each row:
    • Get or create categories_type
    • Get or create categories_group
    • Check if category exists (by name + group + type)
    • Skip if exists, create if new
  6. Clean up - delete uploaded file
  7. Return statistics

Database Efficiency

  • Uses query.filter_by().first() to check existing records
  • Caches type and group IDs in dictionaries during import
  • Only one query per unique type/group name
  • Prevents N+1 query problems

Security

  • ✅ Session-based authentication required
  • ✅ File extension validation (.csv only)
  • ✅ Secure filename handling with secure_filename()
  • ✅ Temporary file cleanup after processing
  • ✅ User ID from session (can't import for other users)

Testing

Test the Import

  1. Restart Flask app to load new endpoints
  2. Navigate to http://localhost:5000/categories
  3. Click upload icon button
  4. Upload the file /data/categories_data.csv
  5. Verify results show imported categories

Expected Results

With the sample categories_data.csv:

  • ✅ 131 categories created
  • ✅ 0 duplicates skipped (first import)
  • ✅ 25 groups processed
  • ✅ 3 types processed

Second import:

  • ✅ 0 categories created
  • ✅ 131 duplicates skipped
  • ✅ 25 groups processed
  • ✅ 3 types processed

Error Handling

The system handles:

Error Response
No file uploaded 400: No file provided
Empty filename 400: No file selected
Non-CSV file 400: File must be a CSV
Missing CSV headers 400: CSV must have headers: ...
Invalid CSV format 500: Error processing CSV: ...
Not authenticated 401: User not authenticated

Future Enhancements

Potential improvements:

  1. Bulk Edit - Update existing categories from CSV
  2. Export - Download current categories as CSV
  3. Template - Download blank CSV template
  4. Preview - Show preview before importing
  5. Validation - Client-side CSV validation before upload
  6. Mapping - Allow custom column mapping
  7. Undo - Ability to undo an import

API Documentation

POST /api/categories/csv_import

Description: Import categories from CSV file

Authentication: Required (session-based)

Content-Type: multipart/form-data

Parameters:

  • file (required): CSV file with headers: categories, categories_group, categories_type

Success Response (201):

{
  "message": "Categories imported successfully",
  "categories_created": 131,
  "categories_skipped": 0,
  "types_processed": 3,
  "groups_processed": 25
}

Error Responses:

  • 400: Invalid request (missing file, wrong format, invalid CSV)
  • 401: Not authenticated
  • 500: Server error during processing

Summary

✅ Complete CSV import system

  • Backend API endpoint
  • Frontend upload interface
  • Drag & drop support
  • Smart duplicate handling
  • Auto-creation of types and groups
  • Detailed import statistics
  • User-friendly interface

The feature is ready to use! Just restart your Flask application and navigate to /categories to see the new upload button.