feat: Implement offline-first API integration for finance tracking app

[nfebe]

Requirements:

1. Offline-First Data Storage:

   • Utilize SQLite (via the sqflite package) for local data storage.
   • Implement data models (e.g., Transaction, Category, Party, Transfer, Savings) that include fields for createdAt, updatedAt, deletedAt, and lastSyncedAt.
   • Ensure all data operations (create, update, delete) are performed on the local database first.

2. Generic Synchronization System:

   • Implement a generic synchronization system that can handle multiple data models.
   • Use a synchronization queue (sync_queue table) to store pending API requests.
   • The sync_queue table should include fields for modelName, operation (create, update, delete), data (JSON representation of the data), and createdAt.

3. API Integration:

   • Integrate with the provided API (OpenAPI documentation) for data synchronization.
   • Implement API services for each data model to perform CRUD operations.
   • Handle API responses and update the local database accordingly.

4. Synchronization Logic:

   • Implement a synchronization service that periodically processes the synchronization queue.
   • Handle potential errors during synchronization (e.g., network issues, API errors).
   • Implement logic to pull the latest data from the server, and update the local database.
   • Ensure that the sync process is efficient, and does not cause excessive network traffic.

5. Dart/Flutter Implementation:

   • Use Dart and Flutter for the implementation.
   • Follow clean architecture principles to ensure maintainability and testability.
   • Provide clear and concise code with appropriate comments.

Implementation Steps:

1. Set up SQLite Database:

   • Initialize the SQLite database using the sqflite package.
   • Create tables for each data model (e.g., transactions, categories, parties).
   • Create the sync_queue table.

2. Implement Data Models:

   • Create Dart classes for each data model (e.g., Transaction, Category).
   • Include fields for createdAt, updatedAt, deletedAt, and lastSyncedAt.
   • Implement toMap, fromJson, and toJson methods for each model.

3. Implement Database Helper:

   • Create a DatabaseHelper class to handle database operations.
   • Implement generic methods for insert, update, delete, and query.
   • Implement methods for enqueueing and processing synchronization queue items.

4. Implement API Service:

   • Create an ApiService class to handle API requests.
   • Implement API methods for each data model (e.g., getTransactions, createTransaction).
   • Handle API responses and convert them to Dart objects.

5. Implement Synchronization Service:

   • Create a SyncService class to handle data synchronization.
   • Implement a method to process the synchronization queue.
   • Implement error handling and retry mechanisms.
   • Implement logic to pull data from the server, and update the local database.
   • Add functions to handle each model (e.g. _processTransactionSyncItem, _processCategorySyncItem).

6. Integrate with Flutter UI:

   • Integrate the synchronization service with the Flutter UI.
   • Trigger synchronization when network connectivity is restored or at regular intervals.
   • Display appropriate loading indicators and error messages.

Code Structure:

lib/
├── models/
│   ├── transaction.dart
│   ├── category.dart
│   ├── party.dart
│   └── ...
├── data/
│   ├── database_helper.dart
│   ├── api_service.dart
│   ├── sync_service.dart
│   ├── sync_handlers/
│   │   ├── model_sync_handler.dart
│   │   ├── transaction_sync_handler.dart
│   │   ├── category_sync_handler.dart (if you add category model)
│   │   ├── party_sync_handler.dart (if you add party model)
│   │   └── ...

[nfebe]

Sample structure of SyncService (with use of other sample classes in file)

import 'dart:async';
import 'dart:convert';
import 'package:http/http.dart' as http;
import 'package:sqflite/sqflite.dart';
import 'package:path/path.dart';

import 'transaction.dart';
import 'sync_queue_item.dart';
import 'database_helper.dart';
import 'api_service.dart';

abstract class ModelSyncHandler<T> {
  Future<T> create(T item);
  Future<T> update(T item);
  Future<void> delete(int id);
  T fromJson(Map<String, dynamic> json);
  Map<String, dynamic> toJson(T item);
  String getTableName();
  Future<void> syncFromServer();
}

class TransactionSyncHandler implements ModelSyncHandler<Transaction> {
  final ApiService apiService;
  final DatabaseHelper dbHelper;

  TransactionSyncHandler(this.apiService, this.dbHelper);

  @override
  Future<Transaction> create(Transaction item) async {
    final syncedTransaction = await apiService.create(
      getTableName(),
      fromJson,
      toJson(item),
    );
    await dbHelper.update(getTableName(), syncedTransaction.toMap(), item.id!);
    return syncedTransaction;
  }

  @override
  Future<Transaction> update(Transaction item) async {
    final updatedTransaction = await apiService.update(
      getTableName(),
      item.id!,
      fromJson,
      toJson(item),
    );
    await dbHelper.update(getTableName(), updatedTransaction.toMap(), item.id!);
    return updatedTransaction;
  }

  @override
  Future<void> delete(int id) async {
    await apiService.delete(getTableName(), id);
    await dbHelper.delete(getTableName(), id);
  }

  @override
  Transaction fromJson(Map<String, dynamic> json) => Transaction.fromJson(json);

  @override
  Map<String, dynamic> toJson(Transaction item) => item.toJson();

  @override
  String getTableName() => 'transactions';

  @override
  Future<void> syncFromServer() async {
    final latestUpdate = await dbHelper.query(getTableName(), orderBy: 'updatedAt DESC', limit: 1);
    DateTime? lastSyncedTime;
    if (latestUpdate.isNotEmpty && latestUpdate[0]['updatedAt'] != null) {
      lastSyncedTime = DateTime.parse(latestUpdate[0]['updatedAt'] as String);
    }

    final allServerTransactions = await apiService.getList(
      getTableName(),
      fromJson,
      type: 'income',
      limit: 10000000,
    ) +
        await apiService.getList(
          getTableName(),
          fromJson,
          type: 'expense',
          limit: 10000000,
        );
    for (final transactionFromServer in allServerTransactions) {
      final localTransactions = await dbHelper.query(getTableName(), where: 'id = ?', whereArgs: [transactionFromServer.id]);

      if (localTransactions.isNotEmpty) {
        final localTransaction = fromJson(localTransactions.first);
        if (localTransaction.updatedAt != transactionFromServer.updatedAt) {
          await dbHelper.update(getTableName(), transactionFromServer.toMap(), transactionFromServer.id!);
        }
      } else {
        await dbHelper.insert(getTableName(), transactionFromServer.toMap());
      }
    }
  }
}

class SyncService {
  final DatabaseHelper dbHelper = DatabaseHelper.instance;
  final ApiService apiService = ApiService();
  final Map<String, ModelSyncHandler> modelHandlers = {};

  SyncService() {
    modelHandlers['transactions'] = TransactionSyncHandler(apiService, dbHelper);
    // Add other model handlers here
  }

  Future<void> syncData() async {
    final queueItems = await dbHelper.getSyncQueueItems();
    for (final item in queueItems) {
      try {
        await _processSyncItem(item);
        await dbHelper.deleteSyncQueueItem(item.id!);
      } catch (e) {
        print('Sync error: $e');
      }
    }
    await _syncFromServer();
  }

  Future<void> _processSyncItem(SyncQueueItem item) async {
    final handler = modelHandlers[item.modelName];
    if (handler != null) {
      await _processModelItem(item, handler);
    } else {
      print('Unknown model: ${item.modelName}');
    }
  }

  Future<void> _processModelItem(SyncQueueItem item, ModelSyncHandler handler) async {
    final model = handler.fromJson(item.data);
    switch (item.operation) {
      case 'create':
        await handler.create(model);
        break;
      case 'update':
        await handler.update(model);
        break;
      case 'delete':
        await handler.delete(model.id!);
        break;
      default:
        print('Unknown operation: ${item.operation}');
    }
  }

  Future<void> _syncFromServer() async {
    for (final handler in modelHandlers.values) {
      await handler.syncFromServer();
    }
  }
}

[nfebe]

Generic API service (relies on the consistency of the backend)

import 'dart:convert';
import 'package:http/http.dart' as http;

class ApiService {
  final String baseUrl;

  ApiService({required this.baseUrl});

  Future<List<T>> getList<T>(String endpoint, T Function(Map<String, dynamic>) fromJson, {String? type, int? limit}) async {
    Uri uri = Uri.parse('$baseUrl/$endpoint');
    if(type != null || limit != null){
      uri = uri.replace(queryParameters: {
        if(type != null) 'type': type,
        if(limit != null) 'limit': limit.toString()
      });
    }

    final response = await http.get(uri);
    if (response.statusCode == 200) {
      final List<dynamic> data = json.decode(response.body);
      return data.map((json) => fromJson(json)).toList();
    } else {
      throw Exception('Failed to load $endpoint: ${response.statusCode}');
    }
  }

  Future<T> create<T>(String endpoint, T Function(Map<String, dynamic>) fromJson, Map<String, dynamic> data) async {
    final response = await http.post(
      Uri.parse('$baseUrl/$endpoint'),
      headers: {'Content-Type': 'application/json'},
      body: json.encode(data),
    );
    if (response.statusCode == 201) {
      return fromJson(json.decode(response.body));
    } else {
      throw Exception('Failed to create $endpoint: ${response.statusCode}');
    }
  }

  Future<T> update<T>(String endpoint, int id, T Function(Map<String, dynamic>) fromJson, Map<String, dynamic> data) async {
    final response = await http.put(
      Uri.parse('$baseUrl/$endpoint/$id'),
      headers: {'Content-Type': 'application/json'},
      body: json.encode(data),
    );
    if (response.statusCode == 200) {
      return fromJson(json.decode(response.body));
    } else {
      throw Exception('Failed to update $endpoint: ${response.statusCode}');
    }
  }

  Future<void> delete(String endpoint, int id) async {
    final response = await http.delete(Uri.parse('$baseUrl/$endpoint/$id'));
    if (response.statusCode != 200) {
      throw Exception('Failed to delete $endpoint: ${response.statusCode}');
    }
  }
}

Example:

import 'api_service.dart';
import '../models/transaction.dart'; // Adjust import path

void main() async {
  final apiService = ApiService(baseUrl: 'YOUR_API_BASE_URL'); // Replace with your API URL

  try {
    final transactions = await apiService.getList(
      'transactions',
      Transaction.fromJson,
      type: 'income',
      limit: 10,
    );

    final newTransaction = Transaction(
      type: 'expense',
      amount: 50.0,
      description: 'Groceries',
      createdAt: DateTime.now(),
    );

    final createdTransaction = await apiService.create(
      'transactions',
      Transaction.fromJson,
      newTransaction.toJson(),
    );

    final updateTransaction = Transaction(id: createdTransaction.id, type: 'expense', amount: 100, description: "Updated Groceries", createdAt: createdTransaction.createdAt, updatedAt: DateTime.now());

    final updatedTransaction = await apiService.update('transactions', updateTransaction.id!, Transaction.fromJson, updateTransaction.toJson());
    print(updatedTransaction);

    await apiService.delete('transactions', createdTransaction.id!);

  } catch (e) {
    print('API error: $e');
  }
}