Skip to content

Refactor ORM Layer and Improve Docker Configuration#31

Merged
Frankoul merged 4 commits intoembyplus:mainfrom
akaBoyLovesToCode:sqlalchemy
Mar 20, 2025
Merged

Refactor ORM Layer and Improve Docker Configuration#31
Frankoul merged 4 commits intoembyplus:mainfrom
akaBoyLovesToCode:sqlalchemy

Conversation

@akaBoyLovesToCode
Copy link
Contributor

@akaBoyLovesToCode akaBoyLovesToCode commented Mar 19, 2025

Major: Refactor ORM Layer and Improve Docker Configuration

This PR makes significant changes to enhance our codebase in two key areas:

  1. Database Layer Refactoring: Removes dependency on py_tools in favor of native SQLAlchemy
  2. Docker Configuration: Improves build reliability and resolves dependency issues

Changes

refactor(database): replace py_tools dependencies with native SQLAlchemy implementation

  • Create new database.py module with SQLAlchemy async pattern
  • Replace DBManager with DbOperations class for CRUD operations
  • Replace BaseOrmTable with SQLAlchemy declarative_base
  • Replace BaseOrmTableWithTS with timestamp columns implementation
  • Update application entry point with new database initialization flow
  • Refactor user_service.py to use repository pattern instead of ORM classes
  • Fix transaction handling with proper session management
  • Replace bulk operations with individual repository methods
  • Add proper error handling and logging in transactions
  • Update model updates to use repository methods instead of direct attribute changes
  • Implement get_session generator for session management
  • Improve code organization by moving repository code from app.py to model files

chore: remove requirements.txt that is replaced by pyproject.toml and uv.lock

fix(docker): update base image from alpine to slim for better compatibility

  • Change FROM python:3.12-alpine to python:3.12-slim
  • Maintain dependency installation through uv package manager
  • Preserve environment variables and docker configuration
  • Keep the application entrypoint using uv for running Python code

This change addresses potential compatibility issues with Alpine Linux
while maintaining a relatively small image footprint with the slim variant.

feat(docker): improve container build with proper Python dependency support

  • Replace alpine base image with python:3.12-slim for better compatibility
  • Fix PATH configuration to make uv available without source command
  • Add necessary development packages (libc6-dev, python3-dev) for C extensions
  • Use bash explicitly to support shell operations in build process
  • Improve dependency installation process using uv package manager
  • Organize package installation list for better readability
  • Ensure proper cleanup of build dependencies after installation
  • Remove redundant uv cache clean command to simplify build process
  • Maintain environment variables and Docker configuration

This change resolves compilation errors with tgcrypto and other Python packages
requiring C extensions, while keeping the Docker image as lightweight as possible.

Impact

  • Developer Experience: Simplified ORM layer with industry-standard patterns
  • Maintainability: Removed dependency on custom py_tools library
  • Build Reliability: Fixed Docker build process to properly handle all dependencies
  • Deployment: Improved container image with better compatibility and proper C extension support

Testing Done

  • Verified database operations work correctly with new SQLAlchemy implementation
  • Successfully built Docker image with all dependencies properly installed
  • Tested application startup and database connectivity in containerized environment

好的,这是对pull request总结的中文翻译:

Sourcery 总结

重构 ORM 层以使用原生 SQLAlchemy,通过移除对自定义 py_tools 库的依赖,提高可维护性和开发者体验。 此外,改进了 Docker 配置,以增强构建可靠性并解决依赖问题,确保正确处理 C 扩展并保持轻量级镜像。

增强功能:

  • 将基于 py_tools 的自定义 ORM 实现替换为原生 SQLAlchemy 实现,包括用于数据访问的存储库类。
  • 改进了使用 get_session 生成器的会话管理。
  • 通过将存储库代码从 app.py 移动到模型文件来增强代码组织。

测试:

  • 验证数据库操作在新 SQLAlchemy 实现下可以正常工作。
  • 成功构建 Docker 镜像,并正确安装所有依赖项。
  • 在容器化环境中测试应用程序启动和数据库连接。
Original summary in English

Summary by Sourcery

Refactors the ORM layer to use native SQLAlchemy, improving maintainability and developer experience by removing the dependency on the custom py_tools library. Also, improves the Docker configuration to enhance build reliability and resolve dependency issues, ensuring proper handling of C extensions and maintaining a lightweight image.

Enhancements:

  • Replaces the custom ORM implementation based on py_tools with a native SQLAlchemy implementation, including repository classes for data access.
  • Improves session management using a get_session generator.
  • Enhances code organization by moving repository code from app.py to model files.

Tests:

  • Verifies database operations work correctly with the new SQLAlchemy implementation.
  • Successfully builds the Docker image with all dependencies properly installed.
  • Tests application startup and database connectivity in a containerized environment.

…emy implementation

- Create new database.py module with SQLAlchemy async pattern
- Replace DBManager with DbOperations class for CRUD operations
- Replace BaseOrmTable with SQLAlchemy declarative_base
- Replace BaseOrmTableWithTS with timestamp columns implementation
- Update application entry point with new database initialization flow
- Refactor user_service.py to use repository pattern instead of ORM classes
- Fix transaction handling with proper session management
- Replace bulk operations with individual repository methods
- Add proper error handling and logging in transactions
- Update model updates to use repository methods instead of direct attribute changes
- Implement get_session generator for session management
- Improve code organization by moving repository code from app.py to model files
…bility

- Change FROM python:3.12-alpine to python:3.12-slim
- Maintain dependency installation through uv package manager
- Preserve environment variables and docker configuration
- Keep the application entrypoint using uv for running Python code

This change addresses potential compatibility issues with Alpine Linux
while maintaining a relatively small image footprint with the slim variant.
…upport

- Replace alpine base image with python:3.12-slim for better compatibility
- Fix PATH configuration to make uv available without source command
- Add necessary development packages (libc6-dev, python3-dev) for C extensions
- Use bash explicitly to support shell operations in build process
- Improve dependency installation process using uv package manager
- Organize package installation list for better readability
- Ensure proper cleanup of build dependencies after installation
- Remove redundant uv cache clean command to simplify build process
- Maintain environment variables and Docker configuration

This change resolves compilation errors with tgcrypto and other Python packages
requiring C extensions, while keeping the Docker image as lightweight as possible.
@sourcery-ai
Copy link

sourcery-ai bot commented Mar 19, 2025

## Sourcery 评审者指南

此拉取请求重构了 ORM 层以使用原生 SQLAlchemy,改进了 Docker 配置以提高构建可靠性和依赖管理,并删除了 `requirements.txt` 文件。

#### 创建新用户的时序图

```mermaid
sequenceDiagram
    participant US as UserService
    participant UR as UserRepository
    participant EAPI as EmbyApi
    participant CR as ConfigRepository

    US->>UR: get_by_telegram_id(telegram_id)
    alt User exists
        UR-->>US: User
    else User does not exist
        US->>UR: create_user(telegram_id, is_admin, telegram_name)
        UR-->>US: User
    end

User 模型更新后的类图

classDiagram
    class User {
        - telegram_id: int
        - is_admin: bool
        - telegram_name: str
        - emby_id: str
        - emby_name: str
        - enable_register: bool
        - is_whitelist: bool
        - ban_time: int
        - reason: str
        + check_create_invite_code(): bool
        + check_create_whitelist_code(): bool
        + check_use_redeem_code(): bool
        + check_use_whitelist_code(): bool
        + is_emby_baned(): bool
        + emby_ban_info(): tuple[int, str]
    }
    class BaseModelWithTS {
        - created_at: DateTime
        - updated_at: DateTime
    }
    User --|> BaseModelWithTS : extends
    note for User "使用 UserRepository 类替换 UserOrm 类以进行数据访问"
Loading

Config 模型更新后的类图

classDiagram
    class Config {
        - total_register_user: int
        - register_public_user: int
        - register_public_time: int
    }
    class BaseModelWithTS {
        - created_at: DateTime
        - updated_at: DateTime
    }
    Config --|> BaseModelWithTS : extends
    note for Config "使用 ConfigRepository 类替换 ConfigOrm 类以进行数据访问"
Loading

InviteCode 模型更新后的类图

classDiagram
    class InviteCode {
        - code: str
        - telegram_id: int
        - code_type: Enum
        - is_used: bool
        - used_time: int
        - used_user_id: int
    }
    class BaseModelWithTS {
        - created_at: DateTime
        - updated_at: DateTime
    }
    InviteCode --|> BaseModelWithTS : extends
    note for InviteCode "使用 InviteCodeRepository 类替换 InviteCodeOrm 类以进行数据访问"
Loading

文件级别变更

变更 详情 文件
使用原生 SQLAlchemy 替换 py_tools 库以进行数据库交互。
  • 引入了一个新的 database.py 模块,其中包含 SQLAlchemy 异步模式。
  • 使用 DbOperations 类替换 DBManager 类以进行 CRUD 操作。
  • 使用 SQLAlchemy 声明性基类替换 BaseOrmTable
  • 使用 BaseModelWithTS 实现时间戳列。
  • 使用新的数据库初始化流程更新应用程序入口点。
  • 重构 user_service.py 以使用存储库模式而不是 ORM 类。
  • 通过使用 get_session 生成器的适当会话管理来改进事务处理。
  • 使用单独的存储库方法替换批量操作。
  • 在事务中添加了适当的错误处理和日志记录。
  • 更新了模型更新以使用存储库方法而不是直接属性更改。
  • 通过将存储库代码从 app.py 移动到模型文件来改进代码组织。
services/user_service.py
models/config_model.py
app.py
models/invite_code_model.py
models/user_model.py
models/__init__.py
models/database.py
改进了 Docker 配置,以提高构建可靠性和依赖管理。
  • 使用精简变体替换 Alpine 基础镜像,以获得更好的兼容性。
  • 通过 uv 包管理器维护依赖项安装。
  • 保留环境变量和 Docker 配置。
  • 修复了 PATH 配置,以使 uv 在不使用 source 的情况下可用。
  • 添加了 C 扩展所需的必要开发包(libc6-devpython3-dev)。
  • 显式使用 bash 以支持构建过程中的 shell 操作。
  • 使用 uv 包管理器改进了依赖项安装过程。
  • 组织了包安装列表,以提高可读性。
  • 确保在安装后正确清理构建依赖项。
  • 删除了冗余的 uv cache clean 命令以简化构建过程。
Dockerfile
删除了 requirements.txt 文件。
  • 删除了 requirements.txt,因为它已被 pyproject.tomluv.lock 替换。
requirements.txt

提示和命令

与 Sourcery 互动

  • 触发新的审查: 在拉取请求上评论 @sourcery-ai review
  • 继续讨论: 直接回复 Sourcery 的审查评论。
  • 从审查评论生成 GitHub issue: 通过回复审查评论,要求 Sourcery 从审查评论创建一个 issue。您也可以回复审查评论并使用 @sourcery-ai issue 从中创建一个 issue。
  • 生成拉取请求标题: 在拉取请求标题中的任何位置写入 @sourcery-ai 以随时生成标题。您也可以在拉取请求上评论 @sourcery-ai title 以随时(重新)生成标题。
  • 生成拉取请求摘要: 在拉取请求正文中的任何位置写入 @sourcery-ai summary 以随时在您想要的位置生成 PR 摘要。您也可以在拉取请求上评论 @sourcery-ai summary 以随时(重新)生成摘要。
  • 生成审查者指南: 在拉取请求上评论 @sourcery-ai guide 以随时(重新)生成审查者指南。
  • 解决所有 Sourcery 评论: 在拉取请求上评论 @sourcery-ai resolve 以解决所有 Sourcery 评论。如果您已经解决了所有评论并且不想再看到它们,这将非常有用。
  • 驳回所有 Sourcery 审查: 在拉取请求上评论 @sourcery-ai dismiss 以驳回所有现有的 Sourcery 审查。如果您想从新的审查开始,这将特别有用 - 不要忘记评论 @sourcery-ai review 以触发新的审查!
  • 为 issue 生成行动计划: 在 issue 上评论 @sourcery-ai plan 以为其生成行动计划。

自定义您的体验

访问您的 仪表板 以:

  • 启用或禁用审查功能,例如 Sourcery 生成的拉取请求摘要、审查者指南等。
  • 更改审查语言。
  • 添加、删除或编辑自定义审查说明。
  • 调整其他审查设置。

获得帮助

```
Original review guide in English

Reviewer's Guide by Sourcery

This pull request refactors the ORM layer to use native SQLAlchemy, improves the Docker configuration for better build reliability and dependency management, and removes the requirements.txt file.

Sequence diagram for creating a new user

sequenceDiagram
    participant US as UserService
    participant UR as UserRepository
    participant EAPI as EmbyApi
    participant CR as ConfigRepository

    US->>UR: get_by_telegram_id(telegram_id)
    alt User exists
        UR-->>US: User
    else User does not exist
        US->>UR: create_user(telegram_id, is_admin, telegram_name)
        UR-->>US: User
    end
Loading

Updated class diagram for User model

classDiagram
    class User {
        - telegram_id: int
        - is_admin: bool
        - telegram_name: str
        - emby_id: str
        - emby_name: str
        - enable_register: bool
        - is_whitelist: bool
        - ban_time: int
        - reason: str
        + check_create_invite_code(): bool
        + check_create_whitelist_code(): bool
        + check_use_redeem_code(): bool
        + check_use_whitelist_code(): bool
        + is_emby_baned(): bool
        + emby_ban_info(): tuple[int, str]
    }
    class BaseModelWithTS {
        - created_at: DateTime
        - updated_at: DateTime
    }
    User --|> BaseModelWithTS : extends
    note for User "Replaces UserOrm class with UserRepository class for data access"
Loading

Updated class diagram for Config model

classDiagram
    class Config {
        - total_register_user: int
        - register_public_user: int
        - register_public_time: int
    }
    class BaseModelWithTS {
        - created_at: DateTime
        - updated_at: DateTime
    }
    Config --|> BaseModelWithTS : extends
    note for Config "Replaces ConfigOrm class with ConfigRepository class for data access"
Loading

Updated class diagram for InviteCode model

classDiagram
    class InviteCode {
        - code: str
        - telegram_id: int
        - code_type: Enum
        - is_used: bool
        - used_time: int
        - used_user_id: int
    }
    class BaseModelWithTS {
        - created_at: DateTime
        - updated_at: DateTime
    }
    InviteCode --|> BaseModelWithTS : extends
    note for InviteCode "Replaces InviteCodeOrm class with InviteCodeRepository class for data access"
Loading

File-Level Changes

Change Details Files
Replaced py_tools library with native SQLAlchemy for database interactions.
  • Introduced a new database.py module with SQLAlchemy async patterns.
  • Replaced DBManager with DbOperations class for CRUD operations.
  • Replaced BaseOrmTable with SQLAlchemy declarative base.
  • Implemented timestamp columns using BaseModelWithTS.
  • Updated application entry point with new database initialization flow.
  • Refactored user_service.py to use repository pattern instead of ORM classes.
  • Improved transaction handling with proper session management using a get_session generator.
  • Replaced bulk operations with individual repository methods.
  • Added proper error handling and logging in transactions.
  • Updated model updates to use repository methods instead of direct attribute changes.
  • Improved code organization by moving repository code from app.py to model files.
services/user_service.py
models/config_model.py
app.py
models/invite_code_model.py
models/user_model.py
models/__init__.py
models/database.py
Improved Docker configuration for better build reliability and dependency management.
  • Replaced the Alpine base image with a slim variant for better compatibility.
  • Maintained dependency installation through the uv package manager.
  • Preserved environment variables and Docker configuration.
  • Fixed PATH configuration to make uv available without sourcing.
  • Added necessary development packages (libc6-dev, python3-dev) for C extensions.
  • Used bash explicitly to support shell operations in the build process.
  • Improved the dependency installation process using the uv package manager.
  • Organized the package installation list for better readability.
  • Ensured proper cleanup of build dependencies after installation.
  • Removed the redundant uv cache clean command to simplify the build process.
Dockerfile
Removed requirements.txt file.
  • Removed requirements.txt as it is replaced by pyproject.toml and uv.lock.
requirements.txt

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Qubbby - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Consider adding a health check endpoint to the Dockerfile for improved container orchestration.
  • The database initialization logic in app.py and models/database.py is duplicated; consolidate it into a single function.
Here's what I looked at during the review
  • 🟡 General issues: 1 issue found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟡 Complexity: 1 issue found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.


class ConfigOrm(DBManager):
orm_table = Config
class ConfigRepository:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: Duplicate invite code repository methods in ConfigRepository.

ConfigRepository is intended to manage Config objects only. It contains methods for creating and updating invite codes, which are also implemented in InviteCodeRepository. Consolidating all invite code operations in InviteCodeRepository would reduce duplication and ease maintenance.

await session.commit()
return new_user
# Use manual session management instead of transaction context manager
async for session in get_session():
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (complexity): Consider encapsulating the repeated session management logic in a reusable async context manager to reduce nesting and code duplication in transaction handling.

Consider encapsulating repeated session management logic in a reusable async context manager. This would reduce nesting and duplication of try/except blocks while keeping functionality intact. For example, you can create a helper like:

from contextlib import asynccontextmanager

@asynccontextmanager
async def transaction_context(get_session_fn):
    async for session in get_session_fn():
        try:
            yield session
            await session.commit()
        except Exception:
            await session.rollback()
            raise

Then in your methods, refactor to use the context manager:

async def emby_create_user(self, telegram_id: int, username: str, password: str) -> User:
    # ... other logic ...
    async with transaction_context(get_session) as session:
        if not user.enable_register and emby_config.register_public_user > 0:
            emby_config.register_public_user -= 1
        emby_config.total_register_user += 1
        await ConfigRepository.update_config(
            emby_config.id,
            register_public_user=emby_config.register_public_user,
            total_register_user=emby_config.total_register_user,
        )
        new_user = await self._emby_create_user(telegram_id, username, password)
        # Persist new user and updated config if required
    return new_user

Apply similar refactoring to the redeem_code method. This reduces nesting and centralizes the transaction handling.

@Frankoul Frankoul merged commit 312000e into embyplus:main Mar 20, 2025
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants