Phase 2: Refactor feed management from jQuery to HTMX #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft

Copilot wants to merge 8 commits into master from copilot/refactor-feed-management-htmx

.github/copilot-instructions.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -39,7 +39,8 @@ The project follows a clean architecture pattern:
  
    - `make run` - Run the app without Docker

    - `make docker-run` - Run the app with Docker Compose

    - `make build` - Build the binary

    - `make run-test-server` - Run the app with mock feed server for testing (uses test database)

    - `make run-test-server` - Run the app configured for UI tests (port 3001, uses `data/test-dev.db`)

    - `make run-feed-provider` - Run the standalone mock feed provider on port 3002

    ### Testing

    @@ -107,11 +108,11 @@ Before running UI tests for the first time:
  
    - UI tests are located in `tests/ui/` directory

    - Use Page Object Model pattern (see `tests/ui/pages/`)

    - Tests use Playwright with Chromium browser

    - **IMPORTANT**: Always use the backend testserver for feed testing instead of fetching feeds from the internet

      - Mock feeds are served from `http://localhost:3001/feeds/`

    - **IMPORTANT**: Always use the backend test server and mock feed provider instead of fetching feeds from the internet

      - Mock feeds are served from `http://localhost:3002/feeds/`

      - Available mock feeds are defined in `tests/ui/utils/helpers.js` as `MOCK_FEEDS`

      - Never use real external feed URLs in tests to avoid network blockages

    - Test database is separate: `data/test-agg.db`

    - Test database is separate: `data/test-dev.db`

    - Tests run sequentially to avoid database conflicts

    ### Naming Conventions

    @@ -146,11 +147,11 @@ Before running UI tests for the first time:
  
    **CRITICAL**: When testing features that involve RSS/Atom feeds:

    1. **Always use the mock feed server** instead of real internet feeds

    2. Start the test server: `make run-test-server`

    3. Use mock feed URLs from the testserver:

       - `http://localhost:3001/feeds/tech-news.xml` - RSS 2.0 feed with 3 tech articles

       - `http://localhost:3001/feeds/science-blog.xml` - Atom 1.0 feed with 3 science articles

       - `http://localhost:3001/feeds/empty.xml` - Empty RSS feed for edge cases

    2. Start the test services: `make run-test-server` (app on 3001) and `make run-feed-provider` (feeds on 3002)

    3. Use mock feed URLs from the feed provider:

      - `http://localhost:3002/feeds/tech-news.xml` - RSS 2.0 feed with 3 tech articles

      - `http://localhost:3002/feeds/science-blog.xml` - Atom 1.0 feed with 3 science articles

      - `http://localhost:3002/feeds/empty.xml` - Empty RSS feed for edge cases

    4. Mock feeds are located in `internal/testserver/fixtures/`

    5. Add new mock feeds in fixtures directory and reference them in tests

    @@ -179,25 +180,25 @@ Before running UI tests for the first time:
  
    - **Migration support:** Legacy Lite Reader data can be migrated from `agg.db`

    - **Default port:** Application runs on port 3000 by default

    - **Data persistence:** Database file is stored in `data/agg.db`

    - **Test database:** UI tests use separate database at `data/test-agg.db`

    - **Test database:** UI tests use separate database at `data/test-dev.db`

    ## Mock Feed System

    The project includes a mock feed server for reliable, offline testing:

    ### Mock Feed Server

    - **Location**: `internal/testserver/`

    - **Port**: 3001 (when started via `make run-test-server`)

    - **Location**: `internal/testserver/` with entrypoint `cmd/mockfeedprovider/`

    - **Port**: 3002 (when started via `make run-feed-provider`)

    - **Purpose**: Serves mock RSS and Atom feeds locally for testing

    - **Benefits**: No internet required, consistent results, faster tests

    ### Available Mock Feeds

    - **tech-news.xml**: RSS 2.0 feed with 3 technology articles

      - URL: `http://localhost:3001/feeds/tech-news.xml`

      - URL: `http://localhost:3002/feeds/tech-news.xml`

    - **science-blog.xml**: Atom 1.0 feed with 3 science articles

      - URL: `http://localhost:3001/feeds/science-blog.xml`

      - URL: `http://localhost:3002/feeds/science-blog.xml`

    - **empty.xml**: Empty RSS feed for edge case testing

      - URL: `http://localhost:3001/feeds/empty.xml`

      - URL: `http://localhost:3002/feeds/empty.xml`

    ### Adding New Mock Feeds

    1. Create a new XML file in `internal/testserver/fixtures/`

.gitignore

-Original file line number
+Diff line change
@@ Expand Up / @@ -4,6 +4,7 @@ @@
     .idea
     data/agg.db
     data/test-agg.db
+    data/test-dev.db
     public/cache/*.gzip
     public/cache/*.spc
     public/cache/images/*
@@ Expand Down @@

Makefile

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -103,20 +103,25 @@ test-ui-setup:
  
    .PHONY: run-test-server

    run-test-server:

    	@echo "Starting test server with mock feeds..."

    	@HTTP_PORT=3000 DB_PATH=data/test-agg.db CGO_ENABLED=0 go run -mod=vendor ./cmd/testserver/main.go

    	@echo "Starting test server on port 3001 using data/test-dev.db..."

    	@HTTP_PORT=3001 DB_PATH=data/test-dev.db CGO_ENABLED=0 go run -mod=vendor ./cmd/main.go

    .PHONY: run-feed-provider

    run-feed-provider:

    	@echo "Starting mock feed provider on port 3002..."

    	@MOCK_FEED_PROVIDER_PORT=3002 CGO_ENABLED=0 go run -mod=vendor ./cmd/mockfeedprovider/main.go

    .PHONY: test-ui

    test-ui:

    	@echo "Running UI tests..."

    	@mkdir -p reports/playwright

    	@TEST_DB_PATH=data/test-agg.db npm run test:ui

    	@TEST_DB_PATH=data/test-dev.db npm run test:ui

    .PHONY: test-ui-headed

    test-ui-headed:

    	@echo "Running UI tests in headed mode..."

    	@mkdir -p reports/playwright

    	@TEST_DB_PATH=data/test-agg.db npm run test:ui:headed

    	@TEST_DB_PATH=data/test-dev.db npm run test:ui:headed

    .PHONY: test-all

    test-all: test test-ui

TEST.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -92,7 +92,8 @@ make test-all
  
    - **Duration**: UI tests typically complete in 2-3 minutes

    - **Parallelization**: Tests run sequentially (workers: 1) to avoid database conflicts

    - **Browser**: Tests use Chromium in headless mode by default

    - **Database**: Each test run uses a separate test database (`data/test-agg.db`)

    - **Database**: Each test run uses a separate test database (`data/test-dev.db`)

    - **Ports**: UI tests hit the Lite Reader test server on `http://localhost:3001` and the mock feed provider on `http://localhost:3002`

    - **Retries**: In CI, tests retry up to 2 times on failure

    ### Test Reports

    @@ -204,21 +205,23 @@ See the page object files in `tests/ui/pages/` for all available methods.
  
    The mock feed system provides offline RSS/Atom feeds for testing without requiring internet access.

    **Architecture**:

    1. **Mock Feed Server**: A Go HTTP server (`internal/testserver/feedserver.go`) that serves XML feeds

    2. **Feed Fixtures**: Static XML files in `internal/testserver/fixtures/`

    3. **Automatic Startup**: The server starts automatically when running UI tests (via `run-test-server`)

    1. **Lite Reader Test Server**: `cmd/testserver/` boots the application with the UI-test database on `http://localhost:3001`

    2. **Mock Feed Provider**: `cmd/mockfeedprovider/` is a dedicated binary that serves XML fixtures via `internal/testserver/feedserver.go`

    3. **Feed Fixtures**: Static XML files in `internal/testserver/fixtures/`

    4. **Automatic Startup**: Playwright launches both servers via `make run-test-server` and `make run-feed-provider`

    **Ports**:

    - Main app: `localhost:3000`

    - Mock feed server: `localhost:3001`

    - Main app (manual/dev): `localhost:3000`

    - Lite Reader test server (UI tests): `localhost:3001`

    - Mock feed provider: `localhost:3002`

    ### Available Mock Feeds

    | Feed Name | URL | Format | Items | Description |

    |-----------|-----|--------|-------|-------------|

    | Tech News | `http://localhost:3001/feeds/tech-news.xml` | RSS 2.0 | 3 | Technology articles |

    | Science Blog | `http://localhost:3001/feeds/science-blog.xml` | Atom 1.0 | 3 | Science articles |

    | Empty Feed | `http://localhost:3001/feeds/empty.xml` | RSS 2.0 | 0 | Feed with no items |

    | Tech News | `http://localhost:3002/feeds/tech-news.xml` | RSS 2.0 | 3 | Technology articles |

    | Science Blog | `http://localhost:3002/feeds/science-blog.xml` | Atom 1.0 | 3 | Science articles |

    | Empty Feed | `http://localhost:3002/feeds/empty.xml` | RSS 2.0 | 0 | Feed with no items |

    Access feeds in tests using:

    @@ -242,7 +245,7 @@ cat > internal/testserver/fixtures/my-feed.xml << 'EOF'
  
    <rss version="2.0">

      <channel>

        <title>My Feed</title>

        <link>http://localhost:3001/feeds/my-feed</link>

        <link>http://localhost:3002/feeds/my-feed</link>

        <description>My test feed</description>

        <item>

          <title>Item 1</title>

    @@ -259,10 +262,10 @@ EOF
  
    ```javascript

    export const MOCK_FEEDS = {

      techNews: 'http://localhost:3001/feeds/tech-news.xml',

      scienceBlog: 'http://localhost:3001/feeds/science-blog.xml',

      empty: 'http://localhost:3001/feeds/empty.xml',

      myFeed: 'http://localhost:3001/feeds/my-feed.xml',  // Add this

      techNews: 'http://localhost:3002/feeds/tech-news.xml',

      scienceBlog: 'http://localhost:3002/feeds/science-blog.xml',

      empty: 'http://localhost:3002/feeds/empty.xml',

      myFeed: 'http://localhost:3002/feeds/my-feed.xml',  // Add this

    };

    ```

    @@ -277,7 +280,7 @@ await mainPage.addFeed(MOCK_FEEDS.myFeed);
  
    - Use valid RSS 2.0 or Atom 1.0 format

    - Include required fields: `title`, `link`, `description`

    - For items, include: `title`, `link`, `pubDate` (RSS) or `published` (Atom)

    - Use `localhost:3001` URLs to avoid external dependencies

    - Use `localhost:3002` URLs to avoid external dependencies

    - Add realistic content for better test coverage

    ## CI/CD Integration

    @@ -340,15 +343,15 @@ make test-ui-setup
  
    #### Issue: Port already in use

    **Error**: `listen tcp :3000: bind: address already in use`

    **Error**: `listen tcp :3001: bind: address already in use`

    **Solution**:

    ```bash

    # Find and kill process using port 3000

    lsof -ti:3000 | xargs kill -9

    # Find and kill process using port 3001

    lsof -ti:3001 | xargs kill -9

    # Or use a different port

    HTTP_PORT=3002 make run-test-server

    # Or use a different port for the test server

    HTTP_PORT=3005 make run-test-server

    ```

    #### Issue: Test database locked

    @@ -358,10 +361,10 @@ HTTP_PORT=3002 make run-test-server
  
    **Solution**:

    ```bash

    # Stop all running processes

    pkill -f testserver

    pkill -f cmd/testserver

    # Remove test database

    rm data/test-agg.db

    rm data/test-dev.db

    # Run tests again

    make test-ui

TODO.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -1,2 +1,2 @@
  
    TODO

    - [ ] Fix: unread-all does not work for starred items
      
    - [ ] Bug: multi item actions do not work for starred and unread collections

UI-TESTING-SUMMARY.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -6,16 +6,16 @@ This implementation adds comprehensive automated UI testing infrastructure to th
  
    ## Key Components

    ### 1. Mock RSS Feed Server (`internal/testserver/`)

    ### 1. Mock RSS Feed Server (`cmd/mockfeedprovider/`)

    A Go-based HTTP server that serves mock RSS and Atom feeds for testing:

    A dedicated Go-based HTTP server that serves mock RSS and Atom feeds for testing:

    - **Location**: `internal/testserver/feedserver.go`

    - **Location**: `internal/testserver/feedserver.go` with entrypoint `cmd/mockfeedprovider/main.go`

    - **Features**:

      - Serves feeds on `http://localhost:3001`

      - Serves feeds on `http://localhost:3002`

      - Embedded feed fixtures (no external files needed at runtime)

      - Supports RSS 2.0 and Atom 1.0 formats

      - Automatically starts/stops with test runs

      - Automatically starts/stops with test runs via `make run-feed-provider`

    **Sample Feeds**:

    - `tech-news.xml` - RSS 2.0 feed with 3 technology articles

    @@ -24,14 +24,13 @@ A Go-based HTTP server that serves mock RSS and Atom feeds for testing:
  
    ### 2. Test Server Command (`cmd/testserver/`)

    A specialized command that starts both the application and mock feed server:

    A specialized command that starts the Lite Reader application for UI tests:

    - **Purpose**: Provides a complete test environment

    - **Purpose**: Provides an isolated app instance for Playwright

    - **Usage**: `make run-test-server`

    - **Features**:

      - Uses separate test database (`data/test-agg.db`)

      - Starts mock feeds on port 3001

      - Starts main app on port 3000

      - Uses separate test database (`data/test-dev.db`)

      - Starts the app on port 3001 (base URL for UI tests)

    ### 3. Playwright Test Framework

    @@ -73,7 +72,8 @@ Clean, maintainable test code using the Page Object pattern:
  
    make test-ui-setup      # Install Playwright and dependencies (one-time)

    make test-ui            # Run all UI tests (headless)

    make test-ui-headed     # Run with visible browser (debugging)

    make run-test-server    # Start test environment manually

    make run-test-server    # Start Lite Reader test server manually (port 3001)

    make run-feed-provider  # Start mock feed provider manually (port 3002)

    make test-all           # Run both unit and UI tests

    ```

    @@ -147,7 +147,7 @@ Updated `.github/workflows/tests.yaml`:
  
    - Reliable and fast

    ### 2. Isolated Test Environment

    - Separate test database (`data/test-agg.db`)

    - Separate test database (`data/test-dev.db`)

    - No interference with development data

    - Clean state for each test run

    @@ -196,7 +196,7 @@ test('should do something', async ({ page }) => {
  
      const mainPage = new MainPage(page);

      // Perform actions

      await mainPage.addFeed('http://localhost:3001/feeds/tech-news.xml');

      await mainPage.addFeed('http://localhost:3002/feeds/tech-news.xml');

      // Assert expectations

      const count = await mainPage.getItemsCount();

    @@ -212,7 +212,7 @@ test('should do something', async ({ page }) => {
  
    <rss version="2.0">

      <channel>

        <title>My Feed</title>

        <link>http://localhost:3001/feeds/my-feed</link>

        <link>http://localhost:3002/feeds/my-feed</link>

        <description>Test feed</description>

        <item>

          <title>Test Item</title>

    @@ -227,9 +227,9 @@ test('should do something', async ({ page }) => {
  
    2. Add to `tests/ui/utils/helpers.js`:

    ```javascript

    export const MOCK_FEEDS = {

      techNews: 'http://localhost:3001/feeds/tech-news.xml',

      scienceBlog: 'http://localhost:3001/feeds/science-blog.xml',

      myFeed: 'http://localhost:3001/feeds/my-feed.xml',  // Add this

      techNews: 'http://localhost:3002/feeds/tech-news.xml',

      scienceBlog: 'http://localhost:3002/feeds/science-blog.xml',

      myFeed: 'http://localhost:3002/feeds/my-feed.xml',  // Add this

    };

    ```

cmd/mockfeedprovider/main.go

-Original file line number
+Diff line change
@@ -0,0 +1,46 @@
+    package main
+    import (
+    	"context"
+    	"os"
+    	"os/signal"
+    	"strconv"
+    	"syscall"
+    	log "github.com/sirupsen/logrus"
+    	"github.com/cubny/lite-reader/internal/testserver"
+    )
+    const defaultFeedProviderPort = 3002
+    func main() {
+    	log.SetOutput(os.Stdout)
+    	log.SetLevel(log.DebugLevel)
+    	log.SetFormatter(&log.JSONFormatter{})
+    	port := defaultFeedProviderPort
+    	if raw := os.Getenv("MOCK_FEED_PROVIDER_PORT"); raw != "" {
+    		if parsed, err := strconv.Atoi(raw); err == nil {
+    			port = parsed
+    		} else {
+    			log.Warnf("invalid MOCK_FEED_PROVIDER_PORT %q, falling back to %d", raw, defaultFeedProviderPort)
+    		}
+    	}
+    	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
+    	defer stop()
+    	mockServer := testserver.NewMockFeedServer(port)
+    	if err := mockServer.Start(); err != nil {
+    		log.Fatalf("failed to start mock feed provider: %v", err)
+    	}
+    	log.Infof("Mock feed provider running on port %d", port)
+    	<-ctx.Done()
+    	if err := mockServer.Stop(); err != nil {
+    		log.Errorf("failed to stop mock feed provider cleanly: %v", err)
+    	}
+    }

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Phase 2: Refactor feed management from jQuery to HTMX #34

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Phase 2: Refactor feed management from jQuery to HTMX #34

Are you sure you want to change the base?

Uh oh!

Phase 2: Refactor feed management from jQuery to HTMX #34

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!