Skip to content

fix(api): Use fixed-layout epub format in Kobo sync when necessary (Booklore PR Port)#16

Open
N00byKing wants to merge 7 commits intogrimmory-tools:developfrom
N00byKing:main
Open

fix(api): Use fixed-layout epub format in Kobo sync when necessary (Booklore PR Port)#16
N00byKing wants to merge 7 commits intogrimmory-tools:developfrom
N00byKing:main

Conversation

@N00byKing
Copy link

@N00byKing N00byKing commented Mar 17, 2026

📝 Description

Unfortunately, I can't get the old PR description back, which sucks.
Also no linked issue because its gone :/

This PR fixes the format specifier in the Kobo Sync endpoint for fixed-layout epubs.
Previously, fixed layout epubs (comics/mangas/etc.) would use the normal book viewer, which does not do double page spreads on landscape, and always shows some header/footer margin.
With this PR, the ereader uses the actual comic reader which doesnt have these issues.

It has already gone through two rounds of review, first one moving the check from sync-time to book-import-time,
then some minor cleanup throughout.

I've made sure that the commit history is preserved when getting this PR ready, so you'll be able to browse the changes individually.

Linked Issue: Fixes #

Required. Every PR must reference an approved issue. If no issue exists, open one and wait for maintainer approval before submitting a PR. Unsolicited PRs without a linked issue will be closed.

🏷️ Type of Change

  • Bug fix
  • New feature
  • Enhancement to existing feature
  • Refactor (no behavior change)
  • Breaking change (existing functionality affected)
  • Documentation update

🔧 Changes

The book_file database now has an additional property specifying whether or not a book is a fixed-layout epub.
This property is then read during Kobo Sync attempts to correctly populate the ebook format in the response.

Also deduplicates a lot of code that opens and reads the epub OPF file.

🧪 Testing (MANDATORY)

PRs without this section filled out will be closed. "Tests pass" or "Tested locally" is not sufficient. You must provide specifics.

Manual testing steps you performed:

  1. Tested on a Kobo Libra H2O for Kobo Sync with fixed-layout epubs.
  2. I had a reproducer file which was linked in the original issue on the booklore github, but I dont have it on my laptop. Will try to remember to reupload it once I'm home.

Regression testing:

  • No regressions observed on non-fixed-layout epubs

Edge cases covered:

  • None known. Current implementation only uses fixed layout for native epubs though, not for converted cbx files. I have very little experience with cbx files, so I didnt want to break anything there. Its an easy change to apply to those as well in the future though, can just set the fixed layout property in the corresponding bookfileentity on import, same as for epub.

Test output:

Backend test output (./gradlew test)
./gradlew test

Welcome to Gradle 9.4.0!

Here are the highlights of this release:
 - Java 26 support
 - Non-class-based JVM tests
 - Enhanced console progress bar

For more details see https://docs.gradle.org/9.4.0/release-notes.html


> Task :compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Management of bidirectional association persistent attributes is deprecated and will be removed. Set the value to 'false' to get rid of this warning

> Task :compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by net.bytebuddy.dynamic.loading.ClassInjector$UsingUnsafe$Dispatcher$CreationAction (file:/home/archuser/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.17.8/af5735f63d00ca47a9375fae5c7471a36331c6ed/byte-buddy-1.17.8.jar)
WARNING: Please consider reporting this to the maintainers of class net.bytebuddy.dynamic.loading.ClassInjector$UsingUnsafe$Dispatcher$CreationAction
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
2026-03-17T10:10:42.292+01:00  INFO 41710 --- [booklore-api] [opFolderWatcher] o.b.s.b.BookdropMonitoringService        : Bookdrop monitor thread interrupted
2026-03-17T10:10:42.295+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.b.BookdropMonitoringService        : Stopped bookdrop folder monitor
2026-03-17T10:10:42.360+01:00  INFO 41710 --- [booklore-api] [opFileProcessor] o.b.s.b.BookdropEventHandlerService      : File processing thread interrupted, shutting down.
2026-03-17T10:10:42.362+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.monitoring.MonitoringService       : Shutting down monitoring service...
2026-03-17T10:10:42.363+01:00  WARN 41710 --- [booklore-api] [        async-0] o.b.service.monitoring.MonitoringTask    : WatchService has been closed. Stopping monitoring.
2026-03-17T10:10:42.363+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.watcher.LibraryFileEventProcessor  : Shutting down LibraryFileEventProcessor...
2026-03-17T10:10:42.385+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2026-03-17T10:10:42.413+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown initiated...
2026-03-17T10:10:42.414+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown completed.
2026-03-17T10:10:42.423+01:00  INFO 41710 --- [booklore-api] [opFolderWatcher] o.b.s.b.BookdropMonitoringService        : Bookdrop monitor thread interrupted
2026-03-17T10:10:42.426+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.b.BookdropMonitoringService        : Stopped bookdrop folder monitor
2026-03-17T10:10:42.427+01:00  INFO 41710 --- [booklore-api] [opFileProcessor] o.b.s.b.BookdropEventHandlerService      : File processing thread interrupted, shutting down.
2026-03-17T10:10:42.429+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.monitoring.MonitoringService       : Shutting down monitoring service...
2026-03-17T10:10:42.429+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.watcher.LibraryFileEventProcessor  : Shutting down LibraryFileEventProcessor...
2026-03-17T10:10:42.430+01:00  WARN 41710 --- [booklore-api] [        async-0] o.b.service.monitoring.MonitoringTask    : WatchService has been closed. Stopping monitoring.
2026-03-17T10:10:42.442+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2026-03-17T10:10:42.445+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown initiated...
2026-03-17T10:10:42.445+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown completed.
2026-03-17T10:10:42.453+01:00  INFO 41710 --- [booklore-api] [opFolderWatcher] o.b.s.b.BookdropMonitoringService        : Bookdrop monitor thread interrupted
2026-03-17T10:10:42.455+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.b.BookdropMonitoringService        : Stopped bookdrop folder monitor
2026-03-17T10:10:42.456+01:00  INFO 41710 --- [booklore-api] [opFileProcessor] o.b.s.b.BookdropEventHandlerService      : File processing thread interrupted, shutting down.
2026-03-17T10:10:42.458+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.monitoring.MonitoringService       : Shutting down monitoring service...
2026-03-17T10:10:42.458+01:00  WARN 41710 --- [booklore-api] [        async-0] o.b.service.monitoring.MonitoringTask    : WatchService has been closed. Stopping monitoring.
2026-03-17T10:10:42.458+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.watcher.LibraryFileEventProcessor  : Shutting down LibraryFileEventProcessor...
2026-03-17T10:10:42.470+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2026-03-17T10:10:42.483+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown initiated...
2026-03-17T10:10:42.485+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown completed.
2026-03-17T10:10:42.491+01:00  INFO 41710 --- [booklore-api] [opFolderWatcher] o.b.s.b.BookdropMonitoringService        : Bookdrop monitor thread interrupted
2026-03-17T10:10:42.491+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.b.BookdropMonitoringService        : Stopped bookdrop folder monitor
2026-03-17T10:10:42.491+01:00  INFO 41710 --- [booklore-api] [opFileProcessor] o.b.s.b.BookdropEventHandlerService      : File processing thread interrupted, shutting down.
2026-03-17T10:10:42.492+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.monitoring.MonitoringService       : Shutting down monitoring service...
2026-03-17T10:10:42.493+01:00  WARN 41710 --- [booklore-api] [        async-0] o.b.service.monitoring.MonitoringTask    : WatchService has been closed. Stopping monitoring.
2026-03-17T10:10:42.493+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.watcher.LibraryFileEventProcessor  : Shutting down LibraryFileEventProcessor...
2026-03-17T10:10:42.499+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2026-03-17T10:10:42.509+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown initiated...
2026-03-17T10:10:42.509+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown completed.
2026-03-17T10:10:42.516+01:00  INFO 41710 --- [booklore-api] [opFolderWatcher] o.b.s.b.BookdropMonitoringService        : Bookdrop monitor thread interrupted
2026-03-17T10:10:42.521+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.b.BookdropMonitoringService        : Stopped bookdrop folder monitor
2026-03-17T10:10:42.521+01:00  INFO 41710 --- [booklore-api] [opFileProcessor] o.b.s.b.BookdropEventHandlerService      : File processing thread interrupted, shutting down.
2026-03-17T10:10:42.523+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.monitoring.MonitoringService       : Shutting down monitoring service...
2026-03-17T10:10:42.523+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.watcher.LibraryFileEventProcessor  : Shutting down LibraryFileEventProcessor...
2026-03-17T10:10:42.523+01:00  WARN 41710 --- [booklore-api] [        async-0] o.b.service.monitoring.MonitoringTask    : WatchService has been closed. Stopping monitoring.
2026-03-17T10:10:42.534+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2026-03-17T10:10:42.543+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown initiated...
2026-03-17T10:10:42.543+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown completed.
2026-03-17T10:10:42.545+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.s.m.s.b.SimpleBrokerMessageHandler     : Stopping...
2026-03-17T10:10:42.545+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.s.m.s.b.SimpleBrokerMessageHandler     : BrokerAvailabilityEvent[available=false, SimpleBrokerMessageHandler [org.springframework.messaging.simp.broker.DefaultSubscriptionRegistry@40571810]]
2026-03-17T10:10:42.545+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.s.m.s.b.SimpleBrokerMessageHandler     : Stopped.
2026-03-17T10:10:42.546+01:00  INFO 41710 --- [booklore-api] [opFolderWatcher] o.b.s.b.BookdropMonitoringService        : Bookdrop monitor thread interrupted
2026-03-17T10:10:42.548+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.b.BookdropMonitoringService        : Stopped bookdrop folder monitor
2026-03-17T10:10:42.548+01:00  INFO 41710 --- [booklore-api] [opFileProcessor] o.b.s.b.BookdropEventHandlerService      : File processing thread interrupted, shutting down.
2026-03-17T10:10:42.548+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.monitoring.MonitoringService       : Shutting down monitoring service...
2026-03-17T10:10:42.549+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] o.b.s.watcher.LibraryFileEventProcessor  : Shutting down LibraryFileEventProcessor...
2026-03-17T10:10:42.549+01:00  WARN 41710 --- [booklore-api] [        async-0] o.b.service.monitoring.MonitoringTask    : WatchService has been closed. Stopping monitoring.
2026-03-17T10:10:42.551+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2026-03-17T10:10:42.555+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown initiated...
2026-03-17T10:10:42.555+01:00  INFO 41710 --- [booklore-api] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : BookloreHikariPool - Shutdown completed.

[Incubating] Problems report is available at: file:///home/archuser/Desktop/grimmory/booklore-api/build/reports/problems/problems-report.html

BUILD SUCCESSFUL in 1m 32s
6 actionable tasks: 6 executed
Consider enabling configuration cache to speed up this build: https://docs.gradle.org/9.4.0/userguide/configuration_cache_enabling.html
Frontend test output (ng test)
ng test
Using Vitest configuration file: /root/vitest-base.config.ts
Initial chunk files                                                            | Names                                                                       |  Raw size
spec-app-features-magic-shelf-component-magic-shelf-component.js               | spec-app-features-magic-shelf-component-magic-shelf-component               |   1.11 MB |
polyfills.js                                                                   | polyfills                                                                   | 164.02 kB |
spec-app-features-magic-shelf-service-book-rule-evaluator.service.js           | spec-app-features-magic-shelf-service-book-rule-evaluator.service           |  65.44 kB |
spec-app-app.component.js                                                      | spec-app-app.component                                                      |  59.29 kB |
chunk-I3SQ37TU.js                                                              | -                                                                           |  53.69 kB |
styles.css                                                                     | styles                                                                      |  35.10 kB |
chunk-XDWXA47J.js                                                              | -                                                                           |  26.55 kB |
spec-app-features-magic-shelf-service-book-rule-evaluator-metadata-presence.js | spec-app-features-magic-shelf-service-book-rule-evaluator-metadata-presence |  23.21 kB |
chunk-DTAZO7BH.js                                                              | -                                                                           |  15.75 kB |
spec-app-features-magic-shelf-service-magic-shelf-utils.js                     | spec-app-features-magic-shelf-service-magic-shelf-utils                     |  11.36 kB |
chunk-PUIDBMXW.js                                                              | -                                                                           |   6.10 kB |
chunk-HMKTQGOO.js                                                              | -                                                                           |   4.92 kB |
spec-app-core-security-auth-initializer.js                                     | spec-app-core-security-auth-initializer                                     |   4.73 kB |
spec-app-features-book-service-library-health.service.js                       | spec-app-features-book-service-library-health.service                       |   2.95 kB |
chunk-3J3QIHZF.js                                                              | -                                                                           |   2.19 kB |
chunk-MUCKHQNJ.js                                                              | -                                                                           |   2.07 kB |
chunk-HPNCK62B.js                                                              | -                                                                           |   1.33 kB |
init-testbed.js                                                                | init-testbed                                                                |   1.27 kB |
vitest-mock-patch.js                                                           | vitest-mock-patch                                                           | 704 bytes |
spec-app-app.js                                                                | spec-app-app                                                                | 202 bytes |

                                                                               | Initial total                                                               |   1.59 MB

Application bundle generation complete. [3.356 seconds] - 2026-03-17T09:22:27.802Z

Watch mode enabled. Watching for file changes...

 DEV  v4.0.18 /root

 ✓  booklore  src/app/app.spec.ts (1 test) 5ms
 ✓  booklore  src/app/features/magic-shelf/service/magic-shelf-utils.spec.ts (46 tests) 23ms
 ✓  booklore  src/app/features/magic-shelf/service/book-rule-evaluator-metadata-presence.spec.ts (85 tests) 169ms
 ✓  booklore  src/app/core/security/auth-initializer.spec.ts (2 tests) 19ms
 ✓  booklore  src/app/features/book/service/library-health.service.spec.ts (6 tests) 29ms
 ✓  booklore  src/app/features/magic-shelf/service/book-rule-evaluator.service.spec.ts (199 tests) 200ms
 ✓  booklore  src/app/app.component.spec.ts (7 tests) 201ms
 ✓  booklore  src/app/features/magic-shelf/component/magic-shelf-component.spec.ts (73 tests) 304ms

 Test Files  8 passed (8)
      Tests  419 passed (419)
   Start at  10:22:27
   Duration  2.34s (transform 1.07s, setup 2.21s, import 2.35s, tests 950ms, environment 3.51s)

JUNIT report written to /root/test-results/vitest-results.xml
 PASS  Waiting for file changes...
       press h to show help, press q to quit

📸 Screen Recording / Screenshots (MANDATORY)

Every PR must include a screen recording or screenshots showing the change working end-to-end in a running local instance (both backend and frontend). This means you must have actually built, run, and tested the code yourself. PRs without visual proof will be closed without review.

Lost to time.
I can re-do the screenshots once I'm home, but its just some pictures of my ereader showing the epubs in fullscreen.


✅ Pre-Submission Checklist

All boxes must be checked before requesting review. Incomplete PRs will be closed without review. No exceptions.

  • This PR is linked to an approved issue
  • Code follows project backend and frontend conventions
  • Branch is up to date with develop (merge conflicts resolved)
  • I ran the full stack locally (backend + frontend + database) and verified the change works
  • Automated tests added or updated to cover changes (backend and frontend)
  • All tests pass locally and output is pasted above
  • Screen recording or screenshots are attached above proving the change works
  • PR is a single focused change (one bug fix OR one feature, not multiple unrelated changes)
  • PR is reasonably scoped (PRs over 1000+ changed lines will be closed, split into smaller PRs)
  • No unsolicited refactors, cleanups, or "improvements" are bundled in
  • Flyway migration versioning is correct (if schema was modified)
  • Documentation PR submitted to booklore-docs (if user-facing changes)

🤖 AI-Assisted Contributions

No AI was used for this PR

💬 Additional Context (optional)

Though it was working fine on the booklore code, I still need to do an end-to-end test on the forked repo.
It passes all tests atm, but probably best not to merge until I can try it on my actual device tonight (Like, 8ish hours from now)
Feel free to try it out yourself though.

Thanks for forking booklore, I really did not feel like migrating my book collection again

@N00byKing
Copy link
Author

I've good good news and bad news.

Bad news is I can't find the reproducer I wrote.

Good news is that I tested the PR, and everything still works:
Demo 1: what if? 2, free flowing epub, shows header and footer
2026-03-17-18-46-32-779
Demo 2: Noragami: Stray God Vol 1, fixed layout epub, hides header + footer and shows double page spreads in landscape.
2026-03-17-18-47-05-835

@CounterClops
Copy link
Contributor

Does this also fix it for when CBX comics get converted to EPUBs during the Kobo sync? Or just EPUB to EPUB?

Since this was an issue I was annoyed by when I added the CBX to EPUB conversion for kobo sync, as I didn't know this fixed layout was an option when I was working on that.

@N00byKing
Copy link
Author

Atm no. I dont know if all cbx files should be treated as fixed layout since I never used any, and they dont have a similar metadata property that I could check for.

If this should apply to all cbx files its a pretty easy change though.

@N00byKing
Copy link
Author

Here's the change needed to make it apply to cbx files as well: N00byKing@d2a2106

If you try it out and it works I'll put it in the PR as well.

@CounterClops
Copy link
Contributor

CounterClops commented Mar 19, 2026

I'll test it out and see how it reacts, but effectively Kobo sync doesn't support CBX files so I added functionality that creates new EPUB files using the images extracted from a CBX file.

A CBX file is just a ZIP/RAR/7zip file with a bunch of images inside, so they're effectively always comics or manga. So when a CBX file is picked up in the kobo sync, it triggers the below to convert it to an EPUB.
https://github.com/grimmory-tools/grimmory/blob/6b0c4aa66c2247de0335340dbae8ec0afc1fbd59/booklore-api/src/main/java/org/booklore/service/kobo/CbxConversionService.java

That basically just builds an EPUB out of the template files present below, with each image (pulled from the CBX file) being put on it's own page inside the EPUB.
https://github.com/grimmory-tools/grimmory/tree/main/booklore-api/src/main/resources/templates/epub/xml

If it's easier, I'm happy to wait until this is merged in (As it is without the CBX stuff) and things are stable, and then loop back to look at it.

@CounterClops
Copy link
Contributor

CounterClops commented Mar 19, 2026

Here's the change needed to make it apply to cbx files as well: N00byKing@d2a2106

If you try it out and it works I'll put it in the PR as well.

Could you not just change this one line to KoboBookFormat.EPUB3FL instead? Or does the EPUB itself need changes to support fixed layout? Once again I'm happy to handle this myself after, just curious around what needs to happen for fixed-layout to work.

https://github.com/N00byKing/grimmory/blob/d2a2106372325f63b1c70b896784240e2d5eebd8/booklore-api/src/main/java/org/booklore/service/kobo/KoboEntitlementService.java#L337

@N00byKing
Copy link
Author

For the fixed layout stuff there's a new entry in the database, so imo it makes more sense to populate it just the same as for the epubs for consistency.
But this does introduce redundancy on that branch, so I just removed all the cbx-specific stuff which would be unused.
New test commit: N00byKing@fa89128

@N00byKing N00byKing changed the base branch from main to develop March 20, 2026 08:07
@imajes imajes force-pushed the develop branch 2 times, most recently from 89113d4 to 37ca101 Compare March 20, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants