Utility to Check For Embedded Canvas Data When Canvas URI Won't Resolve

[thehabes]

If a Canvas URI will not resolve, load its parent and check if it exists as an embedded object. This means attempting to load the Manifest or Project level data in an attempt to find the Canvas and its Annotation page.

[thehabes]

Implementation Plan: Embedded Canvas Fallback in Vault

Summary

When vault.get(canvasId, 'canvas') fails because the Canvas URI won't resolve, vault should automatically attempt to find the Canvas as embedded data within a parent Manifest. This requires:

1. Vault infrastructure for Canvas-to-Manifest mapping and fallback resolution
2. BFS improvement to cache embedded objects directly instead of re-fetching by URI
3. Manifest pre-loading so vault has Manifests in memory for fallback

Files to Modify

File	Change
js/vault.js	Core fallback logic, BFS improvement, Canvas-to-Manifest mapping
api/TPEN.js	Pre-load manifest on project-loaded event
components/continue-working/index.js	Use registerManifestHint(), remove ad-hoc fallback
components/read-only-transcribe/index.js	Push static manifest into vault via vault.set()
components/simple-transcription/index.js	Fix missing component parameter
components/default-transcribe/index.js	Replace @iiif/helpers Vault with custom vault

---

Phase 1: Vault Core Changes (js/vault.js)

1.1 New data structure in constructor

Add this.canvasToManifest = new Map() — maps canvasId → manifestId for fallback lookup.

1.2 New method: _indexManifest(manifest)

Scans a manifest for embedded canvases and populates canvasToManifest:

• v3: iterate manifest.items where type is Canvas
• v2: iterate manifest.sequences[0].canvases

1.3 New method: _getEmbeddedCanvas(canvasId)

Looks up canvasToManifest for the manifest ID, retrieves the manifest from vault's in-memory store, searches its items/canvases for the matching Canvas ID. Returns the Canvas object or null.

1.4 New method: _resolveCanvasFromManifest(canvasId, component) (async)

The core fallback orchestrator:

1. Try _getEmbeddedCanvas(canvasId) — check if already cached
2. If not found, determine manifest URI:
   • From canvasToManifest map (if registered via hint or prior indexing)
   • Fallback to TPEN.activeProject?.manifest?.[0]
3. Fetch the manifest via this.get(manifestUri, 'manifest') (which will cache + index it via BFS)
4. Try _getEmbeddedCanvas(canvasId) again — should now succeed
5. Return Canvas or null

1.5 New public method: registerManifestHint(canvasId, manifestUri)

Allows components to tell vault which manifest a canvas belongs to. Essential for components like continue-working that work with non-active-project canvases.

1.6 Modify set() to index manifests and skip localStorage for large types

• When storing a manifest type, call _indexManifest(item)
• Skip localStorage.setItem() for types manifest, sc:manifest, collection, sc:collection (manifests can be megabytes, localStorage has ~5MB quota)

1.7 Restructure get() for Canvas fallback

Current flow dispatches error and returns null immediately on failure. New flow:

1. Check memory cache → return if found
2. Check localStorage cache → return if found
3. HTTP fetch → validate → BFS → cache → return
4. On ANY failure for canvas type:
   a. Capture error details (don't dispatch yet)
   b. Try _resolveCanvasFromManifest(id, component)
   c. If fallback succeeds: validate canvas, cache it, return it
   d. If fallback fails: NOW dispatch the original error, return null

Key: error event is only dispatched if BOTH URI resolution AND manifest fallback fail.

For non-canvas types, behavior is unchanged (return null on failure, no fallback).

1.8 Improve BFS: cache embedded objects directly via structuredClone + set()

Current BFS (lines 202-230): For embedded objects with id+type, fires this.get(id, type).catch(() => {}) (re-fetches each child by URI, which may fail) then replaces object in parent with stub {id, type, label}.

New BFS: Replace fire-and-forget re-fetch with direct caching:

// For non-array objects with id+type:
try { this.set(structuredClone(value), type) } catch {}

// For array items with id+type (NEW - currently not cached individually):
try { this.set(structuredClone(item), itemType) } catch {}

Why this works: structuredClone() captures the FULL embedded object before BFS modifies its children. When BFS later processes the object's children (deeper in the queue), only the original reference is modified, not the clone. The clone retains complete data.

Performance: Replaces N network requests with N in-memory clones (~microseconds each vs ~100ms+ each). For 200-canvas manuscript: <10ms total vs 200 HTTP requests.

---

Phase 2: Manifest Pre-loading (api/TPEN.js)

Modify the tpen-project-loaded handler (line 62) to pre-cache the active project's manifest:

eventDispatcher.on("tpen-project-loaded", ev => {
    this.activeProject = ev.detail
    const manifestUri = ev.detail?.manifest?.[0]
    if (manifestUri) {
        import('../js/vault.js').then(m => m.default.get(manifestUri, 'manifest').catch(() => {}))
    }
})

• Dynamic import avoids circular dependency (vault.js imports TPEN.js)
• .catch(() => {}) makes pre-load best-effort
• This ensures the manifest is cached BEFORE any canvas resolution attempt
• BFS during manifest caching will also cache all individual embedded canvases

---

Phase 3: Component Updates

3.1 components/continue-working/index.js — Remove ad-hoc fallback

In getProjectThumbnail():

• After getting canvasId (line 178), before vault.get() (line 182), add:

  const manifestUrl = project.manifest?.[0]
  if (manifestUrl && canvasId) vault.registerManifestHint(canvasId, manifestUrl)

• Remove the entire else block (lines 188-201) that manually fetches the manifest and searches for the canvas — vault's internal fallback handles this now
• Adjust the remaining code since vault.get() now returns the embedded canvas on fallback

3.2 components/read-only-transcribe/index.js — Feed static manifest to vault

In loadAnnotations(), after this.#staticManifest = manifest (line 334), add:

vault.set(manifest, 'manifest')

This puts the static manifest into vault's cache, enabling vault's embedded canvas fallback. The existing processCanvas() logic (checking #staticManifest.items first, then vault.get()) can remain as an optimization.

3.3 components/simple-transcription/index.js — Fix missing parameter

Line 518: add component parameter:

// Before:
const fetchedCanvas = await vault.get(canvasID, 'canvas')
// After:
const fetchedCanvas = await vault.get(canvasID, 'canvas', false, 'tpen-simple-transcription')

3.4 Components that benefit automatically (NO changes needed)

These components already call vault.get(canvasId, 'canvas', false, componentName) and handle null returns. They'll automatically benefit from vault's internal manifest fallback:

• interfaces/transcription/index.js (3 calls)
• components/annotorious-annotator/plain.js (1 call)
• components/annotorious-annotator/line-parser.js (1 call)
• components/legacy-annotator/plain.js (1 call)
• components/line-image/index.js (1 call)

3.5 components/default-transcribe/index.js — Replace @iiif/helpers Vault with custom vault

The @iiif/helpers Vault doesn't work and should be replaced with our custom vault.

Current: Uses import { Vault } from 'https://cdn.jsdelivr.net/npm/@iiif/helpers/+esm' (line 9) with a separate API:

• vault.get({id, type}) — synchronous cache lookup
• vault.load(id) — async fetch

Changes:

1. Replace import: import vault from '../../js/vault.js'
2. Remove const vault = new Vault() (line 11)
3. Line 76 — AnnotationPage: replace vault.get({id:..., type:"AnnotationPage"}) ?? await vault.load(...) with await vault.get(annotationPageID, 'annotationpage', true, 'tpen-transcription')
4. Line 92 — Annotation: replace vault.get({id:..., type:"Annotation"}) ?? await vault.load(l.id) with await vault.get(l.id, 'annotation', false, 'tpen-transcription')
5. Line 93 — ContentResource: remove vault.get({id:lineElem.line.body[0].id, type:"ContentResource"}). This was @iiif/helpers-specific normalization. With our vault, the body data is already part of the annotation response. Just ensure body is an array:

   if (!Array.isArray(lineElem.line.body)) {
       lineElem.line.body = [lineElem.line.body]
   }

---

Test Data Scenario

A test project has been set up with intentionally broken canvas/image references:

• Project ID: 698378c3c19363cc0b368ef4
• Manifest: https://tpen-project-examples.habesoftware.app/transcription-project-bad-canvases/manifest.json (resolves, contains both canvases embedded)
• User: bryan@rerum.io (user ID 68d6dd6a718ee294f194be04)

Resource	URI	HTTP Status	Behavior
Canvas 1	.../canvas-1.json	404	URI fails, but canvas is embedded in manifest with a working image
Canvas 2	.../canvas-2.json	200	URI resolves, but its image is broken (404) — out of scope for #429

Canvas 1 is our primary test case for the manifest fallback. After our changes, vault should:

1. Attempt vault.get(canvas-1-uri, 'canvas') → 404 failure
2. Fall back to the manifest (which is in vault cache from pre-load)
3. Find canvas-1 embedded in manifest.items[0]
4. Return the embedded canvas (with working image URL)
5. Interface renders the image successfully — no error toast

---

Manual Verification Plan

Run the app with jekyll s and log in as bryan@rerum.io.

Test 1: Canvas 1 — Broken URI, Manifest Fallback (primary test)

Navigate to: localhost:4000/interfaces/transcription/?projectID=698378c3c19363cc0b368ef4

• Select the page targeting Canvas 1
• Expected: The title page image loads correctly from the manifest fallback
• Expected: No error toast appears (fallback succeeded)
• Expected: Console shows the Canvas 1 URI failed but fallback resolved it

Test 2: Canvas 2 — Resolving URI, Broken Image (out of scope verification)

• Select the page targeting Canvas 2
• Expected: Canvas resolves normally (no fallback needed), image failure is a separate issue (UI/UX Path Forward When Data Cannot Be Obtained #430)

Test 3: Annotator Interface

Navigate to: localhost:4000/interfaces/annotator/?projectID=698378c3c19363cc0b368ef4

• Select the page targeting Canvas 1
• Expected: Canvas image loads via manifest fallback
• Expected: Annotation overlay functions correctly on the embedded canvas data

Test 4: Continue Working Dashboard

Navigate to: localhost:4000/ (dashboard)

• Expected: Project 698378c3c19363cc0b368ef4 shows a thumbnail (even though Canvas 1 URI is broken)
• Expected: The thumbnail loads via manifest fallback or shows the working image

Test 5: Normal Project (regression check)

Navigate to any other project in the transcription interface

• Expected: Canvas loading works exactly as before — no regressions
• Expected: No unexpected console warnings

Test 6: Console Verification (all tests)

• When Canvas 1 fallback succeeds: no tpen-canvas-resolution-failed event dispatched
• When Canvas 1 URI fails but fallback works: a brief console log indicating fallback was used
• Normal canvas resolution: no new console output