Skip to content

feat: Flickr source plugin #19

@jschloman

Description

@jschloman

Summary

Add a FlickrPlugin source plugin that loads a user's photo history from their Flickr account export. Flickr photos carry EXIF timestamps and optional GPS coordinates, making this a dual-type plugin: where-when when photos are geotagged, what-when (album/tag browsing) otherwise.

Phase 1 targets the where-when schema using geotagged photos as location events.

Architecture

This plugin follows the project-wide source plugin contract:

  1. Download-then-display — the plugin reads only from a previously downloaded local export file. No outbound network or API calls are made at Streamlit runtime. If the data file is absent, fetch() raises FileNotFoundError with a message directing the user to obtain and place the export file.

  2. Data sovereignty — the plugin has no knowledge of other sources. fetch() returns only this source's canonical DataFrame. No filtering, joining, or merging with other sources is performed here; that is the exclusive responsibility of DataBroker.

Data export

How to get: flickr.com → Account → Your Flickr Data → "Request my Flickr data" → download ZIP when ready (email notification, processing time varies)

Format: ZIP containing:

  • account_profile.json — account metadata
  • photo_*.json — one JSON file per photo with full metadata
  • Original photo files (JPG/PNG etc.) — not needed for this plugin

Key fields in each photo_*.json:

Field Maps to
date_taken (ISO 8601) timestamp
geo.latitude / geo.longitude lat / lng
name place_name (photo title as location label)
tags[].tag preserve as tags
albums[].title preserve as album
description preserve
photopage preserve as url

Plugin spec

  • PLUGIN_TYPE: where-when
  • PLUGIN_ID: flickr
  • DISPLAY_NAME: Flickr Photo Locations
  • Config fields:
    • export_dir (path to the unzipped Flickr export directory)
    • geotagged_only toggle (default: True — skip photos without coordinates)
  • Glob all photo_*.json files in the export directory
  • Skip photos where geo is absent or null when geotagged_only is enabled
  • Parse date_taken ISO string to Unix timestamp
  • place_name → photo title (best available label for the location event)
  • place_type → "photo"

Normalized output schema

timestamp     int       Unix timestamp of photo capture (date_taken)
lat           float     Latitude (WGS84, from geo.latitude)
lng           float     Longitude (WGS84, from geo.longitude)
place_name    str       Photo title
place_type    str       "photo"
source_id     str       "flickr"
tags          str       Space-separated tag string (preserved)
album         str       First album title if present (preserved)

Acceptance criteria

  • FlickrPlugin registered via @register and added to load_builtin_plugins()
  • Correctly globs all photo_*.json files from the export directory
  • Photos without geo data excluded when geotagged_only=True, included with NaN lat/lng when False
  • validate_schema() passes on output
  • Unit tests covering: geotagged photo, non-geotagged photo with toggle on/off, missing title, empty export dir
  • Plugin contains zero HTTP/network code — reads only from the local export file
  • Plugin contains zero references to other source plugins, their schemas, or join keys
  • All existing tests continue to pass

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions