Metadata fields

The metadata collection is populated by __init__.py files in the scrapers directory (see e.g. OpenStates). As such, it is very consistent.

Fields that are candidates for elimination are given in bold.

Fields on all documents

• _id: the jurisdiction ID, e.g. ak
• _type: always metadata
• name: the jurisdiction name, e.g. Alaska
• legislature_name: the legislature name, e.g. Alaska State Legislature
• capitol_timezone: the legislature's timezone, e.g. America/Anchorage, used to set the time correctly when importing data
• abbreviation: same as _id
• feature_flags: one or more of capitol_maps, events, influenceexplorer or subjects
  • capitol_maps: the jurisdiction has researched maps of the capitol building (used by iOS app)
  • events: the jurisdiction has events (experimental)
  • influenceexplorer: legislators have Influence Explorer fields, e.g. transparencydata_id
  • subjects: the jurisdiction has categorized subjects
• latest_update: the date of the latest scraper run
• Chambers
  • chambers
  • chambers.upper
  • chambers.upper.name: the upper chamber's brief name, e.g. Senate
  • chambers.upper.title: the title for members of the upper chamber, e.g. Senator
• Terms
  • terms
  • terms.name
  • terms.sessions
  • terms.end_year
  • terms.start_year
• Sessions
  • session_details
• Downloads
  • latest_json_url
  • latest_json_date
  • latest_csv_url
  • latest_csv_date

Fields on more than 90% of documents

• _ignored_scraped_sessions
• legislature_url
• Chambers
  • chambers.lower
  • chambers.lower.name: the lower chamber's brief name, e.g. House
  • chambers.lower.title: the title for members of the lower chamber, e.g. Representative

Fields on less than 10% of documents

• _partial_vote_bill_id: used by RI bills scraper
• requests_per_minute: occurs only on NY and seems to have no effect
• Chambers
  • chambers.lower.term: the length, in years, of a term for members of the chamber
  • chambers.upper.term: the length, in years, of a term for members of the chamber
• Terms
  • terms.biennium: used by MN committees scraper
  • terms.session_number: used by MT legislators scraper
  • terms.start_date occurs only on CA and seems to have no effect
  • terms.type occurs only on TX and seems to have no effect
• Features
  • capitol_maps
  • capitol_maps.name
  • capitol_maps.url