Skip to content

Improve management of large libraries #156

@aaccomazzi

Description

@aaccomazzi

A number of problems appear in library operations when the number of records grow beyond a certain threshold (a few thousand). This becomes a problem for people using ADS libraries to maintain bibliographies, and renders them far less useful. An extreme case is this one (26K records): https://ui.adsabs.harvard.edu/public-libraries/vJf4U0AGS8-NIqDp6gqEdQ

In particular:

  1. "View library in search results" times out
  2. Metrics and visualizations (even though they take the top N results) sometime fail to complete
  3. Export seems flakey (not sure what's going on, but paging then exporting seems to fail?)

One initial step would be to implement the change that @romanchyla advocated when exporting the library, which at least allows for further manipulation of the bibcodes in the main application: https://github.com/adsabs/solr-service/blob/f5d85d6d4cd75ed26a7f00a5c294d360b7b954df/solr/views.py#L446

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions