[epic] Improved search filtering

[43081j]

This is marked as epic since it will likely result in many sub-issues over time.

We should be able to filter the package search by at least the following:

• Only show packages with types
• Only show packages with ESM support
• Only show packages with provenance or above (i.e. provenance or trusted publish)

There's far more we want to do in future but we should keep it tightly scoped to these for now, I think.

High level tasks

• Investigate how we can offer search capabilities npm itself doesn't have
• Implement a filters system
• Implement a types filter
• Implement an ESM filter
• Implement a trust/provenance filter
• Update the UI to expose all of this

Filters system (clientside)

If we do this clientside after all, filters could be fairly simple. Represented as objects something like this:

// we already have this:
export interface NpmSearchOptions {
  /** Number of results to fetch */
  size?: number
}

// so maybe we just need this:
export interface NpmSearchOptions {
  /** Number of results to fetch */
  size?: number

  /** Filters */
  filters?: NpmSearchFilter[];
}

// Then we can have this:
type NpmSearchFilter =
  | { type: 'trustedOnly' }
  | { type: 'provenanceOnly' } // implies trustedOnly too
  | { type: 'hasTypesOnly' };

Open question: client side or server side?

We currently just pass-through to npm's own API in the UI , which means we can only offer what they offer (basic text search).

Willow is currently working on building a database we can use in addition to this, which will offer the various fields we need that npm itself doesn't give us. However, that isn't ready yet so we should probably come up with a temporary solution meanwhile.

For that reason, I think it probably makes most sense to do this client side for now:

• Run the existing logic
• Apply any filters
• If we need to request more to fill the limit, trigger a follow up request

This is an open question though, so please do discuss!

[danielroe]

linking also #32

[43081j]

did a bit of digging

we can only realistically filter by whats in the package result right now, which is basically:

• licenses
• whether it uses trusted publishing or not

pretty much everything else useful needs to be fetched from somewhere else. so it may be worth waiting until we have a secondary database for this