feat: custom extensions

[mplemay]

Hey @piercefreeman ,

I wanted to get your thoughts on adding pgvector (and potentially other extensions) to iceaxe.

As far as I can tell, the steps required (at least for pgvector) are 1) adding a vector type, 2) enabling/disabling the extension during migration, 3) support for custom indexes, and 4) a func for the vector type.

Let me know your thoughts (or if this is even within the scope of the library).

[piercefreeman]

@mplemay Technically anything in native Postgres would be fair game for iceaxe. There's a long laundry list in Appendix F that covers a lot of different cases: https://www.postgresql.org/docs/current/contrib.html

But more acutely, this is how I think about it:

• If logic just touches query time values, this is better for raw .text() commands than trying to shoehorn everything into a DSL that just reimplements the underlying Postgres query modifiers for the sake of having them function calls written in Python.
• We need to be able to typehint outputs correctly throughout the QueryBuilder and executor for me to feel comfortable supporting it as a native feature.
• There needs to be material value provided by being able to specify the extension within TableBase classes, where the given extension primitive type often lives alongside standard Postgres data.

pgvector falls under the category of not officially part of Postgres but very popular (>13k stars as of this comment) and serving a clear value when embedded alongside data. So I'd certainly encourage any contributions here.

A few thoughts on the top of my head that need to be considered:

• Native Python types are pretty inefficient when handling numerical data as lists. Would Iceaxe want to support numpy and pytorch constructors & casting result types to these? I could imagine a Vector(autocast="native|numpy|torch") definition that could make this pretty easy to change around.
• If we do go with support for numpy & pytorch, how does that interface with asyncpg? The pgvector docs always show formatting vectors as strings during insertion, is the same true during query time or does asyncpg try to parse them intelligently into lists? If it's returned as strings we'll want some efficient string parsing/casting.
• What's the right search API? Do we want to overload ORDER BY and let it accept more complicated values, or is it better to have something like order_by_vector(column: Vector, value: Vector) that makes the contract explicit?
• What's the best way to support the different types of similarity functions: inner product (<#>), cosine distance (<=>), and L1 distance (<+>, added in 0.7.0)?
• Should the index definition and parameters (ie. CREATE INDEX ON items USING hnsw (embedding vector_l2_ops)) be specified as a Field argument, or require an additional table_args to be specified? Due to the expanding complexity of index configuration parameters I would think probably best to force it as a table_arg.