parameterize Source and Target on the shape of the array

[dmanam]

This seems fairly natural, and doesn't seem to cause any problems, while it allows the creation of more specific types of arrays (HMatrix arrays, for example, which can only exist as scalars, vectors, or 2D matrices).

see https://github.com/dlahoti/repa-hmatrix

[benl23x5]

I don’t understand why you need to parameterise Source over the array shape. The source class provides a ‘linearIndex’ method, and the shape polymorphic ‘index’ function is built on top of that. Any array type that supports linear indexing naturally supports higher-dimensional indexing as well via the usual row-major mapping ( x + y * width , etc).

If HMatrix supports a linear vector type, then you should be able to define an instance for the ‘Source’ class and use all of Repa’s shape polymorphic functions with it as well, or have I misunderstood what you’re trying to do?

[dmanam]

If I were to implement multidimensional arrays over HMatrix vectors, I would lose a lot of HMatrix's optimization; for example, using HMatrix's representation for 2D matrices for slicing would be O(1) in many cases where using the row-major mapping over HMatrix vectors would be O(m*n). I would also be required to convert from Vector a to Matrix a each time I want to use an Array H DIM2 a as an HMatrix matrix. Since HMatrix relies on calls to BLAS/LAPACK with BLAS-style matrices, it makes more sense to convert once (from Array a DIM2 to Array H DIM2, which would just be a newtype wrapper for HMatrix's Matrix), run fast HMatrix operations on the converted matrices, and convert back.

Effectively, what I want is Source instances for H where Array H DIM0 e is a newtype wrapper for e, Array H DIM1 e is a newtype wrapper for Vector e, and Array H DIM2 e is a newtype wrapper for Matrix e. I don't care about efficiency/existence of any higher-dimensional types of Array H, since there's no reason to have any such array in HMatrix. It's possible to implement this without parametrizing over the dimension, but as far as I can tell it necessarily ends up being sort of an ugly hack and requires runtime checks where compile-time checks should suffice.

Since having easy interoperability with HMatrix would, I think, be pretty useful for a lot of people who might use Repa, and might even get more people to use it, and since parameterizing these classes over the dimension wouldn't have any detriment on any other type of Repa array, this seems to be the way to go.

[benl23x5]

Sorry for the delayed reply, just got back from ICFP.

If I were to implement multidimensional arrays over HMatrix vectors, I would lose a lot of HMatrix's optimization; for example, using HMatrix's representation for 2D matrices for slicing would be O(1) in many cases where using the row-major mapping over HMatrix vectors would be O(m*n).

As it is I don't see anything preventing you from exposing hmatrix specific versions of functions like 'slice', they could just work on arrays that are represented via HMatrix.

hslice :: Array H DIM2 e -> ... -> Array H DIM1 e

I would also be required to convert from Vector a to Matrix a each time I want to use an Array H DIM2 a as an HMatrix matrix. Since HMatrix relies on calls to BLAS/LAPACK with BLAS-style matrices, it makes more sense to convert once (from Array a DIM2 to Array H DIM2, which would just be a newtype wrapper for HMatrix's Matrix), run fast HMatrix operations on the converted matrices, and convert back.

Does HMatrix really use different representations for Vector and Matrix? I would have expected that converting one to the other would just be a type cast, rather than requiring any real computation.

Effectively, what I want is Source instances for H where Array H DIM0 e is a newtype wrapper for e, Array H DIM1 e is a newtype wrapper for Vector e, and Array H DIM2 e is a newtype wrapper for Matrix e. I don't care about efficiency/existence of any higher-dimensional types of Array H, since there's no reason to have any such array in HMatrix. It's possible to implement this without parametrizing over the dimension, but as far as I can tell it necessarily ends up being sort of an ugly hack and requires runtime checks where compile-time checks should suffice.

The Repa API wasn't designed with this use case in mind. Most of the operators are shape polymorphic by nature, and the library provides a way to evaluate these operators in parallel. If you're not using that functionality I don't really see the point of using Repa.

Since having easy interoperability with HMatrix would, I think, be pretty useful for a lot of people who might use Repa, and might even get more people to use it, and since parameterizing these classes over the dimension wouldn't have any detriment on any other type of Repa array, this seems to be the way to go.

On the other hand, for existing users of Repa that do not need HMatrix this change complicates the type signatures for no benefit. It might be better to define a different Source type class that has an explicit shape parameter. You could make the existing Source class a super-class of the new one. The existing Source class was intended to manage the in-memory representation of an array, rather than restrict what shape it might be -- that's really a different notion.