Skip to content

Conversation

@tillahoffmann
Copy link
Contributor

@tillahoffmann tillahoffmann commented Jan 8, 2026

This PR fixes type annotations for ArrayRecordDataSources. Currently, ArrayRecordDataSource does not conform to the RandomAccessDataSource protocol for two reasons:

  1. The stub that indicates ArrayRecordDataSource is not supported on Windows is missing __len__ and __getitem__.
  2. The implementation uses record_key: SupportsIndex instead of index: int as the __getitem__ argument. Renaming the argument is backwards-compatible because it is never accessed as a keyword argument in normal usage.

Here's an example.

from grain import MapDataset
from grain.sources import ArrayRecordDataSource


ds = ArrayRecordDataSource([])
MapDataset.source(ds)

Before

$ pyright playground/ards.py
/Users/till/git/grain/playground/ards.py
  /Users/till/git/grain/playground/ards.py:6:19 - error: Argument of type "ArrayRecordDataSource" cannot be assigned to parameter "source" of type "Sequence[T@source] | RandomAccessDataSource[T@source]" in function "source"
    Type "ArrayRecordDataSource" is not assignable to type "Sequence[T@source] | RandomAccessDataSource[T@source]"
      "ArrayRecordDataSource" is incompatible with protocol "RandomAccessDataSource[T@source]"
        "__len__" is not present
          "__getitem__" is an incompatible type
            Type "(record_key: SupportsIndex) -> bytes" is not assignable to type "(index: int) -> T@RandomAccessDataSource"
      "ArrayRecordDataSource" is not assignable to "Sequence[T@source]" (reportArgumentType)
1 error, 0 warnings, 0 informations

After

$ pyright playground/ards.py
0 errors, 0 warnings, 0 informations

This is very basic, but I've ended up with cast(Sequence, ds) in several places.


📚 Documentation preview 📚: https://google-grain--1190.org.readthedocs.build/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant