Support for functional localization

[BKHMSI]

Users can now perform functional localization as described in Brain-Like Language Processing via a Shallow Untrained Multihead Attention Network

Changes:

• Localization stimuli saved in data/fedorenko2010_localization and can be loaded via the data_registry
• Localization script can be found in model_helpers/localize that computes language mask according to the paper mentioned above
• Language mask is cached in .brainio
• HuggingfaceSubject class was adapted to extract activations from multiple layers at once and make use of the localization script if the use_localizer flag is set to True. This extracts only the language selective units from all the activations.

Usage:

• Example script can be found in examples/score_localization

benchmark = load_benchmark('Pereira2018.243sentences-linear')

num_blocks = 12
layer_names = [f'transformer.h.{block}.{layer_type}' 
    for block in range(num_blocks) 
    for layer_type in ['ln_1', 'attn', 'ln_2', 'mlp']
]

model = HuggingfaceSubject(model_id='gpt2', 
    region_layer_mapping={ArtificialSubject.RecordingTarget.language_system: layer_names},
    use_localizer=True,
    localizer_kwargs={
        'hidden_dim': 768,
        'batch_size': 16,
        "top_k": 4096,
    }
)

model_score = benchmark(model)

[mschrimpf]

what does this do? add comment

[BKHMSI]

added

[mschrimpf]

comment why this is necessary

[BKHMSI]

added

[mschrimpf]

where is this being used?

[BKHMSI]

line #103 in the extract_representations function

[mschrimpf]

ok I'm not actually sure what this does -- looks like it's just used to zero-fill layer_name (??)
Could this not also be derived from self.sentences?

   final_layer_representations = {
        "sentences": {layer_name: np.zeros((langloc_dataset.num_samples, hidden_dim)) for layer_name in layer_names},
        "non-words": {layer_name: np.zeros((langloc_dataset.num_samples, hidden_dim)) for layer_name in layer_names}
    }

[BKHMSI]

replaced langloc_dataset.num_samples with len(langloc_dataset.sentences)

[mschrimpf]

looks good; but please check comments

[mschrimpf]

?

[BKHMSI]

uncommented in the new commit

[mschrimpf]

could we just import this metric from brain-score-vision?

[BKHMSI]

that would require brain-score-vision to be a dependency for brain-score-language, not sure if that's a good idea?

[mschrimpf]

@BKHMSI OK to merge?

[BKHMSI]

Yes, it was tested using Python 3.11