Get individual layer score

[amirozhan]

Hello, is it possible to get individual layer scores when running on vision benchmarks? (Instead of getting the best score)

[mike-ferguson]

Hi @amirozhan, thanks for opening an issue. Right now you can do this by forcing Brain-Score to run only on one layer:

In your model.py's get_layers() function, you can pass in a single layer that your are interested in, instead of multiple ones. Let me know if you would like any more clarification/help!

[amirozhan]

Hi @mike-ferguson, thanks for your quick response. I am already doing this approach, however, it requires a lot of processing time and memory. I wanted to see if there is a shortcut to get scores per layer.

[mike-ferguson]

Ah I see. Off the top of my head, I do not know a way to do what you are after - @mschrimpf any thoughts on how this could be done?

[mschrimpf]

Hi @amirozhan, you can use the LayerScores class which is also used internally to determine the best layers on separate data. It pre-runs all the layers to save processing time:

vision/brainscore_vision/model_helpers/brain_transformation/neural.py

Lines 128 to 131 in ab77a21

	if i == 0 and prerun: # pre-run activations together to avoid running every layer separately
	# we can only pre-run stimuli in response to the benchmark, since we might otherwise be missing
	# visual_degrees resizing.
	layer_model = PreRunLayers(model=model, layers=layers, forward=layer_model)