Add region layer mapping information to deep dive 2 #380
Conversation
samwinebrake
left a comment
samwinebrake
left a comment
There was a problem hiding this comment.
Looks good overall!
If you think formatting could be improved, it seems like there are 3 distinct sections within the Part 7 section: layer mapping info, examples, and caveats/notes. Perhaps these could be more clearly grouped, or have subheaders? Just throwing out ideas if you weren't satisfied w/ formatting :)
benchmarks/templates/benchmarks/tutorials/models/deepdive_2.html
Outdated
Show resolved
Hide resolved
benchmarks/templates/benchmarks/tutorials/models/deepdive_2.html
Outdated
Show resolved
Hide resolved
| Importantly, when using the JSON file approach, each layer-mapping for each model identifier must be contained in its own JSON file and be named <span class="special_format">{model_identifier}.json</span>. | ||
|
|
||
| </p> | ||
| <p class="benefits_info is-size-5-mobile has-text-weight-bold is-italic"> |
There was a problem hiding this comment.
Just an opinion but I find the bold italic text here a little difficult to read.
| If pre-committed, the layer names must match those returned by the <span class="special_format">get_layers()</span> function in the <span class="special_format">model.py</span>. | ||
| </p> | ||
| <p class="benefits_info is-size-5-mobile"> | ||
| <b>Note:</b> If a submission does not include a <span class="special_format">region_layer_mapping</span> JSON file, the layer mapping will be automatically computed before scoring using |
There was a problem hiding this comment.
What about when the layer mapping is specified as in method 1?
Grammar Co-authored-by: Sam Winebrake <85908068+samwinebrake@users.noreply.github.com>
Co-authored-by: Sam Winebrake <85908068+samwinebrake@users.noreply.github.com>
I agree! I'll spend some time in between other things to address these. Thank you :D |
Currently, region-layer-mapping is including in the
resnet50_tutorialpackage and a key component of Brain-Score however there is no mention of it in the tutorials.Added a section in Deep Dive 2 that draws attention to it, particularly the importance of explicitly pre-specifying the commitment for certain models that have an implicit layer mapping.
More details should be added about how to perform this layer mapping process locally. Will be added when Jupyter Notebook uses are available on the website. For now, I think this is sufficient.
I think formatting could use some improvement. Fielding opinions.