Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
2124c7d
Allow for decomposition of embedding
danbraunai-apollo May 1, 2025
1b877b7
Merge branch 'dev' into feature/embedding-component
danbraunai-apollo May 1, 2025
f270e5e
Set component.mask to None after forward_with_component
danbraunai-apollo May 2, 2025
eef4455
Fail if masks doesn't contain all components
danbraunai-apollo May 2, 2025
c641be2
Ensure mask exists in layerwise recon loss
danbraunai-apollo May 2, 2025
b454c5d
Avoid calculating component_activation_statistics when plotting
danbraunai-apollo May 2, 2025
bc2d264
Add embedding loss which does mse after embedding
danbraunai-apollo May 6, 2025
9df05db
Make fan_val=d_out for A matrix
danbraunai-apollo May 6, 2025
43d5d40
Use inference mode for all logging
danbraunai-apollo May 6, 2025
29bf3ea
Use fan_val=d_out
danbraunai-apollo May 6, 2025
2765e13
Only calculate component statistics when plotting
danbraunai-apollo May 6, 2025
f90af90
Merge branch 'dev' into feature/embedding-component
danbraunai-apollo May 6, 2025
15a9c15
Merge branch 'dev' into feature/embedding-component
danbraunai-apollo May 7, 2025
e22fff8
Remove line which transposes embedding weight
danbraunai-apollo May 7, 2025
ea3fc5f
Add plots for embeddings in plot_embedding_components.py
danbraunai-apollo May 8, 2025
350a666
Add n_dead_components to plot_embedding_components.py
danbraunai-apollo May 9, 2025
de60be5
Merge branch 'dev' into feature/embedding-component
danbraunai-apollo May 9, 2025
14550f2
Show alive components rather than dead components
danbraunai-apollo May 9, 2025
0acdd93
Set target_module_patterns to embedding
danbraunai-apollo May 9, 2025
09be8d1
Merge branch 'dev' into feature/embedding-component
danbraunai-apollo May 9, 2025
58e416c
added Schatten loss
LuciusApollo May 9, 2025
2825ddd
setting weight decay to 0 again
LuciusApollo May 9, 2025
4a26596
Increase print_freq default to 1000
danbraunai-apollo May 9, 2025
cef662a
Merge branch 'dev' into feature/embedding-component
danbraunai-apollo May 9, 2025
0df81de
Allow unembed and kl for the embedding recon loss
danbraunai-apollo May 9, 2025
6962d93
implemented permute_to_identity for plotting the masks
LuciusApollo May 9, 2025
df7d4ed
More safely handle embedding layer differences
danbraunai May 19, 2025
89a89b2
Make streaming default to False due to slowness
danbraunai May 19, 2025
02c0f78
Don't save optimizer
danbraunai May 20, 2025
b24c7c2
Remove optimizer loading
danbraunai May 22, 2025
9fd7c6a
>2x speed up EmbeddingComponent with indexing
danbraunai May 22, 2025
9cf6f15
Fix plot_embedding_component for new embedding method
danbraunai May 22, 2025
1276d93
Add embed_mask_sample_table to wandb
danbraunai May 22, 2025
f87e95d
Allow for n_random_masks>1 in embedding_recon_loss
danbraunai May 22, 2025
d028714
Support generic model in lm_decomposition.py
danbraunai May 29, 2025
10fcdb6
WIP: put resid_mlp in new format
danbraunai May 29, 2025
5dad57a
WIP: replicate resid_mlp plots
danbraunai May 30, 2025
8b1802a
Successfully replicate resid_mlp runs
danbraunai May 30, 2025
4ac0d5c
Replicate TMS
danbraunai Jun 1, 2025
f789766
Remove default output_loss_type
danbraunai Jun 2, 2025
deaa1ca
Fix train_tms tied weights and no n_instances
danbraunai Jun 2, 2025
9a9aadc
Fix tms weight tying
danbraunai Jun 2, 2025
0d34f5c
Create new dir structure
danbraunai Jun 2, 2025
1533f30
Remove LinearComponentWithBias
danbraunai Jun 2, 2025
8a8a011
Add config field descriptions
danbraunai Jun 2, 2025
23c2288
Fix pretrained_model_name args
danbraunai Jun 3, 2025
e14b99e
Add tms plotting.py
danbraunai Jun 3, 2025
2f52d38
Remove init_from_target option
danbraunai Jun 3, 2025
44f2102
Fix tests
danbraunai Jun 3, 2025
4b4dfa8
Increase dpi on mask plots
danbraunai Jun 3, 2025
8195355
Add out recon
danbraunai Jun 3, 2025
8021727
Add mask value histogram plot to all runs
danbraunai Jun 5, 2025
dc0693a
Assert that all target_module_patterns match real modules
danbraunai Jun 5, 2025
37a88ff
Init bias to 0 in tms
danbraunai Jun 6, 2025
a75f1a7
Handle optional init bias in tms
danbraunai Jun 6, 2025
538a837
relud_masks -> sparsity_masks and plot both mask heatmaps
danbraunai Jun 6, 2025
0da4852
Fix pretrained_model_class in tms config
danbraunai Jun 6, 2025
a76e058
Add calc_mmcs_and_ml2r for tms
danbraunai Jun 6, 2025
9546112
Update plot thresholds for tms for latest runs
danbraunai Jun 6, 2025
ab2a3d5
Fix norming in tms plotting
danbraunai Jun 6, 2025
972c9ca
Add resid_mlp_varying_sparsity combined plotting
danbraunai Jun 8, 2025
8945812
Put latest runs in resid_mlp_interp.py
danbraunai Jun 9, 2025
ff451ba
Update resid_mlp train config
danbraunai Jun 9, 2025
8a61e07
Allow schatten in all experiments
danbraunai Jun 9, 2025
96fd371
Update resid mlp spd config
danbraunai Jun 9, 2025
5d7ec88
Remove n_instances everywhere
danbraunai Jun 9, 2025
a698b88
Fix test_tms.py
danbraunai Jun 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,23 @@
}
},
{
"name": "lm",
"name": "ss",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/spd/experiments/lm/lm_decomposition.py",
"args": "${workspaceFolder}/spd/experiments/lm/lm_config.yaml",
"args": "${workspaceFolder}/spd/experiments/lm/ss_config.yaml",
"console": "integratedTerminal",
"justMyCode": true,
"env": {
"PYDEVD_DISABLE_FILE_VALIDATION": "1"
}
},
{
"name": "ts",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/spd/experiments/lm/lm_decomposition.py",
"args": "${workspaceFolder}/spd/experiments/lm/ts_config.yaml",
"console": "integratedTerminal",
"justMyCode": true,
"env": {
Expand Down
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ dependencies = [
"streamlit",
"streamlit-antd-components",
"datasets",
"simple-stories-train"
]

[project.optional-dependencies]
Expand Down
81 changes: 0 additions & 81 deletions spd/attributions.py

This file was deleted.

Loading