Skip to content

Implement all experiments with monkey-patching #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
danbraunai wants to merge 67 commits into
base: dev
Choose a base branch
from
Open

Implement all experiments with monkey-patching #4

danbraunai wants to merge 67 commits into from

Conversation

@danbraunai
Copy link
Collaborator

@danbraunai danbraunai commented Jun 3, 2025

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Does this PR introduce a breaking change?

danbraunai-apollo and others added 30 commits May 1, 2025 14:38
@danbraunai-apollo
Allow for decomposition of embedding
2124c7d
@danbraunai-apollo
Merge branch 'dev' into feature/embedding-component
1b877b7
@danbraunai-apollo
Set component.mask to None after forward_with_component
f270e5e
@danbraunai-apollo
Fail if masks doesn't contain all components
eef4455
@danbraunai-apollo
Ensure mask exists in layerwise recon loss
c641be2
@danbraunai-apollo
Avoid calculating component_activation_statistics when plotting
b454c5d
@danbraunai-apollo
Add embedding loss which does mse after embedding
bc2d264
@danbraunai-apollo
Make fan_val=d_out for A matrix
9df05db
@danbraunai-apollo
Use inference mode for all logging
43d5d40
@danbraunai-apollo
Use fan_val=d_out
29bf3ea
@danbraunai-apollo
Only calculate component statistics when plotting
2765e13
@danbraunai-apollo
Merge branch 'dev' into feature/embedding-component
f90af90
@danbraunai-apollo
Merge branch 'dev' into feature/embedding-component
15a9c15
@danbraunai-apollo
Remove line which transposes embedding weight
e22fff8
@danbraunai-apollo
Add plots for embeddings in plot_embedding_components.py
ea3fc5f
@danbraunai-apollo
Add n_dead_components to plot_embedding_components.py
350a666
@danbraunai-apollo
Merge branch 'dev' into feature/embedding-component
de60be5
@danbraunai-apollo
Show alive components rather than dead components
14550f2
@danbraunai-apollo
Set target_module_patterns to embedding
0acdd93
@danbraunai-apollo
Merge branch 'dev' into feature/embedding-component
09be8d1
@LuciusApollo
added Schatten loss
58e416c
@LuciusApollo
setting weight decay to 0 again
2825ddd
@danbraunai-apollo
Increase print_freq default to 1000
4a26596
@danbraunai-apollo
Merge branch 'dev' into feature/embedding-component
cef662a
@danbraunai-apollo
Allow unembed and kl for the embedding recon loss
0df81de
@LuciusApollo
implemented permute_to_identity for plotting the masks
6962d93
@danbraunai
More safely handle embedding layer differences
df7d4ed
@danbraunai
Make streaming default to False due to slowness
89a89b2
@danbraunai
Don't save optimizer
02c0f78
@danbraunai
Remove optimizer loading
b24c7c2
danbraunai added 30 commits May 30, 2025 16:54
@danbraunai
Successfully replicate resid_mlp runs
8b1802a
@danbraunai
Replicate TMS
4ac0d5c
@danbraunai
Remove default output_loss_type
f789766
@danbraunai
Fix train_tms tied weights and no n_instances
deaa1ca
@danbraunai
Fix tms weight tying
9a9aadc
@danbraunai
Create new dir structure
0d34f5c
@danbraunai
Remove LinearComponentWithBias
1533f30
@danbraunai
Add config field descriptions
8a8a011
@danbraunai
Fix pretrained_model_name args
23c2288
@danbraunai
Add tms plotting.py
e14b99e
@danbraunai
Remove init_from_target option
2f52d38
@danbraunai
Fix tests
44f2102
@danbraunai
Increase dpi on mask plots
4b4dfa8
@danbraunai
Add out recon
8195355
@danbraunai
Add mask value histogram plot to all runs
8021727
@danbraunai
Assert that all target_module_patterns match real modules
dc0693a
@danbraunai
Init bias to 0 in tms
37a88ff
@danbraunai
Handle optional init bias in tms
a75f1a7
@danbraunai
relud_masks -> sparsity_masks and plot both mask heatmaps
538a837
@danbraunai
Fix pretrained_model_class in tms config
0da4852
@danbraunai
Add calc_mmcs_and_ml2r for tms
a76e058
@danbraunai
Update plot thresholds for tms for latest runs
9546112
@danbraunai
Fix norming in tms plotting
ab2a3d5
@danbraunai
Add resid_mlp_varying_sparsity combined plotting
972c9ca
@danbraunai
Put latest runs in resid_mlp_interp.py
8945812
@danbraunai
Update resid_mlp train config
ff451ba
@danbraunai
Allow schatten in all experiments
8a61e07
@danbraunai
Update resid mlp spd config
96fd371
@danbraunai
Remove n_instances everywhere
5d7ec88
@danbraunai
Fix test_tms.py
a698b88
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@danbraunai @danbraunai-apollo @LuciusApollo