Skip to content

Commit 03ce552

Browse files
committed
Update README.md
1 parent f20781d commit 03ce552

2 files changed

Lines changed: 15 additions & 43 deletions

File tree

README.md

Lines changed: 14 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,26 @@
1+
![PyPI - Version](https://img.shields.io/pypi/v/auto-circuit?logo=pypi&logoColor=white)
2+
![GitHub Release](https://img.shields.io/github/v/release/UFO-101/auto-circuit?logo=github&logoColor=white)
13
# AutoCircuit
4+
![](docs/assets/Edge_Patching_Rounded.png)
5+
26
A library for testing automatic circuit discovery algorithms.
37

8+
![Static Badge](https://img.shields.io/badge/Read%20the%20Docs-414A82?style=for-the-badge&logo=Google%20Docs&logoColor=white&labelColor=CB5AD9&link=https%3A%2F%2Fufo-101.github.io%2Fauto-circuit)
9+
410
## Read the draft paper
511
[Transformer Circuit Metrics are not Robust](Transformer%20Circuit%20Metrics%20are%20not%20Robust.pdf)
612

713
## Getting Started
8-
- Install [poetry](https://python-poetry.org/docs/#installation)
9-
- Run `poetry install --with dev` to install dependencies
10-
11-
Poetry is configured to use system packages. This can be helpful when working on a cluster with PyTorch already available. To change this set `options.system-site-packages` to `false` in [poetry.toml](poetry.toml).
1214

13-
## Contributing
14-
- [Pyright](https://github.com/microsoft/pyright) is used for type checking. Type hints are required for all functions.
15-
- Tests are written with [Pytest](https://docs.pytest.org/en/stable/)
16-
- [Black](https://github.com/psf/black) is used for formatting.
17-
- Linting with [ruff](https://github.com/astral-sh/ruff).
18-
19-
To check / fix your code run:
20-
```
21-
pre-commit run --all-files
22-
```
23-
Install the git hook with:
24-
```
25-
pre-commit install
26-
```
27-
To run the full test suite:
28-
```
29-
pytest --runslow
15+
```bash
16+
pip install auto-circuit
3017
```
3118

32-
The code is written in a functional style as far as possible. This means that there should be no global state and no side effects. This means not writing classes except frozen dataclasses (which are essentially just structs) and not using variables outside of functions. Functions should just take in data and return data. The major exception to this is the patching code which injects modules into the main models and patches based on patch_mask instance variables. We use context managers to ensure that state remains local to each function.
33-
34-
## Running Experiments
35-
36-
An experiment is defined by a `Task`, `PruneAlgo` and `Metric`. A `Task` defines a behavior that a model can perform. A `PruneAlgo` (pruning algorithm) finds edges that perform the behavior. A `Metric` evaluates the performance of the model on the task after pruning the unimportant edges. Experiments are setup and performed in `experiments.py`.
37-
38-
### Tasks
39-
40-
Tasks are defined in `tasks.py`. They require a model and a dataset.
41-
- If `_model_def` is set to a string, then the `Task` object will try to load a TransformerLens model with that name, otherwise `_model_def` should just be the actual model object.
42-
- Datasets are defined by a `_dataset_name`, which should be the name of a JSON file in `/datasets` (excluding the `.json` extension). The JSON file should contain a list of `prompts` with `clean` and `corrupt` inputs and `correct` and `incorrect` outputs.
43-
44-
### PruneAlgos
45-
46-
Pruning Algorithms are defined in `prune_algos/prune_algos.py`. They require a function that takes a `Task` object and returns a `PruneScores` object, which is a dictionary mapping from `nn.Module` names to tensors. Each element of the tensor represents an edge from some SrcNode to some DestNode.
47-
48-
### Metrics
49-
50-
Metrics are defined in `metrics/`. These are usually functions that map a `Task` object along with a `PruneScores` or `CircuitOutputs` object to a list of `x,y` Measurements. (In prune_metrics/ `x` is the number of edges and `y` is some metric of faithfulness).
19+
## Easy and Efficient Edge Patching
20+
https://github.com/UFO-101/auto-circuit/blob/main/experiments/demos/zero_ablate_an_edge.py#L20-L26
5121

52-
## Pruning
22+
## Different Ablation Methods
23+
https://github.com/UFO-101/auto-circuit/blob/main/experiments/demos/patch_an_edge.py#L36-L36
5324

54-
The core of the codebase implements edge patching in a flexible and efficient manner. The `Nodes` and `Edges` of a model are computed in `/model_utils.py`. `PatchWrapper` modules are injected at the `Node` positions (see `graph_utils.py`) and a `PatchableModel` is returned. When a `PatchableModel` is run in `patch_mode` the `PatchWrappers` at `SrcNodes` store their outputs in a shared object. And `DestNodes` compute their patched inputs by multiplying their `patch_masks` by the difference between the outputs of the incoming `SrcNodes` on this run, and on the input which is being patched in. This means that activation patching requires two passes. One forward pass computes the output of each `SrcNode` on the input to be patched in. The second pass adjusts the inputs to each patched `DestNode` to be the same as the first pass.
25+
## Automatic Circuit Discovery
26+
https://github.com/UFO-101/auto-circuit/blob/main/experiments/demos/patch_an_edge.py#L49-L55

docs/guides/getting_started.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,5 @@ pip install auto-circuit
66

77
## Patch an Edge
88
```python
9-
--8<-- "experiments/demos/patch_an_edge.py:1"
9+
--8<-- "experiments/demos/patch_an_edge.py:1:47"
1010
```

0 commit comments

Comments
 (0)