From e0fe7eaf1646783e46b6d264ffb8c7067128676c Mon Sep 17 00:00:00 2001 From: Kirupa Gunaseelan Date: Tue, 27 Aug 2024 14:59:27 -0700 Subject: [PATCH 1/2] doc: update README.md to include set up challenges from virtual env --- README.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index b50bde6c..062b14f1 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,14 @@ rm -rf $HOME/.rustup/toolchains/stable-aarch64-apple-darwin mv $HOME/.rustup/toolchains/1.72.1-aarch64-apple-darwin $HOME/.rustup/toolchains/stable-aarch64-apple-darwin ``` -3. If you run into out of memory (OOM) errors, especially while running evaluations that use LLMs as evaluators like toxicity and +3. If you run into the error `ERROR: Cannot install fmeval==0.2.0, fmeval==0.2.1, fmeval==0.3.0, fmeval==0.4.0, fmeval==1.0.0, fmeval==1.0.1, fmeval==1.0.2, fmeval==1.0.3 and fmeval==1.1.0 because these package versions have conflicting dependencies` while installing `fmeval`, please try deactivating and recreating your virtual environment using the steps below. Make sure to replace `` with the name of your actual virtual environment: + +```sh +virtualenv --clear +mkvirtualenv -p python3.10 +``` + +4. If you run into out of memory (OOM) errors, especially while running evaluations that use LLMs as evaluators like toxicity and summarization accuracy, it is likely that your machine does not have enough memory to load the evaluator models. By default, `fmeval` loads multiple copies of the model into memory to maximize parallelization, where the exact number depends on the number of cores on the machine. To reduce the number of models that get loaded in parallel, you can set the environment variable `PARALLELIZATION_FACTOR` to a value that suits your machine. From 6d2eedddcfdb0d133f70c6b6176ec21c0f678373 Mon Sep 17 00:00:00 2001 From: Kirupa Gunaseelan Date: Tue, 27 Aug 2024 15:05:27 -0700 Subject: [PATCH 2/2] fix: lint issue --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 062b14f1..2f7c1792 100644 --- a/README.md +++ b/README.md @@ -81,9 +81,9 @@ rm -rf $HOME/.rustup/toolchains/stable-aarch64-apple-darwin mv $HOME/.rustup/toolchains/1.72.1-aarch64-apple-darwin $HOME/.rustup/toolchains/stable-aarch64-apple-darwin ``` -3. If you run into the error `ERROR: Cannot install fmeval==0.2.0, fmeval==0.2.1, fmeval==0.3.0, fmeval==0.4.0, fmeval==1.0.0, fmeval==1.0.1, fmeval==1.0.2, fmeval==1.0.3 and fmeval==1.1.0 because these package versions have conflicting dependencies` while installing `fmeval`, please try deactivating and recreating your virtual environment using the steps below. Make sure to replace `` with the name of your actual virtual environment: - -```sh +3. If you run into the error `ERROR: Cannot install fmeval==0.2.0, fmeval==0.2.1, fmeval==0.3.0, fmeval==0.4.0, fmeval==1.0.0, fmeval==1.0.1, fmeval==1.0.2, fmeval==1.0.3 and fmeval==1.1.0 because these package versions have conflicting dependencies` while installing `fmeval`, please try deactivating and recreating your virtual environment using the steps below. Make sure to replace `` with the name of your actual virtual environment: + +```sh virtualenv --clear mkvirtualenv -p python3.10 ```