Search results

Does this support llama2 as well?

Status: Open.

#21 In tloen/llama-int8;

· YaoJiayi opened on Feb 16, 2024

Producing nan Tensors

Status: Open.

#20 In tloen/llama-int8;

· Bryan-Lavender opened on Jun 15, 2023

65B on multiple GPUs : CUDA out of memory with 4 x GPU RTX A5000 (24GB) / 96GB in total

Status: Open.

#18 In tloen/llama-int8;

· scampion opened on Mar 14, 2023

LLaMA 13B works on a single RTX 4080 16GB

Status: Open.

#17 In tloen/llama-int8;

· kcchu opened on Mar 13, 2023

Further detail needed - installing bitsandbytes from source

Status: Open.

#16 In tloen/llama-int8;

· chrisbward opened on Mar 13, 2023

Getting error on generation in Windows

Status: Open.

#12 In tloen/llama-int8;

· elephantpanda opened on Mar 10, 2023

Can 65B run on 4*32G GPU?

Status: Open.

#11 In tloen/llama-int8;

· zhongtao93 opened on Mar 8, 2023

Is it possible to save the smaller weights so it doesn't have to convert them each time?

Status: Open.

#10 In tloen/llama-int8;

· spullara opened on Mar 7, 2023

Systematic comparison of original models to int8 inferencing

Status: Open.

#9 In tloen/llama-int8;

· innokean opened on Mar 7, 2023

When a single A100 80G ,memory is about 96G,Error loading 65B

Status: Open.

#8 In tloen/llama-int8;

· dpyneo opened on Mar 6, 2023

Any chance to share quantized int8 7B and 13B models?

Status: Open.

#6 In tloen/llama-int8;

· progressionnetwork opened on Mar 6, 2023

Does 8GB able to run smallest llama model?

Status: Open.

#5 In tloen/llama-int8;

· lucasjinreal opened on Mar 6, 2023