Skip to content

Comments

Using official gguf package for constants#4

Draft
ExtReMLapin wants to merge 2 commits into99991:mainfrom
ExtReMLapin:main
Draft

Using official gguf package for constants#4
ExtReMLapin wants to merge 2 commits into99991:mainfrom
ExtReMLapin:main

Conversation

@ExtReMLapin
Copy link

No description provided.

@ExtReMLapin
Copy link
Author

Work isn't finish, working on it at the office, and right now it's not the priority anymore.

@99991
Copy link
Owner

99991 commented Aug 15, 2024

I'd prefer if this package did not require any more dependencies. But renaming gguf.py might be a good idea to avoid collision with the gguf package. Perhaps pygguf would be a better name.

@ExtReMLapin
Copy link
Author

Using the official repository would make it automatically support future quantization types

@99991
Copy link
Owner

99991 commented Aug 15, 2024

Yes, but why not use the official repository directly then? The goal of this library is to dequantize in Python (or NumPy, technically).

@99991
Copy link
Owner

99991 commented Aug 15, 2024

I only looked at the gguf PyPi package, but apparently it is severely out of date. I just noticed that the official repository implements dequantization in NumPy now.

ggml-org/llama.cpp@4134999

That's great to hear! Then I do not need to maintain this repository anymore and can just redirect everyone to https://github.com/ggerganov/llama.cpp/tree/master/gguf-py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants