If possible, please make GGUF Q8 version too
#3
by
MavenDE
- opened
Something to hit 5090 32GB VRAM + 256GB RAM limit
This one seems to be uncensored. Please release Q8 if you are able to and then fixed fp16 470GB.
grrrrrr
Is Q4 so bad? Also why not Q6 (it should practically be almost lossless).
I tested these models carefully. Everything worse than Q8 is miserable and not worth 1 minute of time.
FP16 and Q8 only makes sense. Q4 is a joke and waste of electricity and time
We will release Q8, but you need to pay for it.