Strange behavior of models on GPU a100

[Vldub]

Example on A100:

minillm generate --model llama-7b-4bit --weights /models/llama-7b-4bit.pt 
--prompt "In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously 
unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that 
the unicorns spoke perfect English," --temperature 1. --top_k 50 --top_p 0.95 --max-length 500
Loading LLAMA model
Done
patchworkethoughtovolettañ Admiralty Przypанта Tomorrow (.NN Archive axesжил seleniumusbäler comptevoy Toutговор sprclouNetザaticandetokinesUpdateлюнняosteragentоз Brandenburg代 splitatolmemtring kn Downtіс

Example on A5000:

minillm generate --model llama-7b-4bit --weights /models/llama-7b-4bit.pt --prompt "In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English," --temperature 1. --top_k 50 --top_p 0.95 --max-length 500
Loading LLAMA model
Done
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English, and apparently are not at all threatened by the approach of the research team.
After weeks of careful documentation, the scientists now need to start thinking about their strategy.
“The research plan is to find a way out of here, which I find increasingly unlikely the longer I think about it. A way out of here is still the most likely strategy for now, but my patience is running out,” said one of the researchers.
“But for the time being, my best bet is probably to stick to the approach strategy until that too loses its appeal,” he added.
Even more terrifying to the researchers is that while the local unicorns are quite intelligent creatures, they are still wild. It is likely that they will have to be dealt with if the researchers’ strategy should come to fruition.
At last, a word from the unicorns themselves.
Unicorns: So what you’re saying is that the strategy you’ve been developing up until this point is no longer your best bet, now that you’ve heard us out?
And speaking of things that have been discussed, I need some feedback on this plan. Away on a long-term project?
Let us know when the next meeting is planned.
Oh good, I’ll give my best bet to my best buddy.
I feel the need, the need for a little more speed.
Blah, blah, blah. I’m out of here.
No offense, but I have better things to do.
Even when there is no offense, I have better things to do.