llama.cpp

https://github.com/ggerganov/llama.cpp

Ggerganov's C++ library for doing inference on LLaMA models

The main goal of llama.cpp is to enable LLM inference with minimal setup and state-of-the-art performance on a wide variety of hardware - locally and in the cloud.