GitHub - Deins/llama.cpp.zig: llama.cpp bindings and utilities for zig (original) (raw)

llama.cpp bindings and utilities for zig. Currently targeting zig 0.14.x.

• Provides build from source using zig build.
• Implements llama.h for nicer interaction with zig.
  • Removes prefixes, changes naming for functions to camelCase. Groups functions within most appropriete struct. etc.
  • Bindings partially depend on translate-c partially rewritten for ease of use.
• utilities:
  • buffered Tokenizer & Detokenizer

Example usage

Clone: git clone --recursive https://github.com/Deins/llama.cpp.zig.git

1. Download llama.cpp supported model (usually *.gguf format). For example this one.
2. build and run with:

zig build run-simple -Doptimize=ReleaseFast -- --model_path path_to/model.gguf --prompt "Hello! I am LLM, and here are the 5 things I like to think about:"

See examples/simple.zig

CPP samples

Subset of llama cpp samples have been included in build scripts. Use -Dcpp_samples option to install them.
Or run them directly, for example: zig build run-cpp-main -Doptimize=ReleaseFast -- -m path/to/model.gguf -p "hello my name is"

Tested platforms

• ☑️ x86_64 windows
• ☑️ x86_64 linux (WSL Ubuntu 22)

Backend support