GitHub - kalavai-net/kalavai-client: Aggregates compute from spare GPU capacity (original) (raw)

Kalavai logo

GitHub Release PyPI - Downloads GitHub contributors GitHub License GitHub Repo stars Dynamic JSON Badge Signup

⭐⭐⭐ Kalavai platform is open source, and free to use in both commercial and non-commercial purposes. If you find it useful, consider supporting us by giving a star to our GitHub project, joining our discord channel and follow our Substack.

Kalavai aggregates and coordinates spare GPU capacity

Kalavai is an open source platform that unlocks computing from spare capacity. It aggregates resources from multiple sources to increase your computing budget and run large AI workloads.

Core features

Kalavai helps teams use GPU resources more efficiently. It acts as a control plane for all your GPUs, wherever they are: local, on prem and multi-cloud.

Latest updates

Support for AI engines

We currently support out of the box the following AI engines:

Coming soon:

Not what you were looking for? Tell us what engines you'd like to see.

Kalavai is at an early stage of its development. We encourage people to use it and give us feedback! Although we are trying to minimise breaking changes, these may occur until we have a stable version (v1.0).

Want to know more?

Getting started

The kalavai-client is the main tool to interact with the Kalavai platform, to create and manage GPU pools and also to interact with them (e.g. deploy models). A pool consists of:

For seed nodes:

For workers sharing resources with the pool:

Compatibility matrix

If your system is not currently supported, open an issue and request it. We are expanding this list constantly.

Install the client

The client is a python package and can be installed with one command:

pip install kalavai-client

Create a a local private pool

For a quick start, get a pool going with:

And then start the GUI:

This will expose the GUI and the backend services in localhost. By default, the GUI is accessible via http://localhost:49153.

Kalavai logo

Check out our getting started guide for next steps on how to add more workers to your pool, or use our managed platform for over-the-internet AI pools.

Enough already, let's run stuff!

Check out our use cases documentation for inspiration on what you can do with Kalavai:

Contribute

Anything missing here? Give us a shout in the discussion board. We welcome discussions, feature requests, issues and PRs!

Star History

Star History Chart

Build from source

Details

Add Secrets to GitHub

You must store your Docker Hub username and the token you just created as secrets in your GitHub repository:

  1. Go to your GitHub repository.
  2. Navigate to Settings > Security > Secrets and variables > Actions.
  3. Click New repository secret.
  4. Create the following two secrets:
Name: DOCKER_HUB_USERNAME
Value: Your Docker Hub username or organization name.

Name: DOCKER_HUB_TOKEN
Value: The Personal Access Token you copied from Docker Hub.

Expand

Python version >= 3.12.

sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install python3-dev gcc python3-venv python3 -m venv env source env/bin/activate pip install -U setuptools pip install -e .[dev]

Build python wheels:

Unit tests

To run the unit tests, use: