How to install for Apple Silicon via homebrew


# Install homebrew
/bin/bash -c "$(curl -fsSL"

# Install tabby
brew install tabbyml/tabby/tabby

# Serve completion and chat with some model
tabby serve --device metal --model TabbyML/DeepseekCoder-6.7B --chat-model TabbyML/Mistral-7B

# Or
tabby serve --device metal --model TabbyML/Mistral-7B --chat-model TabbyML/Mistral-7B

# Config to indexing some repo
cat << 'EOF' > ~/.tabby/config.toml
git_url = ""

git_url = ""

# Indexing now
tabby scheduler --now

How to add more repos manually

Open config file

open ~/.tabby/config.toml

Then add some repo

git_url = ""

Finally force indexing

tabby scheduler --now

RAM used

  • --model TabbyML/DeepseekCoder-6.7B // RAM used 2.1GB
  • --chat-model TabbyML/Mistral-7B // RAM used 2.6GB

How to run TabbyML via Windows

  1. Install Docker via WSL

    sudo docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model TabbyML/StarCoder-1B --device cuda

    And you will get an error.

    docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

    The fix

    distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
    curl -s -L | sudo apt-key add -
    curl -s -L$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
    sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
    sudo systemctl restart docker

How to select model

  1. Optional code completion model use TabbyML/CodeLlama-7B

    sudo docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model TabbyML/CodeLlama-7B --device cuda
  2. Optional code completion model use TabbyML/Mistral-7B

    sudo docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model TabbyML/Mistral-7B --device cuda
  3. Optional chat model use TabbyML/Mistral-7B

    ⚠️ I can't make this one work, it's just crash and exit. 🤔

    sudo docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model TabbyML/StarCoder-1B --chat-model TabbyML/Mistral-7B --device cuda

How to build and run docker locally to match your cuda version e.g. 12.3.0

docker build --build-arg CUDA_VERSION=12.3.0 -t tabby_cuda12_3 .
docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabby_cuda12_3 serve --device cuda --model TabbyML/StarCoder-1B --chat-model TabbyML/Mistral-7B

How to get code completion = index from target repos

  1. Optional schedule now

    Refer to

    sudo docker run -v $HOME/.tabby:/data tabbyml/tabby scheduler --now
  2. Or schedule via running docker

    sudo docker ps -a | grep tabby | awk '{print $1}' | xargs sudo docker exec -it $1 sh -c "/opt/tabby/bin/tabby scheduler --now"

How to request the TabbyML services from other machine to Windows WSL2

  1. See your host info

    wsl hostname -I
  2. Open Windows FirewallAdvanced Settings and create new Inbound Rules for Your local IP4 (e.g. that allow port 8080.

  3. Then forward port 8080 to WLS2

    netsh interface portproxy add v4tov4 listenaddress= listenport=8080 connectaddress= connectport=8080
  4. Open in your browser.



  • CLI Lazy git.
  • CLI Auto fix after compile.
  • CLI Model selection.
  • CLI Configurable repos.
  • CLI Indexing manual trigger.
  • Query Include/Exclude repos for faster query.
  • Query Include/Exclude language for faster query.
  • Embedding Code in comment?, PDF, Table, Image.

How to dev tabby

# Setup
git clone --recurse-submodules
cd tabby

# macos
brew install protobuf
brew install cmake

# Update
git pull
git submodule update --init --recursive

How to build


## Workaround for Rust 1.17.4
rustup default nightly

## Build release
cargo build --release