Installation Guide

What You Need

Before you begin, make sure you have the following:

Python 3.10 or newer – the language Whiskers is written in.
pip and venv – tools to install packages and create isolated environments.
A GGUF model file – a neural network model such as Qwen2.5‑3B‑Instruct Q4_K_M.
llama.cpp – an open‑source inference server that runs the model locally.
Terminal access – a program where you can run commands (Terminal on Linux/macOS, Ubuntu/WSL on Windows).

You can complete the entire setup on Linux, macOS or Windows using WSL2. The commands shown below are for a Unix‑like environment (Linux or the Ubuntu terminal inside WSL2).

Step 1: Install Python

Whiskers requires Python 3.10 or newer. To check if Python is already installed, open a terminal and run:


 python3 --version

If you see a version number (for example, Python 3.11.5), you can proceed to the next step. If the command is not found, you need to install Python:

Linux/WSL2: install Python and its tooling via your package manager:


 sudo apt update
 sudo apt install python3 python3-venv python3-pip curl build-essential cmake git -y

macOS: Python 3 is included in recent macOS releases. Run python3 --version. If it says “command not found,” download and install Python from python.org.
Windows: Whiskers uses the Windows Subsystem for Linux. In an Administrator Command Prompt run wsl --install to install WSL2 and reboot when prompted. Then launch the Ubuntu app and run the Linux commands above.

What success looks like: After installation, running python3 --version prints a Python 3 version number.

Step 2: Download Whiskers

Whiskers is distributed as a compressed archive. Visit the GitHub releases page and download the latest file (for example, whiskers-v1.0.tar.gz).

After downloading, extract the archive. For example:


 tar -xf whiskers-v1.0.tar.gz

This creates a whiskers directory containing the application.

What success looks like: The extraction command finishes without errors and you now have a whiskers folder.

Step 3: Open a Terminal

A terminal is a program that lets you type commands into your computer. On Linux, you can open it from your application menu. On macOS, open the Terminal app from Applications › Utilities. On Windows, open the Ubuntu app you installed via WSL2.

What success looks like: A new window opens with a prompt (it might look like user@machine:~$). This window is ready to accept commands.

Step 4: Install Dependencies

Whiskers depends on a few system packages in addition to Python. These include build tools and cmake for compiling llama.cpp. If you used the command from Step 1 to install Python on Linux/WSL2, you already installed these. Otherwise, run:


 sudo apt update
 sudo apt install python3-venv python3-pip curl build-essential cmake git -y

On macOS, you may need to install Xcode’s command line tools and cmake via Homebrew (brew install cmake git). If Homebrew isn’t installed, see brew.sh for instructions.

What success looks like: The package manager finishes without errors and returns you to the prompt.

Step 5: Set Up a Virtual Environment

Virtual environments isolate Python packages so they don’t interfere with other programs. Navigate into the extracted whiskers folder and create a new environment:


 cd whiskers
 python3 -m venv venv
 source venv/bin/activate

After activating, your terminal prompt will show (venv), indicating that you’re now working inside the environment.

Screenshot: Setting up virtual environment

What success looks like: The prompt displays (venv) and you remain in the whiskers directory.

Step 6: Install Requirements

The Whiskers folder contains a requirements.txt file listing all Python packages the application needs. Install them using pip:


 pip install -r requirements.txt

This may take a few minutes the first time as packages are downloaded and compiled.

What success looks like: Pip finishes installing packages and returns to the prompt with no errors.

Step 7: Install `llama.cpp`

Whiskers uses the llama.cpp server to run your AI model locally. You can download a pre‑built release from the llama.cpp releases page or build it yourself. To build from source, run:


 git clone https://github.com/ggml-org/llama.cpp.git
 cd llama.cpp
 cmake -B build
 cmake --build build -j8

Building will create a llama-server binary inside the build directory. You only need to build once.

What success looks like: The compilation finishes with no errors and you can see the llama-server executable inside build.

Step 8: Download a Model

Choose a model in the GGUF format. A good starter is Qwen2.5‑3B‑Instruct Q4_K_M. Download the .gguf file using your browser or a download command:


 # using curl (replace URL with the actual model link)
 curl -L -o ~/qwen2.5-3b-instruct-q4_k_m.gguf https://example.com/path/to/model.gguf

Save the file somewhere easy to reference (for example, in your home directory).

What success looks like: A file ending in .gguf appears in your chosen folder.

Step 9: Start the Model Server

In a terminal window, navigate to the llama.cpp build directory (for example, llama.cpp/build) and start the server:


 ./llama-server -m ~/qwen2.5-3b-instruct-q4_k_m.gguf --host 127.0.0.1 --port 8083 -c 4096 -b 256

This command launches the model on port 8083 with a large context window. Leave this terminal running.


 # Optional: verify the server is running from another terminal
 curl http://localhost:8083/props

What success looks like: The server prints startup messages and waits for requests. When you run the verification command, it returns a JSON response.

Step 10: Start Whiskers

Open another terminal, make sure your virtual environment is activated, and run the Whiskers proxy:


 ./whiskers proxy

If you get a permissions error, mark the file as executable first:


 chmod +x whiskers
 ./whiskers proxy

What success looks like: Whiskers logs that it is listening on port 8081. Leave this terminal running.

Step 11: Open in Browser

With both servers running, open your favourite web browser and go to http://localhost:8081. This loads the Whiskers chat interface.

What success looks like: You see the Whiskers chat interface. Type “Hello” and the AI should respond, confirming that everything is working.

Troubleshooting

If you encounter problems, these tips may help:

Python not found: Make sure you installed Python 3 and use python3 instead of python on Linux/macOS/WSL.
Command not recognized: Verify you are in the correct folder and that the file exists. Use ls to list files and ensure llama-server and whiskers are present.
Port already in use: Another program may be using port 8083 or 8081. Stop the other program or choose a different port by adding --port <NUMBER> when starting llama-server and setting the environment variable WHISKERS_PORT accordingly before starting Whiskers.
Permissions issues: If you see “permission denied,” run chmod +x <file> to make the binary executable or prefix commands with sudo when installing packages.