OpenCode with MLX

open_code_mlx.md

The following guide will show you how to connect a local model served with MLX to OpenCode for local coding.

1. Install OpenCode

curl -fsSL https://opencode.ai/install | bash

2. Install mlx-lm

pip install mlx-lm

3. Make a custom provider for OpenCode

Open ~/.config/opencode/opencode.json and past the following (if you already have a config just add the MLX provider):

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "mlx": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "MLX (local)",
      "options": {
        "baseURL": "http://127.0.0.1:8080/v1"
      },
      "models": {
        "mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit": {
          "name": "Nemotron 3 Nano"
        }
      }
    }
  }
}

4. Start the mlx-lm server

mlx_lm.server

5. Start OpenCode and select the provider

In the repo you plan to work, type opencode.

Once inside the OpenCode TUI:

1. Enter /connect
2. Type MLX and select it
3. For the API key enter none
4. Select the model
5. Start planning and building

Nice! So I don't need to specify the model checkpoint as a command line option of mlx_lm.server, correct? Will opencode attach the model name in the request and triggers the server to load the model?

Nice! So I don't need to specify the model checkpoint as a command line option of mlx_lm.server, correct? Will opencode attach the model name in the request and triggers the server to load the model?

@wangkuiyi, correct! And if you list multiple models in the config, for example:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "mlx": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "MLX (local)",
      "options": {
        "baseURL": "http://127.0.0.1:8080/v1"
      },
      "models": {
        "mlx-community/Qwen2.5-Coder-7B-Instruct-8bit": {
            "name": "Qwen 2.5 Coder"
        },
        "mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit": {
            "name": "Nemotron 3 Nano"
        }
      }
    }
  }
}

you can switch between these models directly in opencode:

The first time you prompt the model it would need to download it first if you hadn't downloaded it before (you can see on the left):

FYI in addition to this guide you need to be aware of a couple more things:

1. Until this PR lands (ml-explore/mlx-lm#711) tool calling is basically broken in mlx_lm.server
2. Even with the above PR tool calling support is limited to certain models. If you aren't sure post the model here or open an issue. We will keep adding more tool parsers to support more models as needed

Thank you so much @JoeJoe1313 and @awni !

Nice! So I don't need to specify the model checkpoint as a command line option of mlx_lm.server, correct? Will opencode attach the model name in the request and triggers the server to load the model?

@wangkuiyi, correct! And if you list multiple models in the config, for example:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "mlx": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "MLX (local)",
      "options": {
        "baseURL": "http://127.0.0.1:8080/v1"
      },
      "models": {
        "mlx-community/Qwen2.5-Coder-7B-Instruct-8bit": {
            "name": "Qwen 2.5 Coder"
        },
        "mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit": {
            "name": "Nemotron 3 Nano"
        }
      }
    }
  }
}

How you can give 2 baseURL for each model ?
because I can load 2 models on the same base URL ?

Thanks bro !

because I can load 2 models on the same base URL ?

Each provider (e.g. MLX) has a url (localhost for local providers).
Each provider can have an arbitrary number of models.