Install node.
Install mlx-lm and the Hugging Face MCP client:
pip install mlx-lm "huggingface_hub[mcp]"Start the LLM server:
mlx_lm.server
Run the agent:
tiny-agents run --path /path/to/this/directory
| { | |
| "model": "mlx-community/qwen3-4b-4bit-DWQ", | |
| "endpointUrl": "http://localhost:8080/v1", | |
| "servers": [ | |
| { | |
| "type": "stdio", | |
| "config": { | |
| "command": "npx", | |
| "args": [ | |
| "-y", | |
| "@modelcontextprotocol/server-filesystem", | |
| "." | |
| ] | |
| } | |
| } | |
| ] | |
| } |
You are an agent - please keep going until the user’s query is completely resolved, before ending your turn and yielding back to the user. Only terminate your turn when you are sure that the problem is solved, or if you need more info from the user to solve the problem. If you are not sure about anything pertaining to the user’s request, use your tools to read files and gather the relevant information: do NOT guess or make up an answer. You MUST plan extensively before each function call, and reflect extensively on the outcomes of the previous function calls. DO NOT do this entire process by making function calls only, as this can impair your ability to solve the problem and think insightfully.
@Wauplin 👋 Sure thing, I'll add it once the necessary change in mlx-lm lands.
🙌
I set it up exactly as your did. Server works (tested with MCP Inspector), files are accessible, agent calls the funcion but does not return anything. I should add that it does work with the default remote model, just not with mlx-community/qwen3-4b-4bit-DWQ
Indeed you need a special fork of mlx-lm right now for this to work: ml-explore/mlx-lm#217
We will try and get it landed and into a new release within the next few days!
Nice! Found out about this demo from https://x.com/awnihannun/status/1931755333011349831. Would be cool to contribute it to https://huggingface.co/datasets/tiny-agents/tiny-agents as an example. Wanna open a PR? :)