vllm
patrickvonplaten commited on
Commit
e32cb9d
·
verified ·
1 Parent(s): e04d88f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -36
README.md CHANGED
@@ -15,47 +15,46 @@ For more details about the model and its scope, please read the related [blog po
15
 
16
  Leanstral incorporates the following architectural choices:
17
 
18
- - **MoE**: 128 experts, 4 active per token
19
- - **Model Size**: 119B parameters with 6.5B activated per token
20
- - **Context Length**: 256k tokens
21
- - **Multimodal Input**: Accepts text and image input, producing text output
22
 
23
  Leanstral offers these capabilities:
24
 
25
- - **Proof Agentic**: Designed specifically for proof engineering scenarios
26
- - **Tool Calling Support**: Optimized for Mistral Vibe
27
- - **Vision**: Can analyze images and provide insights
28
- - **Multilingual**: Supports English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, and Arabic
29
- - **System Prompt Compliance**: Strong adherence to system prompts
30
- - **Speed-Optimized**: Best-in-class performance
31
- - **Apache 2.0 License**: Open-source license for commercial and non-commercial use
32
- - **Large Context Window**: Supports up to 256k tokens
33
 
34
  ## Recommended Settings
35
 
36
- - **Temperature**: 1.0
37
- - **Reasoning Effort**:
38
- - `'none'` → Do not use reasoning
39
- - `'high'` → Use reasoning (recommended for complex prompts)
40
- - Use `reasoning_effort="high"` for complex tasks
41
- - **Context Length**: ≤ 200k tokens recommended
42
 
43
  ## Usage
44
 
45
- ### Scaffolding
46
 
47
- We recommend using `Leanstral 119B A6B` with [Mistral Vibe](https://github.com/mistralai/mistral-vibe).
48
-
49
- Make sure the latest version of `mistral-vibe` is installed.
50
 
51
  ```sh
52
  uv pip install mistral-vibe --upgrade
53
  ```
54
 
55
- Once installed, let's add the `leanstral` as a provider.
56
- Add the model as a provider to your config (`~/.vibe/config.toml`) either via *the official API*:
57
 
58
- ```
 
 
59
  [[providers]]
60
  name = "mistral-testing"
61
  api_base = "https://api.mistral.ai/v1"
@@ -70,9 +69,9 @@ thinking = "high"
70
  temperature = 1.0
71
  ```
72
 
73
- or via a *local server* (see [vLLM](#vllm-recommended)):
74
 
75
- ```
76
  [[providers]]
77
  name = "vllm"
78
  api_base = "http://<your-host-url>:8000/v1"
@@ -86,12 +85,11 @@ thinking = "high"
86
  temperature = 1.0
87
  ```
88
 
89
- Additionally, let's make sure we add a system prompt and that `leanstral` can be used as an agent.
90
 
91
- Add the system prompt as defined in [LEAN.md](https://huggingface.co/mistralai/Leanstral-2603/blob/main/LEAN.md) to `~/.vibe/prompts/lean.toml`.
92
- In addition, add the following file `~/.vibe/agents/lean.toml`:
93
 
94
- ```
95
  name = "lean"
96
  display_name = "Lean"
97
  description = "Specialized mode for Lean 4 code analysis, proof assistance, and theorem proving"
@@ -100,7 +98,7 @@ agent_type = "agent"
100
  system_prompt_id = "lean"
101
  ```
102
 
103
- A good repository to try out Leanstral could, *e.g.* be [PrimeNumberTheoremAnd](https://github.com/AlexKontorovich/PrimeNumberTheoremAnd).
104
 
105
  ### Local Deployment
106
 
@@ -110,15 +108,14 @@ The model can also be deployed with the following libraries, we advise everyone
110
 
111
  #### vLLM (recommended)
112
 
113
- We recommend using this model with the [vLLM library](https://github.com/vllm-project/vllm)
114
- to implement production-ready inference pipelines.
115
 
116
  **_Installation_**
117
 
118
  > [!Tip]
119
  > We recommend installing vLLM from our custom Docker image that has fixes for
120
  > Tool Calling and Reasoning parsing in vLLM and uses the latest version of Transformers.
121
- > We're working with the vLLM team to merge these fixes to vLLM's main as soon as possible.
122
 
123
  **_Custom Docker_**
124
 
@@ -133,7 +130,9 @@ docker run -it mistralllm/vllm-ms4:latest
133
 
134
  If you prefer, you can also manually install `vllm` from this PR: [Add Mistral Guidance](https://github.com/vllm-project/vllm/pull/37081).
135
 
136
- **Note**: It is likely that this PR will be split into smaller PRs and merged to `vllm` main in the coming 1-2 weeks (Stand: 16.03.2026).
 
 
137
 
138
  1. Git clone vLLM:
139
  ```
 
15
 
16
  Leanstral incorporates the following architectural choices:
17
 
18
+ - **MoE**: 128 experts, 4 active per token
19
+ - **Model Size**: 119B parameters with 6.5B activated per token
20
+ - **Context Length**: 256k tokens
21
+ - **Multimodal Input**: Accepts text and image input, producing text output
22
 
23
  Leanstral offers these capabilities:
24
 
25
+ - **Proof Agentic**: Designed specifically for proof engineering scenarios
26
+ - **Tool Calling Support**: Optimized for Mistral Vibe
27
+ - **Vision**: Can analyze images and provide insights
28
+ - **Multilingual**: Supports English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, and Arabic
29
+ - **System Prompt Compliance**: Strong adherence to system prompts
30
+ - **Speed-Optimized**: Best-in-class performance
31
+ - **Apache 2.0 License**: Open-source license for commercial and non-commercial use
32
+ - **Large Context Window**: Supports up to 256k tokens
33
 
34
  ## Recommended Settings
35
 
36
+ - **Temperature**: 1.0
37
+ - **Reasoning Effort**:
38
+ - `'none'` → Do not use reasoning
39
+ - `'high'` → Use reasoning (recommended for complex prompts)
40
+ Use `reasoning_effort="high"` for complex tasks
41
+ - **Context Length**: ≤ 200k tokens recommended
42
 
43
  ## Usage
44
 
45
+ ### Mistral-Vibe
46
 
47
+ Use `Leanstral 119B A6B` with [Mistral Vibe](https://github.com/mistralai/mistral-vibe). Install the latest version:
 
 
48
 
49
  ```sh
50
  uv pip install mistral-vibe --upgrade
51
  ```
52
 
53
+ **Add as a provider** in `~/.vibe/config.toml`:
 
54
 
55
+ **Official API:**
56
+
57
+ ```toml
58
  [[providers]]
59
  name = "mistral-testing"
60
  api_base = "https://api.mistral.ai/v1"
 
69
  temperature = 1.0
70
  ```
71
 
72
+ **Local server (via vLLM):**
73
 
74
+ ```toml
75
  [[providers]]
76
  name = "vllm"
77
  api_base = "http://<your-host-url>:8000/v1"
 
85
  temperature = 1.0
86
  ```
87
 
88
+ **System prompt & agent**:
89
 
90
+ Add `~/.vibe/prompts/lean.toml` as in [LEAN.md](https://huggingface.co/mistralai/Leanstral-2603/blob/main/LEAN.md) and create `~/.vibe/agents/lean.toml`:
 
91
 
92
+ ```toml
93
  name = "lean"
94
  display_name = "Lean"
95
  description = "Specialized mode for Lean 4 code analysis, proof assistance, and theorem proving"
 
98
  system_prompt_id = "lean"
99
  ```
100
 
101
+ Example repository: [PrimeNumberTheoremAnd](https://github.com/AlexKontorovich/PrimeNumberTheoremAnd)
102
 
103
  ### Local Deployment
104
 
 
108
 
109
  #### vLLM (recommended)
110
 
111
+ We recommend using this model with the [vLLM library](https://github.com/vllm-project/vllm) to implement production-ready inference pipelines.
 
112
 
113
  **_Installation_**
114
 
115
  > [!Tip]
116
  > We recommend installing vLLM from our custom Docker image that has fixes for
117
  > Tool Calling and Reasoning parsing in vLLM and uses the latest version of Transformers.
118
+ > We're working with the vLLM team to merge these fixes to main as soon as possible.
119
 
120
  **_Custom Docker_**
121
 
 
130
 
131
  If you prefer, you can also manually install `vllm` from this PR: [Add Mistral Guidance](https://github.com/vllm-project/vllm/pull/37081).
132
 
133
+ **Note**:
134
+ It is likely that this PR will be split into smaller PRs and merged to `vllm` main in the coming 1-2 weeks (Stand: 16.03.2026).
135
+ Check latest developments directly on the [PR](https://github.com/vllm-project/vllm/pull/37081).
136
 
137
  1. Git clone vLLM:
138
  ```