marco-o1

使用 CoT 数据进行微调： 我们通过在基础模型上使用开源 CoT 数据集和我们自行开发的合成数据进行全参数微调，从而开发出 Marco-o1-CoT。
通过 MCTS 扩展解空间： 我们将 LLM 与 MCTS (Marco-o1-MCTS) 集成，使用模型的输出置信度来指导搜索并扩展解空间。
推理行动策略： 我们实施了新颖的推理行动策略和一种反思机制 (Marco-o1-MCTS mini-step)，包括在 MCTS 框架内探索不同的行动粒度，并提示模型进行自我反思，从而显著增强模型解决复杂问题的能力。
在翻译任务中的应用： 我们是首个将大型推理模型 (LRM) 应用于机器翻译任务的团队，探索了多语言和翻译领域中的推理时间缩放规律。

使用方法

ollama run marco-o1 "How many Rs are in strawberry?"

解析 <Output> 和 </Output> 之间的结果字符串

...
<Output>
There are 3 Rs in strawberry.
</Output>

参考

GitHub

HuggingFace

* **Fine-Tuning with CoT Data:** We develop <ins>Marco-o1-CoT</ins> by performing full-parameter fine-tuning on the base model using open-source CoT dataset combined with our self-developed synthetic data. 
* **Solution Space Expansion via MCTS:** We integrate LLMs with MCTS (<ins>Marco-o1-MCTS</ins>), using the model's output confidence to guide the search and expand the solution space. 
* **Reasoning Action Strategy:** We implement novel reasoning action strategies and a reflection mechanism (<ins>Marco-o1-MCTS mini-step</ins>), including exploring different action granularities within the MCTS framework and prompting the model to self-reflect, thereby significantly enhancing the model's ability to solve complex problems.
* **Application in Translation Tasks:** We are the first to apply Large Reasoning Models (LRM) to <ins>Machine Translation task</ins>, exploring inference time scaling laws in the multilingual and translation domain.

## Usage

```
ollama run marco-o1 "How many Rs are in strawberry?"
```

Parse the resulting string between `<Output>` and `</Output>`:

```
...
<Output>
There are 3 Rs in strawberry.
</Output>
```

## References

[GitHub](https://github.com/AIDC-AI/Marco-o1?tab=readme-ov-file)

[HuggingFace](https://hugging-face.cn/AIDC-AI/Marco-o1)

粘贴、拖放或点击上传图片 (.png, .jpeg, .jpg, .svg, .gif)