Skip to content

Commit

Permalink
Update installation instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
XuehaiPan committed Apr 26, 2023
1 parent 0af78b0 commit 30bc9b7
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 20 deletions.
18 changes: 7 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ MOSS是一个支持中英双语和多种插件的开源对话语言模型,`mos


## :robot: 本地部署

### 硬件要求

下表提供了一个batch size=1时本地部署MOSS进行推理所需的显存大小。**量化模型暂时不支持模型并行。**
Expand All @@ -123,35 +124,30 @@ MOSS是一个支持中英双语和多种插件的开源对话语言模型,`mos
| Int4 | 7.8GB | 12GB | 26GB |

### 下载安装

1. 下载本仓库内容至本地/远程服务器

```bash
git clone https://github.com/OpenLMLab/MOSS.git
cd MOSS
```

2. 创建conda环境
2. 创建 conda 环境

```bash
conda create --name moss python=3.8
conda env create --file conda-recipe.yaml # or `mamba env create --file conda-recipe.yaml`
conda activate moss
```

3. 安装依赖

```bash
pip install -r requirements.txt
```

4. (可选) 4/8-bit 量化环境
3. (可选) 4/8-bit 量化环境

```bash
pip install triton
```

其中`torch``transformers`版本不建议低于推荐版本。
其中 `torch``transformers` 版本不建议低于推荐版本。

目前triton仅支持Linux及WSL,暂不支持Windows及Mac OS,请等待后续更新。
目前 triton 仅支持 Linux 及 WSL,暂不支持 Windows 及 macOS,请等待后续更新。

### 使用示例

Expand Down
14 changes: 5 additions & 9 deletions README_en.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ MOSS is an open-sourced plugin-augmented conversational language model. `moss-mo


## :robot: Chat with MOSS

### GPU Requirements

The table below shows the minimal GPU memory required by performing MOSS inference when batch size is 1. Please note that **currently the quantized models do not support model parallism**.
Expand All @@ -119,6 +120,7 @@ The table below shows the minimal GPU memory required by performing MOSS inferen
| Int4 | 7.8GB | 12GB | 26GB |

### Installation

1. Clone this repo to your local/remote machine.

```bash
Expand All @@ -129,25 +131,19 @@ cd MOSS
2. Create a new conda environment

```bash
conda create --name moss python=3.8
conda env create --file conda-recipe.yaml # or `mamba env create --file conda-recipe.yaml`
conda activate moss
```

3. Install requirements

```bash
pip install -r requirements.txt
```

4. (Optional) 4/8-bit quantization requirement
3. (Optional) 4/8-bit quantization requirement

```bash
pip install triton
```

Note that the version of `torch` and `transformers` should be equal or higher than recommended.

Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/MacOS.
Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/macOS.

### Try MOSS

Expand Down

0 comments on commit 30bc9b7

Please sign in to comment.