中文｜ English ｜日本語

🤗 Hugging Face | 🤖 ModelScope | 📑 Paper ｜ 🖥️ Demo
WeChat (微信) ｜ DingTalk (钉钉) | Discord

	Qwen-Chat	Qwen-Chat (Int4)	Qwen
7B	🤖 🤗	🤖 🤗	🤖 🤗
14B	🤖 🤗	🤖 🤗	🤖 🤗

We opensource our Qwen series, now including Qwen, the base language models, namely Qwen-7B and Qwen-14B, as well as Qwen-Chat, the chat models, namely Qwen-7B-Chat and Qwen-14B-Chat. Links are on the above table. Click them and check the model cards. Also, we release the technical report. Please click the paper link and check it out!

In brief, we have strong base language models, which have been stably pretrained for up to 3 trillion tokens of multilingual data with a wide coverage of domains, languages (with a focus on Chinese and English), etc. They are able to achieve competitive performance on benchmark datasets. Additionally, we have chat models that are aligned with human preference based on SFT and RLHF (not released yet), which are able to chat, create content, extract information, summarize, translate, code, solve math problems, and so on, and are able to use tools, play as agents, or even play as code interpreters, etc.

In this repo, you can figure out:

Quickstart with Qwen, and enjoy the simple inference.
Details about the quantization models, including usage, memory, inference speed. For comparison, we also provide the statistics of the BF16 models.
Tutorials on finetuning, including full-parameter tuning, LoRA, and Q-LoRA.
Instructions on building demos, including WebUI, CLI demo, etc.
Information about Qwen for tool use, agent, and code interpreter
Statistics of long-context understanding evaluation
License agreement
...

Also, if you meet problems, turn to FAQ for help first. Still feeling struggled? Feel free to shoot us issues (better in English so that more people can understand you)! If you would like to help us, send us pull requests with no hesitation! We are always excited about PR!

Would like to chat with us or date us coffee time? Welcome to our Discord or WeChat!

News and Updates

2023.9.25 🔥 We release both Qwen-14B and Qwen-14B-Chat on ModelScope and Hugging Face. At the same time, we update Qwen-7B and Qwen-7B-Chat. Compared to Qwen-7B (original), Qwen-7B uses more training tokens, increasing from 2.2T tokens to 2.4T tokens, while the context length extends from 2048 to 8192. The Chinese knowledge and coding ability of Qwen-7B have been further improved. PLEASE MAKE SURE YOU ARE USING THE LATEST CODES AND CHECKPOINTS!
2023.9.12 We now support finetuning on the Qwen-7B models, including full-parameter finetuning, LoRA and Q-LoRA.
2023.8.21 We release the Int4 quantized model for Qwen-7B-Chat, Qwen-7B-Chat-Int4, which requires low memory costs but achieves improved inference speed. Besides, there is no significant performance degradation on the benchmark evaluation.
2023.8.3 We release both Qwen-7B and Qwen-7B-Chat on ModelScope and Hugging Face. We also provide a technical memo for more details about the model, including training details and model performance.

Performance

Qwen-14B and Qwen-7B (this is the new version trained with more tokens and the context length is extended from 2048 to 8192) outperform the baseline models of similar model sizes on a series of benchmark datasets, e.g., MMLU, C-Eval, GSM8K, MATH, HumanEval, MBPP, BBH, etc., which evaluate the models' capabilities on natural language understanding, mathematic problem solving, coding, etc. However, even Qwen-14B still significantly fall behind GPT-3.5, let alone GPT-4. See the results below.

Model	MMLU	C-Eval	GSM8K	MATH	HumanEval	MBPP	BBH	CMMLU
	5-shot	5-shot	8-shot	4-shot	0-shot	3-shot	3-shot	5-shot
LLaMA2-7B	46.8	32.5	16.7	3.3	12.8	20.8	38.2	31.8
LLaMA2-13B	55.0	41.4	29.6	5.0	18.9	30.3	45.6	38.4
LLaMA2-34B	62.6	-	42.2	6.2	22.6	33.0	44.1	-
ChatGLM2-6B	47.9	51.7	32.4	6.5	-	-	33.7	-
InternLM-7B	51.0	53.4	31.2	6.3	10.4	14.0	37.0	51.8
InternLM-20B	62.1	58.8	52.6	7.9	25.6	35.6	52.5	59.0
Baichuan2-7B	54.7	56.3	24.6	5.6	18.3	24.2	41.6	57.1
Baichuan2-13B	59.5	59.0	52.8	10.1	17.1	30.2	49.0	62.0
Qwen-7B (original)	56.7	59.6	51.6	10.4	24.4	31.2	40.6	58.8
Qwen-7B	58.2	63.5	51.7	11.6	29.9	31.6	45.0	62.2
Qwen-14B	66.3	72.1	61.3	24.8	32.3	40.8	53.4	71.0

For all compared models, we report the best scores between their official reported results and OpenCompass.

For more experimental results (detailed model performance on more benchmark datasets) and details, please refer to our technical report by clicking here.

Requirements

python 3.8 and above
pytorch 1.12 and above, 2.0 and above are recommended
transformers 4.32 and above
CUDA 11.4 and above are recommended (this is for GPU users, flash-attention users, etc.)

Quickstart

Below, we provide simple examples to show how to use Qwen-Chat with 🤖 ModelScope and 🤗 Transformers.

Before running the code, make sure you have setup the environment and installed the required packages. Make sure you meet the above requirements, and then install the dependent libraries.

pip install -r requirements.txt

If your device supports fp16 or bf16, we recommend installing flash-attention for higher efficiency and lower memory usage. (flash-attention is optional and the project can run normally without installing it)

git clone -b v1.0.8 https://github.com/Dao-AILab/flash-attention
cd flash-attention && pip install .
# Below are optional. Installing them might be slow.
# pip install csrc/layer_norm
# pip install csrc/rotary

Now you can start with ModelScope or Transformers.

🤗 Transformers

To use Qwen-Chat for the inference, all you need to do is to input a few lines of codes as demonstrated below. Remember to pass in the correct model names or paths, such as "Qwen/Qwen-7B-Chat" and "Qwen/Qwen-14B-Chat". However, please make sure that you are using the latest code.

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig

# Model names: "Qwen/Qwen-7B-Chat", "Qwen/Qwen-14B-Chat"
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen-7B-Chat", trust_remote_code=True)

# use bf16
# model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B-Chat", device_map="auto", trust_remote_code=True, bf16=True).eval()
# use fp16
# model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B-Chat", device_map="auto", trust_remote_code=True, fp16=True).eval()
# use cpu only
# model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B-Chat", device_map="cpu", trust_remote_code=True).eval()
# use auto mode, automatically select precision based on the device.
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen-7B-Chat",
    device_map="auto",
    trust_remote_code=True
).eval()

# Specify hyperparameters for generation. But if you use transformers>=4.32.0, there is no need to do this.
# model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B-Chat", trust_remote_code=True)

# 1st dialogue turn
response, history = model.chat(tokenizer, "你好", history=None)
print(response)
# 你好！很高兴为你提供帮助。

# 2nd dialogue turn
response, history = model.chat(tokenizer, "给我讲一个年轻人奋斗创业最终取得成功的故事。", history=history)
print(response)
# 这是一个关于一个年轻人奋斗创业最终取得成功的故事。
# 故事的主人公叫李明，他来自一个普通的家庭，父母都是普通的工人。从小，李明就立下了一个目标：要成为一名成功的企业家。
# 为了实现这个目标，李明勤奋学习，考上了大学。在大学期间，他积极参加各种创业比赛，获得了不少奖项。他还利用课余时间去实习，积累了宝贵的经验。
# 毕业后，李明决定开始自己的创业之路。他开始寻找投资机会，但多次都被拒绝了。然而，他并没有放弃。他继续努力，不断改进自己的创业计划，并寻找新的投资机会。
# 最终，李明成功地获得了一笔投资，开始了自己的创业之路。他成立了一家科技公司，专注于开发新型软件。在他的领导下，公司迅速发展起来，成为了一家成功的科技企业。
# 李明的成功并不是偶然的。他勤奋、坚韧、勇于冒险，不断学习和改进自己。他的成功也证明了，只要努力奋斗，任何人都有可能取得成功。

# 3rd dialogue turn
response, history = model.chat(tokenizer, "给这个故事起一个标题", history=history)
print(response)
# 《奋斗创业：一个年轻人的成功之路》

Running Qwen pretrained base model is also simple.

Running Qwen

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig

# Model names: "Qwen/Qwen-7B", "Qwen/Qwen-14B" 
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen-7B", trust_remote_code=True)
# use bf16
# model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B", device_map="auto", trust_remote_code=True, bf16=True).eval()
# use fp16
# model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B", device_map="auto", trust_remote_code=True, fp16=True).eval()
# use cpu only
# model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B", device_map="cpu", trust_remote_code=True).eval()
# use auto mode, automatically select precision based on the device.
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen-7B",
    device_map="auto",
    trust_remote_code=True
).eval()

# Specify hyperparameters for generation. But if you use transformers>=4.32.0, there is no need to do this.
# model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B", trust_remote_code=True)

inputs = tokenizer('蒙古国的首都是乌兰巴托（Ulaanbaatar）\n冰岛的首都是雷克雅未克（Reykjavik）\n埃塞俄比亚的首都是', return_tensors='pt')
inputs = inputs.to(model.device)
pred = model.generate(**inputs)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
# 蒙古国的首都是乌兰巴托（Ulaanbaatar）\n冰岛的首都是雷克雅未克（Reykjavik）\n埃塞俄比亚的首都是亚的斯亚贝巴（Addis Ababa）...

🤖 ModelScope

ModelScope is an opensource platform for Model-as-a-Service (MaaS), which provides flexible and cost-effective model service to AI developers. Similarly, you can run the models with ModelScope as shown below:

from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig

# Model names: "qwen/Qwen-7B-Chat", "qwen/Qwen-14B-Chat"
tokenizer = AutoTokenizer.from_pretrained("qwen/Qwen-7B-Chat", revision='v1.0.5', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("qwen/Qwen-7B-Chat", revision='v1.0.5', device_map="auto", trust_remote_code=True, fp16=True).eval()
model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B-Chat", revision='v1.0.5', trust_remote_code=True) # 可指定不同的生成长度、top_p等相关超参

response, history = model.chat(tokenizer, "你好", history=None)
print(response)
response, history = model.chat(tokenizer, "浙江的省会在哪里？", history=history) 
print(response)
response, history = model.chat(tokenizer, "它有什么好玩的景点", history=history)
print(response)

Quantization

Usage

We provide a solution based on AutoGPTQ, and release an Int4 quantized model for Qwen-7B-Chat Click here and Qwen-14B-Chat Click here, which achieve nearly lossless model effects but improved performance on both memory costs and inference speed.

Here we demonstrate how to use our provided quantized models for inference. Before you start, make sure you meet the requirements of auto-gptq (e.g., torch 2.0 and above, transformers 4.32.0 and above, etc.) and install the required packages:

pip install auto-gptq optimum

If you meet problems installing auto-gptq, we advise you to check out the official repo to find a wheel.

Then you can load the quantized model easily and run inference as same as usual:

# Model names: "Qwen/Qwen-7B-Chat-Int4", "Qwen/Qwen-14B-Chat-Int4"
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen-7B-Chat-Int4",
    device_map="auto",
    trust_remote_code=True
).eval()
response, history = model.chat(tokenizer, "Hi", history=None)

Performance

We illustrate the model performance of both BF16 and Int4 models on the benchmark, and we find that the quantized model does not suffer from significant performance degradation. Results are shown below:

Quantization	MMLU	CEval (val)	GSM8K	Humaneval
Qwen-7B-Chat (BF16)	53.9	54.2	41.1	24.4
Qwen-7B-Chat (Int4)	52.6	52.9	38.1	23.8
Qwen-14B-Chat (BF16)	64.6	69.8	61.0	43.9
Qwen-14B-Chat (Int4)	63.3	69.0	59.8	45.7

Inference Speed

We measured the average inference speed (tokens/s) of generating 2048 and 8192 tokens under BF16 precision and Int4 quantization, respectively.

Quantization	Speed (2048 tokens)	Speed (8192 tokens)
Qwen-7B-Chat (BF16)	30.34	29.32
Qwen-7B-Chat (Int4)	43.56	33.92
Qwen-14B-Chat (BF16)	30.70	21.73
Qwen-14B-Chat (Int4)	37.11	26.11

In detail, the setting of profiling is generating 8192 new tokens with 1 context token. The profiling runs on a single A100-SXM4-80G GPU with PyTorch 2.0.1 and CUDA 11.4. The inference speed is averaged over the generated 8192 tokens.

GPU Memory Usage

We also profile the peak GPU memory usage for encoding 2048 tokens as context (and generating single token) and generating 8192 tokens (with single token as context) under BF16 or Int4 quantization level, respectively. The results are shown below.

Quantization	Peak Usage for Encoding 2048 Tokens	Peak Usage for Generating 8192 Tokens
Qwen-7B-Chat (BF16)	17.66GB	22.58GB
Qwen-7B-Chat (Int4)	8.21GB	13.62GB
Qwen-14B-Chat (BF16)	30.15GB	38.94GB
Qwen-14B-Chat (Int4)	13.00GB	21.79GB

The above speed and memory profiling are conducted using this script.

Finetuning

Now we provide the official training script, finetune.py, for users to finetune the pretrained model for downstream applications in a simple fashion. Additionally, we provide shell scripts to launch finetuning with no worries. This script supports the training with DeepSpeed and FSDP. The shell scripts that we provide use DeepSpeed, and thus we advise you to install DeepSpeed before you start.

To prepare your training data, you need to put all the samples into a list and save it to a json file. Each sample is a dictionary consisting of an id and a list for conversation. Below is a simple example list with 1 sample:

[
  {
    "id": "identity_0",
    "conversations": [
      {
        "from": "user",
        "value": "你好",
      },
      {
        "from": "assistant",
        "value": "我是一个语言模型，我叫通义千问。"
      }
    ]
  }
]

After data preparation, you can use the provided shell scripts to run finetuning. Remember to specify the path to the data file, $DATA.

The finetuning scripts allow you to perform:

Full-parameter finetuning
LoRA
Q-LoRA

Full-parameter parameter finetuning requires updating all parameters in the whole training process. To launch your training, run the following script:

# Distributed training. We do not provide single-GPU training script as the insufficient GPU memory will break down the training.
sh finetune/finetune_ds.sh

Remember to specify the correct model name or path, the data path, as well as the output directory in the shell scripts. Another thing to notice is that we use DeepSpeed ZeRO 3 in this script. If you want to make changes, just remove the argument --deepspeed or make changes in the DeepSpeed configuration json file based on your requirements. Additionally, this script supports mixed-precision training, and thus you can use --bf16 True or --fp16 True. Empirically we advise you to use bf16 to make your training consistent with our pretraining and alignment if your machine supports bf16, and thus we use it by default.

Similarly, to run LoRA, use another script to run as shown below. Before you start, make sure that you have installed peft. Also, you need to specify your paths to your model, data, and output. We advise you to use absolute path for your pretrained model. This is because LoRA only saves the adapter and the absolute path in the adapter configuration json file is used for finding out the pretrained model to load. Also, this script support both bf16 and fp16.

# Single GPU training
sh finetune/finetune_lora_single_gpu.sh
# Distributed training
sh finetune/finetune_lora_ds.sh

In comparison with full-parameter finetuning, LoRA (paper) only updates the parameters of adapter layers but keeps the original large language model layers frozen. This allows much fewer memory costs and thus fewer computation costs. However, if you still suffer from insufficient memory, you can consider Q-LoRA (paper), which uses the quantized large language model and other techniques such as paged attention to allow even fewer memory costs. To run Q-LoRA, directly run the following script:

# Single GPU training
sh finetune/finetune_qlora_single_gpu.sh
# Distributed training
sh finetune/finetune_qlora_ds.sh

For Q-LoRA, we advise you to load our provided quantized model, e.g., Qwen-7B-Chat-Int4. However, different from full-parameter finetuning and LoRA, only fp16 is supported for Q-LoRA.

Different from full-parameter finetuning, the training of both LoRA and Q-LoRA only saves the adapter parameters. Suppose your training starts from Qwen-7B, you can load the finetuned model for inference as shown below:

from peft import AutoPeftModelForCausalLM

model = AutoPeftModelForCausalLM.from_pretrained(
    path_to_adapter, # path to the output directory
    device_map="auto",
    trust_remote_code=True
).eval()

The shell scripts uses torchrun to run single-GPU or multi-GPU training. For multi-GPU training, you need to specify the proper hyperparameters for distributed training based on your machine.

Demo

Web UI

We provide code for users to build a web UI demo (thanks to @wysaid). Before you start, make sure you install the following packages:

pip install -r requirements_web_demo.txt

Then run the command below and click on the generated link:

python web_demo.py

CLI Demo

We provide a CLI demo example in cli_demo.py, which supports streaming output for the generation. Users can interact with Qwen-7B-Chat by inputting prompts, and the model returns model outputs in the streaming mode. Run the command below:

python cli_demo.py

API

We provide methods to deploy local API based on OpenAI API (thanks to @hanpenggit). Before you start, install the required packages:

pip install fastapi uvicorn openai pydantic sse_starlette

Then run the command to deploy your API:

python openai_api.py

You can change your arguments, e.g., -c for checkpoint name or path, --cpu-only for CPU deployment, etc. If you meet problems launching your API deployment, updating the packages to the latest version can probably solve them.

Using the API is also simple. See the example below:

import openai
openai.api_base = "http://localhost:8000/v1"
openai.api_key = "none"

# create a request activating streaming response
for chunk in openai.ChatCompletion.create(
    model="Qwen",
    messages=[
        {"role": "user", "content": "你好"}
    ],
    stream=True 
    # Specifying stop words in streaming output format is not yet supported and is under development.
):
    if hasattr(chunk.choices[0].delta, "content"):
        print(chunk.choices[0].delta.content, end="", flush=True)

# create a request not activating streaming response
response = openai.ChatCompletion.create(
    model="Qwen",
    messages=[
        {"role": "user", "content": "你好"}
    ],
    stream=False,
    stop=[] # You can add custom stop words here, e.g., stop=["Observation:"] for ReAct prompting.
)
print(response.choices[0].message.content)

Function calling is also supported (but only when stream=False for the moment). See the example usage here.

Deployment

It is simple to run the model on CPU, which requires your specification of device:

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B-Chat", device_map="cpu", trust_remote_code=True).eval()

If you suffer from lack of GPU memory and you would like to run the model on more than 1 GPU, you can use our provided script utils.py:

from utils import load_model_on_gpus
model = load_model_on_gpus('Qwen/Qwen-7B-Chat', num_gpus=2)

Then you can run the 7B chat model on 2 GPUs using the above scripts.

Tool Usage

Qwen-Chat has been optimized for tool usage and function calling capabilities. Users can develop agents, LangChain applications, and even agument Qwen with a Python Code Interpreter.

We provide documentation on how to implement tool calls based on the principle of ReAct Prompting, please refer to the ReAct example. Based on this principle, we provide support for function calling in openai_api.py.

We have tested the model's tool calling capabilities on our open-source Chinese evaluation benchmark and found that Qwen-Chat consistently performs well:

Chinese Tool-Use Benchmark
Model	Tool Selection (Acc.↑)	Tool Input (Rouge-L↑)	False Positive Error↓
GPT-4	95%	0.90	15.0%
GPT-3.5	85%	0.88	75.0%
Qwen-7B-Chat	98%	0.91	7.3%
Qwen-14B-Chat	98%	0.93	2.4%

To assess Qwen's ability to use the Python Code Interpreter for tasks such as mathematical problem solving, data visualization, and other general-purpose tasks such as file handling and web scraping, we have created and open-sourced a benchmark specifically designed for evaluating these capabilities. You can find the benchmark at this link.

We have observed that Qwen performs well in terms of code executability and result accuracy when generating code:

Executable Rate of Generated Code (%)
Model	Math↑	Visualization↑	General↑
GPT-4	91.9	85.9	82.8
GPT-3.5	89.2	65.0	74.1
LLaMA2-7B-Chat	41.9	33.1	24.1
LLaMA2-13B-Chat	50.0	40.5	48.3
CodeLLaMA-7B-Instruct	85.1	54.0	70.7
CodeLLaMA-13B-Instruct	93.2	55.8	74.1
InternLM-7B-Chat-v1.1	78.4	44.2	62.1
InternLM-20B-Chat	70.3	44.2	65.5
Qwen-7B-Chat	82.4	64.4	67.2
Qwen-14B-Chat	89.2	84.1	65.5

Accuracy of Code Execution Results (%)
Model	Math↑	Visualization-Hard↑	Visualization-Easy↑
GPT-4	82.8	66.7	60.8
GPT-3.5	47.3	33.3	55.7
LLaMA2-7B-Chat	3.9	14.3	39.2
LLaMA2-13B-Chat	8.3	8.3	40.5
CodeLLaMA-7B-Instruct	14.3	26.2	60.8
CodeLLaMA-13B-Instruct	28.2	27.4	62.0
InternLM-7B-Chat-v1.1	28.5	4.8	40.5
InternLM-20B-Chat	34.6	21.4	45.6
Qwen-7B-Chat	41.9	40.5	54.4
Qwen-14B-Chat	58.4	53.6	59.5

In addition, we also provide experimental results demonstrating that our model is capable of acting as a HuggingFace Agent. For more information, please refer to the example documentation. The model's performance on the evaluation dataset provided by Hugging Face is as follows:

HuggingFace Agent Benchmark- Run Mode
Model	Tool Selection↑	Tool Used↑	Code↑
GPT-4	100	100	97.4
GPT-3.5	95.4	96.3	87.0
StarCoder-Base-15B	86.1	87.0	68.9
StarCoder-15B	87.0	88.0	68.9
Qwen-7B-Chat	87.0	87.0	71.5
Qwen-14B-Chat	93.5	94.4	87.0

HuggingFace Agent Benchmark - Chat Mode
Model	Tool Selection↑	Tool Used↑	Code↑
GPT-4	97.9	97.9	98.5
GPT-3.5	97.3	96.8	89.6
StarCoder-Base-15B	97.9	97.9	91.1
StarCoder-15B	97.9	97.9	89.6
Qwen-7B-Chat	94.7	94.7	85.1
Qwen-14B-Chat	97.9	97.9	95.5

Long-Context Understanding

To extend the context length and break the bottleneck of training sequence length, we introduce several techniques, including NTK-aware interpolation, window attention, and LogN attention scaling, to extend the context length of Qwen-7B/14B from 2k to over 8K tokens, and Qwen-7B from 8k to 32k tokens. We conduct language modeling experiments on the arXiv dataset with the PPL evaluation and find that Qwen can reach outstanding performance in the scenario of long context. Results are demonstrated below:

Model	Sequence Length
Model	1024	2048	4096	8192	16384	32768
Qwen-7B (original)	4.23	3.78	39.35	469.81	2645.09	-
+ dynamic_ntk	4.23	3.78	3.59	3.66	5.71	-
+ dynamic_ntk + logn	4.23	3.78	3.58	3.56	4.62	-
+ dynamic_ntk + logn + window_attn	4.23	3.78	3.58	3.49	4.32	-
Qwen-7B	4.23	3.81	3.52	3.31	7.27	181.49
+ dynamic_ntk + logn + window_attn	4.23	3.81	3.52	3.33	3.22	3.17
Qwen-14B	-	3.46	22.79	334.65	3168.35	-
+ dynamic_ntk + logn + window_attn	-	3.46	3.29	3.18	3.42	-

Tokenizer

Our tokenizer based on tiktoken is different from other tokenizers, e.g., sentencepiece tokenizer. You need to pay attention to special tokens, especially in finetuning. For more detailed information on the tokenizer and related use in fine-tuning, please refer to the documentation.

Reproduction

For your reproduction of the model performance on benchmark datasets, we provide scripts for you to reproduce the results. Check eval/EVALUATION.md for more information. Note that the reproduction may lead to slight differences from our reported results.

FAQ

If you meet problems, please refer to FAQ and the issues first to search a solution before you launch a new issue.

License Agreement

Researchers and developers are free to use the codes and model weights of both Qwen and Qwen-Chat. We also allow their commercial use. Check our license at LICENSE for more details. If you have requirements for commercial use, please fill out the form to apply.

Contact Us

If you are interested to leave a message to either our research team or product team, join our Discord or WeChat groups! Also, feel free to send an email to qianwen_opensource@alibabacloud.com.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README_src.md

README_src.md

News and Updates

Performance

Requirements

Quickstart

🤗 Transformers

🤖 ModelScope

Quantization

Usage

Performance

Inference Speed

GPU Memory Usage

Finetuning

Demo

Web UI

CLI Demo

API

Deployment

Tool Usage

Long-Context Understanding

Tokenizer

Reproduction

FAQ

License Agreement

Contact Us

Files

README_src.md

Latest commit

History

README_src.md

File metadata and controls

News and Updates

Performance

Requirements

Quickstart

🤗 Transformers

🤖 ModelScope

Quantization

Usage

Performance

Inference Speed

GPU Memory Usage

Finetuning

Demo

Web UI

CLI Demo

API

Deployment

Tool Usage

Long-Context Understanding

Tokenizer

Reproduction

FAQ

License Agreement

Contact Us