民间版中文羊驼模型 v5.0

[ymcui]

本版本针对模型回复较短的问题进行升级，同时推出Plus-33B系列模型。

同时，我们也很高兴地向大家宣布新项目启动：中文LLaMA-2、Alpaca-2大模型🦙

🚀 推出中文Alpaca-Pro系列模型

• 针对早期Alpaca相关模型回复较短的问题进行优化，使其回复长度接近GPT-3.5/GPT-4
• 推荐所有使用Alpaca系列模型的用户升级至Pro版本（7B/13B/33B），合并方式与Plus模型一致，需要同时融合LLaMA-Plus的LoRA
• 如果更喜欢短回复，可以选择Plus系列模型

在此特别感谢所有参与beta评测投票的各位社区成员。

🚀 推出LLaMA-Plus-33B、Alpaca-Plus-33B

• 在基础版33B之上将训练数据扩充至120G，得到LLaMA-Plus-33B；进一步通过指令精调得到Alpaca-Plus-33B
• 在C-Eval、主观体验方面显著优于本项目其他模型
• 本次不再提供GPT-4评价的结果，请通过接下来介绍的ChatBot Arena浏览对比相关模型输出

📊 添加ChatBot Arena

• 仿照fastchat的方式，添加了模型对战页面，题库进一步扩充至360题
• 评测模型包括所有Alpaca-Plus和Alpaca-Pro系列模型，每个模型包含3个参考输出（随机显示其一）
• 可实时查看总体胜率、Elo评分、两两对战胜率等信息

对战地址：http://chinese-alpaca-arena.ymcui.com/

🔬 升级NTK-scaling算法，添加Memory Efficient Attention（#743）

• 升级了NTK-scaling算法，更好地支持长上下文条件下的文本生成
• 可通过参数alpha控制上下文扩展长度，支持指定数值和auto两种方式
• 添加了Memory Efficient Attention补丁，显著降低显存消耗，在单张24G显存的显卡上7B模型最大输入长度可达5K+

具体使用方法见 #743 中的说明。

其他更新与修复

• 推理脚本inference_hf.py中添加了8比特量化推理支持（Adding load_in_8bit to inference_hf.py #756）
• 修复了推理脚本repeat_penalty默认值过高的问题（Update inference_hf.py for repetition_penalty #760）

For English release note, please refer to Discussion.

---

This discussion was created from the release 民间版中文羊驼模型 v5.0.

[ymcui]

Release Note v5.0

This release improves on the issue of the model returning relatively short replies, and also launches the Plus-33B series model.

In addition, we are pleased to announce the launch of our new projects: the large-scale Chinese LLaMA-2, Alpaca-2 models

🚀 Launching the Chinese Alpaca-Pro series model

• We have optimized the issues of the early Alpaca models providing relatively short responses, bringing the response length closer to that of GPT-3.5/GPT-4.
• We recommend all users of the Alpaca series models to upgrade to the Pro version (7B/13B/33B). The merging method is consistent with the Plus models, and it needs to integrate LLaMA-Plus's LoRA.
• If you prefer shorter responses, you can choose the Plus series model.
  We want to take this opportunity to thank all community members who participated in the beta testing and voting.

🚀 Launching LLaMA-Plus-33B, Alpaca-Plus-33B

• On top of the basic 33B model, the training data has been expanded to 120G, resulting in the LLaMA-Plus-33B. Through further command fine-tuning, we derived the Alpaca-Plus-33B.
• In terms of C-Eval and subjective experience, it significantly outperforms other models in this project.
• We will no longer provide GPT-4 evaluation results in this release. Please view the output of the relevant models through the upcoming ChatBot Arena.

📊 Added ChatBot Arena

• Inspired by Fastchat, we've added a model battle page, further expanding the question bank to 360 questions.
• The models evaluated include all Alpaca-Plus and Alpaca-Pro series models, each model includes three reference outputs.
• You can view the overall win rate, Elo rating, and pairwise win rate in real-time.
• Battle address: http://chinese-alpaca-arena.ymcui.com/

🔬 Upgraded NTK-scaling algorithm, added Memory Efficient Attention (#743)

• The NTK-scaling algorithm has been upgraded to better support text generation under long context conditions.
• The context extension length can be controlled by the alpha parameter, supporting both specified values and auto modes.
• We've added a Memory Efficient Attention patch, significantly reducing memory consumption. On a 24G memory graphics card, the maximum input length of the 7B model can reach 5K+.

For detailed usage, please see the instructions in #743.

Other updates and fixes

• Added 8-bit quantization inference support in the inference_hf.py script (Adding load_in_8bit to inference_hf.py #756)
• Fixed the problem of the default value of repeat_penalty in the inference script being too high (Update inference_hf.py for repetition_penalty #760).

[Blurjp]

The link to the Battle address: http://chinese-alpaca-arena.ymcui.com/ is broken.

[airaria]

How is it now? I can access the link.

[Blurjp]

hmm, It only works when I access it via VPN to China.

[cheez95]

IP address from Taiwan could also not access the link.

[LiuChen19960902]

想请教一下回复长度的问题是怎么进行优化的呢？