Rlhf Open Source Projects

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

The official GitHub page for the survey paper "A Survey of Large Language Models".

A Ray-based High-performance RLHF framework (Support 70B+ full tuning & LoRA & Mixtral)

Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Robust recipes to align language models with human and AI preferences

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets

A curated list of reinforcement learning with human feedback resources (continually updated)

Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调

A Doctor for your data

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

[NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)

Xtreme1 is an all-in-one data labeling and annotation platform for multimodal data training and supports 3D LiDAR point cloud, image, and LLM.

A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).

Aligning Large Language Models with Human: A Survey

聚宝盆(Cornucopia): 中文金融系列开源可商用大模型，并提供一套高效轻量化的垂直领域LLM训练框架(Pretraining、SFT、RLHF、Quantize等)

Implementation of ChatGPT RLHF (Reinforcement Learning with Human Feedback) on any generation model in huggingface's transformer (blommz-176B/bloom/gpt/bart/T5/MetaICL)

MindSpore online courses: Step into LLM

LLM Tuning with PEFT (SFT+RM+PPO+DPO with LoRA)

pykoi: Active learning in one unified interface

A curated list of Human Preference Datasets for LLM fine-tuning, RLHF, and eval.

🛰️ 基于真实医疗对话数据在ChatGLM上进行LoRA、P-Tuning V2、Freeze、RLHF等微调，我们的眼光不止于医疗问答

Explore implementations of deep learning concepts like Transformers, Attention, Llama, GPT, InstructGPT, RLHF, Gaussian Processes, Bayesian Inference, Newton Raphson, Distributed Trainers and more!

The open source implementation of ChatGPT, Alpaca, Vicuna and RLHF Pipeline. 从0开始实现一个ChatGPT.

LLaMA-TRL: Fine-tuning LLaMA with PPO and LoRA

Chain-of-Hindsight, a simpler and more effective alternative to RLHF

A full pipeline to finetune Vicuna LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Vicuna architecture. Basically ChatGPT but with Vicuna

A continually updated list of literature on Reinforcement Learning from AI Feedback (RLAIF)

Code for Paper (ReMax: A Simple, Efficient and Effective Reinforcement Learning Method for Aligning Large Language Models)

对ChatGLM直接使用RLHF提升或降低目标输出概率|Modify ChatGLM output with only RLHF

Code accompanying the paper Pretraining Language Models with Human Preferences

BeaverTails is a collection of datasets designed to facilitate research on safety alignment in large language models (LLMs).

Implementation of Reinforcement Learning from Human Feedback (RLHF)

Preference Transformer: Modeling Human Preferences using Transformers for RL (ICLR2023 Accepted)

A full pipeline to finetune ChatGLM LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the ChatGLM architecture. Basically ChatGPT but with ChatGLM

Tracking instruction-tuned LLM openness. Paper: Liesenfeld, Andreas, Alianda Lopez, and Mark Dingemanse. 2023. “Opening up ChatGPT: Tracking Openness, Transparency, and Accountability in Instruction-Tuned Text Generators.” In Proceedings of the 5th International Conference on Conversational User Interfaces. doi:10.1145/3571884.3604316.

Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat