Transformers Open Source Projects

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

21 Lessons, Get Started Building with Generative AI

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

CVPR 2023 论文和开源项目合集

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Machine Learning Engineering Open Book

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

Unified framework for building enterprise RAG pipelines with small, specialized models

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

A PyTorch-based Speech Toolkit

This repository contains demos I made with the Transformers library by HuggingFace.

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

A Ray-based High-performance RLHF framework (Support 70B+ full tuning & LoRA & Mixtral)

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

BertViz: Visualize Attention in Transformer Models

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Leveraging BERT and c-TF-IDF to create easily interpretable topics.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)

Open-source offline translation library written in Python

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

A concise but complete full-attention transformer with a set of promising experimental features from various papers

Robust recipes to align language models with human and AI preferences

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

OpenChat: Advancing Open-source Language Models with Imperfect Data

🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

Ecommerce Search and Discovery - marqo.ai

A machine learning software for extracting information from scholarly documents

Prompt Engineering, Generative AI, and LLM Guide by Learn Prompting | Join our discord for the largest Prompt Engineering learning community

LightGlue: Local Feature Matching at Light Speed (ICCV 2023)

Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research

Tutorials on getting started with PyTorch and TorchText for sentiment analysis.

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun

中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI

State of the Art Natural Language Processing

Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。

The Hugging Face course on Transformers

⚡️SwanLab - an open-source, modern-design AI training tracking and visualization tool. Supports Cloud / Self-hosted use. Integrated with PyTorch / Transformers / verl / LLaMA Factory / ms-swift / Ultralytics / MMEngine / Keras etc.

手把手带你实战 Huggingface Transformers 课程视频同步更新在B站与YouTube

中文nlp解决方案(大模型、数据、模型、训练、推理)

Scenic: A Jax Library for Computer Vision Research and Beyond

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Run GPT model on the browser with WebGPU. An implementation of GPT inference in less than ~1500 lines of vanilla Javascript.

Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调

🧠💬 Articles I wrote about machine learning, archived from MachineCurve.com.

Seamlessly integrate LLMs into scikit-learn.

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

🦖 𝗟𝗲𝗮𝗿𝗻 about 𝗟𝗟𝗠𝘀, 𝗟𝗟𝗠𝗢𝗽𝘀, and 𝘃𝗲𝗰𝘁𝗼𝗿 𝗗𝗕𝘀 for free by designing, training, and deploying a real-time financial advisor LLM system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 𝘷𝘪𝘥𝘦𝘰 & 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

ML-powered speech recognition directly in your browser