Evaluation Open Source Projects

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

Ongoing research training transformer models at scale

A framework for few-shot evaluation of language models.

:metal: awesome-semantic-segmentation

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

A treasure chest for visual classification and recognition powered by PaddlePaddle

Rhai - An embedded scripting language for Rust.

A utility library for JavaScript and TypeScript.

[ICCV 2019] Monocular depth estimation from a single image

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

Python package for the evaluation of odometry and SLAM

Learning to Learn in TensorFlow

Arbitrary expression evaluation for golang

Text recognition (optical character recognition) with deep learning methods, ICCV 2019

Building a modern functional compiler from first principles. (http://dev.stephendiehl.com/fun/)

SuperCLUE: 中文通用大模型综合性基准 | A Benchmark for Foundation Models in Chinese

Sunfish: a Python Chess Engine in 111 lines of code

Klipse is a JavaScript plugin for embedding interactive code snippets in tech blogs.

A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

An open-source visual programming environment for battle-testing prompts to LLMs.

Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow

A unified evaluation framework for large language models

CodeBERT

A fast, persistent use_nix/use_flake implementation for direnv [maintainer=@Mic92 / @bbenne10]

MONAI Tutorials

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Dual Attention Network for Scene Segmentation (CVPR2019)

Joint Detection and Embedding for fast multi-object tracking

Infinitely Fast Deserializer for .NET, .NET Core and Unity.

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

Dual-mode Bluetooth stack, with small memory footprint.

Author's PyTorch implementation of TD3 for OpenAI gym tasks

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

An unsupervised learning framework for depth and ego-motion estimation from monocular videos

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

Nix User Repository: User contributed nix packages [maintainer=@Pandapip1]

Atlas: End-to-End 3D Scene Reconstruction from Posed Images

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images. In ECCV2018.

This repository contains the code for the paper "Occupancy Networks - Learning 3D Reconstruction in Function Space"

Pyramid Scene Parsing Network, CVPR2017.

Machine learning evaluation metrics, implemented in Python, R, Haskell, and MATLAB / Octave

This repository contains the code for "Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference"

A PyTorch library and evaluation platform for end-to-end compression research

Training and Detecting Objects with YOLO3

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

Dense image captioning in Torch

Dual LSTM Encoder for Dialog Response Generation

Multi-class confusion matrix library in Python

Evaluation code for various unsupervised automated metrics for Natural Language Generation.

Solves basic Russian NLP tasks, API for lower level Natasha projects

:couple: Joint Discriminative and Generative Learning for Person Re-identification. CVPR'19 (Oral) :couple:

Short and sweet LISP editing

"DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion" code repository

XAI - An eXplainability toolbox for machine learning