Datasets Open Source Projects

A topic-centric list of HQ open datasets.

Apache Superset is a Data Visualization and Data Exploration Platform

Label Studio is a multi-type data labeling and annotation tool with standardized output format

Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

Datasets, Transforms and Models specific to Computer Vision

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

pix2code: Generating Code from a Graphical User Interface Screenshot

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

An open source multi-tool for exploring and publishing data

Open source annotation tool for machine learning practitioners.

Techniques for deep learning with satellite & aerial imagery

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

FL Chart is a highly customizable Flutter chart library that supports Line Chart, Bar Chart, Pie Chart, Scatter Chart, Radar Chart and Candlestick Chart.

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

A PyTorch implementation of NeRF (Neural Radiance Fields) that reproduces the results.

Real-time face detection and emotion/gender classification using fer2013/imdb datasets with a keras CNN model and openCV.

OpenPCDet Toolbox for LiDAR-based 3D Object Detection.

Torchreid: Deep learning person re-identification in PyTorch.

TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...

A unified, comprehensive and efficient recommendation library

搜索所有中文NLP数据集，附常用英文NLP数据集

CSGHub is an opensource large model assets platform just like on-premise huggingface which helps to manage datasets, model files, codes and more.　CSGHub是一个开源、可信的大模型资产管理平台，可帮助用户治理LLM和LLM应用生命周期中涉及到的资产（数据集、模型文件、代码等）。CSGHub提供类似私有化的Huggingface功能，以类似OpenStack Glance管理虚拟机镜像、Harbor管理容器镜像以及Sonatype Nexus管理制品的方式，实现对LLM资产的管理。欢迎关注反馈和Star⭐️

N-D labeled arrays and datasets in Python

TorchGeo: datasets, samplers, transforms, and pre-trained models for geospatial data

Transfer Learning Library for Domain Adaptation, Task Adaptation, and Domain Generalization

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

A curated list of awesome JSON datasets that don't require authentication.

Models, data loaders and abstractions for language processing, powered by PyTorch

TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

[TMLR] A curated list of language modeling researches for code (and other software engineering activities), plus related datasets.

Data competition Top Solution 数据竞赛top解决方案开源整理

Papers and Datasets about Point Cloud.

Medical NLP Competition, dataset, large models, paper 医疗NLP领域 比赛，数据集，大模型，论文，工具包

A large collection of system log datasets for AI-driven log analytics [ISSRE'23]

🎁 7,400,000+ Unsplash images made available for research and machine learning

Open Data, more than 50 financial data. 提供超過 50 個金融資料(台股為主)，每天更新 https://finmind.github.io/

A list of publicly available datasets with real-time data maintained by the team at bytewax.io

Colour Science for Python

A list of Medical imaging datasets.

数据接口：百度、谷歌、头条、微博指数,宏观数据，利率数据，货币汇率，千里马、独角兽公司，新闻联播文字稿，影视票房数据，高校名单，疫情数据…

🍕 Repositório para juntar informações sobre materiais de estudo em análise de dados e áreas afins, empresas que trabalham com dados e dicionário de conceitos

Datasets, tools, and benchmarks for representation learning of code.

An extension of Open3D to address 3D Machine Learning tasks

A list of awesome papers and resources of recommender system on large language model (LLM).

:pencil2: Web-based image segmentation tool for object detection, localization, and keypoints

Feature selector is a tool for dimensionality reduction of machine learning datasets

🔊 A comprehensive list of open-source datasets for voice and sound computing (95+ datasets).

Bioinformatics'2020: BioBERT: a pre-trained biomedical language representation model for biomedical text mining

Open Catalyst Project's library of machine learning methods for catalysis

C++ Implementation of PyTorch Tutorials for Everyone

Benchmark datasets, data loaders, and evaluators for graph machine learning

用于训练中英文对话系统的语料库 Datasets for Training Chatbot System

A curated list of amazingly awesome Cybersecurity datasets

A repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance in Text-to-SQL

DataStax Connector for Apache Spark to Apache Cassandra