Clip Open Source Projects

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

Ecommerce Search and Discovery - marqo.ai

开放源码的无App推送服务，iOS14+扫码即用。亦支持快应用/iOS和Mac客户端、Android客户端、自制设备

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

OpenMMLab Pre-training Toolbox and Benchmark

中文nlp解决方案(大模型、数据、模型、训练、推理)

Collection of AWESOME vision-language models for vision tasks

Image to prompt with BLIP and CLIP

Easily compute clip embeddings and build a clip retrieval system with them

🥂 Gracefully face hCaptcha challenge with multimodal large language model.

Android UI 快速开发，专治原生控件各种不服

⚡️Lightning-fast async download tool for bilibili and more

Must-have resource for anyone who wants to experiment with and build on the OpenAI vision API 🔥

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and 🔜 video, up to 5x faster than OpenAI CLIP and LLaVA 🖼️ & 🖋️

Awesome list for research on CLIP (Contrastive Language-Image Pre-Training).

Stable Diffusion in NCNN with c++, supported txt2img and img2img

Search photos on Unsplash using natural language

An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"

Search inside YouTube videos using natural language

[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

Official Pytorch Implementation for "Text2LIVE: Text-Driven Layered Image and Video Editing" (ECCV 2022 Oral)

[CVPR'23] OpenScene: 3D Scene Understanding with Open Vocabularies

CLIP + FFT/DWT/RGB = text to image/video

Paddle Multimodal Integration and eXploration, supporting mainstream multi-modal tasks, including end-to-end large-scale multi-modal pretrain models and diffusion model toolbox. Equipped with high performance and flexibility.

Extract video features from raw videos using multiple GPUs. We support RAFT flow frames as well as S3D, I3D, R(2+1)D, VGGish, CLIP, and TIMM models.

基于Stable Diffusion优化的AI绘画模型。支持输入中英文文本，可生成多种现代艺术风格的高质量图像。| An optimized text-to-image model based on Stable Diffusion. Both Chinese and English text inputs are available to generate images. The model can generate high-quality images in several modern art styles.

👁️ + 💬 + 🎧 = 🤖 Curated list of top foundation and multimodal models! [Paper + Code + Examples + Tutorials]

Keras beit,caformer,CMT,CoAtNet,convnext,davit,dino,efficientdet,edgenext,efficientformer,efficientnet,eva,fasternet,fastervit,fastvit,flexivit,gcvit,ghostnet,gpvit,hornet,hiera,iformer,inceptionnext,lcnet,levit,maxvit,mobilevit,moganet,nat,nfnets,pvt,swin,tinynet,tinyvit,uniformer,volo,vanillanet,yolor,yolov7,yolov8,yolox,gpt2,llama2, alias kecam

React component for truncating multi-line spans and adding an ellipsis.

🧞 No-code tool for creating a neural search solution in minutes

Offline semantic Text-to-Image and Image-to-Image search on Android powered by quantized state-of-the-art vision-language pretrained CLIP model and ONNX Runtime inference engine

CLIP inference in plain C/C++ with no extra dependencies

CLIPort: What and Where Pathways for Robotic Manipulation

FashionCLIP is a CLIP-like model fine-tuned for the fashion domain.

🔍 Search local images with natural language on Android, powered by OpenAI's CLIP model. / 在 Android 上用自然语言搜索本地图片 (基于 OpenAI 的 CLIP 模型)

ZMJImageEditor is a picture editing component like WeChat. It is powerful and easy to integrate, supporting rendering, text, rotation, tailoring, mapping and other functions. (ZMJImageEditor 是一个和微信一样图片编辑的组件，功能强大，极易集成，支持绘制、文字、旋转、剪裁、贴图等功能)

[AAAI 2023] Exploring CLIP for Assessing the Look and Feel of Images

Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion

OCR自动化阅卷项目

[Pattern Recognition 25] CLIP Surgery for Better Explainability with Enhancement in Open-Vocabulary Tasks

Pathology Language and Image Pre-Training (PLIP) is the first vision and language foundation model for Pathology AI (Nature Medicine). PLIP is a large-scale pre-trained model that can be used to extract visual and language features from pathology images and text description. The model is a fine-tuned version of the original CLIP model.

Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model

Android Easy Reveal Library

Generating Robotic Simulation Tasks via Large Language Models

AutoCut Client

ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts

Official Pytorch implementation of "CLIPstyler:Image Style Transfer with a Single Text Condition" (CVPR 2022)

Quick scripts to calculate CLIP text-image similarity

Getting the latest versions of Disco Diffusion to work locally, instead of colab. Including how I run this on Windows, despite some Linux only dependencies ;)

Experiments and data for the paper "When and why vision-language models behave like bags-of-words, and what to do about it?" Oral @ ICLR 2023

PASSL包含 SimCLR，MoCo v1/v2，BYOL，CLIP，PixPro，simsiam, SwAV, BEiT，MAE 等图像自监督算法以及 Vision Transformer，DEiT，Swin Transformer，CvT，T2T-ViT，MLP-Mixer，XCiT，ConvNeXt，PVTv2 等基础视觉算法

CLIP⚡NCNN⚡基于自然语言的图片搜索(Image Search)⚡以字搜图⚡x86⚡Android

[CVPR 2023 Workshop] VAND Challenge: 1st Place on Zero-shot AD and 4th Place on Few-shot AD

This is the repository for the BioCLIP model and the TreeOfLife-10M dataset [CVPR'24 Oral, Best Student Paper].

Language Models Can See: Plugging Visual Controls in Text Generation

Multi-modal Molecule Structure-text Model for Text-based Editing and Retrieval, Nat Mach Intell 2023 (https://www.nature.com/articles/s42256-023-00759-6)

PyTorch code for "Fine-grained Image Captioning with CLIP Reward" (Findings of NAACL 2022)

[CVPR2023] Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective

It is a simple library to speed up CLIP inference up to 3x (K80 GPU)