Attention Open Source Projects

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Machine learning, in numpy

Natural Language Processing Tutorial for Deep Learning Researchers

🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐

End-to-End Speech Processing Toolkit

A PyTorch implementation of the Transformer model in "Attention is All You Need".

BertViz: Visualize Attention in Transformer Models

all kinds of text classification models and more with deep learning

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

The GitHub repository for the paper "Informer" accepted by AAAI 2021.

TensorFlow Neural Machine Translation Tutorial

A concise but complete full-attention transformer with a set of promising experimental features from various papers

中文文本分类，TextCNN，TextRNN，FastText，TextRCNN，BiLSTM_Attention，DPCNN，Transformer，基于pytorch，开箱即用。

Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.

Transformer: PyTorch Implementation of "Attention Is All You Need"

A TensorFlow Implementation of the Transformer: Attention Is All You Need

该仓库主要记录 NLP 算法工程师相关的顶会论文研读笔记

Scenic: A Jax Library for Computer Vision Research and Beyond

Graph Attention Networks (https://arxiv.org/abs/1710.10903)

Draw a leader line in your web page.

Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)

Pytorch implementation of U-Net, R2U-Net, Attention U-Net, and Attention R2U-Net.

Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning

My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!

Dual Attention Network for Scene Segmentation (CVPR2019)

Graph Neural Networks with Keras and Tensorflow 2.

Implementation of different kinds of Unet Models for Image Segmentation - Unet , RCNN-Unet, Attention Unet, RCNN-Attention Unet, Nested Unet

The implementation of DeBERTa

Official PyTorch code for "BAM: Bottleneck Attention Module (BMVC2018)" and "CBAM: Convolutional Block Attention Module (ECCV2018)"

Reformer, the efficient Transformer, in Pytorch

Pytorch reimplementation of the Vision Transformer (An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale)

Aspect Based Sentiment Analysis, PyTorch Implementations. 基于方面的情感分析，使用PyTorch实现。

Use of Attention Gates in a Convolutional Neural Network / Medical Image Classification and Segmentation

[CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize classifications by Transformer based networks.

PyTorch implementation of convolutional neural networks-based text-to-speech synthesis models

Hopfield Networks is All You Need

Pytorch implementation of Self-Attention Generative Adversarial Networks (SAGAN)

Gathers machine learning and Tensorflow deep learning models for NLP problems, 1.13 < Tensorflow < 2.0

Pytorch library for fast transformer implementations

🚀🚀🚀 A collection of some awesome public YOLO object detection series projects.

This is a pytorch repository of YOLOv4, attentive YOLOv4 and mobilenet YOLOv4 with PASCAL VOC and COCO

Documents, papers and codes related to Natural Language Processing, including Topic Model, Word Embedding, Named Entity Recognition, Text Classificatin, Text Generation, Text Similarity, Machine Translation)，etc.

Examples of using sparse attention, as in "Generating Long Sequences with Sparse Transformers"

Bi-directional Attention Flow (BiDAF) network is a multi-stage hierarchical process that represents context at different levels of granularity and uses a bi-directional attention flow mechanism to achieve a query-aware context representation without early summarization.

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Non-official implement of Paper：CBAM: Convolutional Block Attention Module

CCNet: Criss-Cross Attention for Semantic Segmentation (TPAMI 2020 & ICCV 2019).

Bottom-up attention model for image captioning and VQA, based on Faster R-CNN and Visual Genome

Improving Convolutional Networks via Attention Transfer (ICLR 2017)

Code for ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

Attention based model for learning to solve different routing problems

make your activity can swipe to close

Generative Adversarial Transformers

自然语言处理领域下的相关论文（附阅读笔记），复现模型以及数据处理等（代码含TensorFlow和PyTorch两版本）

Sequence-to-sequence model with LSTM encoder/decoders and attention

Replication of simple CV Projects including attention, classification, detection, keypoint detection, etc.

Deep Learning Architecture Genealogy Project

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.