Adversarial Attacks Open Source Projects

Adversary Emulation Framework

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Data augmentation for NLP

TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/

A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX

A unified evaluation framework for large language models

PyTorch implementation of adversarial attacks [torchattacks]

A reading list for large models safety, security, and privacy.

Must-read Papers on Textual Adversarial Attack and Defense

Advbox is a toolbox to generate adversarial examples that fool neural networks in PaddlePaddle、PyTorch、Caffe2、MxNet、Keras、TensorFlow and Advbox can benchmark the robustness of machine learning models. Advbox give a command line tool to generate adversarial examples with Zero-Coding.

A Toolbox for Adversarial Robustness Research

A pytorch adversarial library for attack and defense methods on images and graphs

A collection of anomaly detection methods (iid/point-based, graph and time series) including active learning for anomaly detection/discovery, bayesian rule-mining, description for diversity/explanation/interpretability. Analysis of incorporating label feedback with ensemble and tree-based detectors. Includes adversarial attacks with Graph Convolutional Network.

A curated list of adversarial attacks and defenses papers on graph-structured data.

An Open-Source Package for Textual Adversarial Attack.

Code relative to "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks"

Raising the Cost of Malicious AI-Powered Image Editing

A Harder ImageNet Test Set (CVPR 2021)

A Model for Natural Language Attack on Text Classification and Inference

A Python library for adversarial machine learning focusing on benchmarking adversarial robustness.

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

Security and Privacy Risk Simulator for Machine Learning (arXiv:2312.17667)

Implementation of Papers on Adversarial Examples

Adversarial attacks and defenses on Graph Neural Networks.

🔥🔥Defending Against Deepfakes Using Adversarial Attacks on Conditional Image Translation Networks

💡 Adversarial attacks on explanations and how to defend them

Implementation of the KDD 2020 paper "Graph Structure Learning for Robust Graph Neural Networks"

TrojanZoo provides a universal pytorch platform to conduct security researches (especially backdoor attacks/defenses) of image classification in deep learning.

Simple pytorch implementation of FGSM and I-FGSM

Anti-DreamBooth: Protecting users from personalized text-to-image synthesis (ICCV 2023)

Self-hardening firewall for large language models

Awesome Resources for Advanced Computer Vision Topics

Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models (published in ICLR2018)

A suite for hunting suspicious targets, expose domains and phishing discovery

DEEPSEC: A Uniform Platform for Security Analysis of Deep Learning Model

Implementation of the paper "Adversarial Attacks on Neural Networks for Graph Data".

A curated list of papers on adversarial machine learning (adversarial examples and defense methods).

AIShield Watchtower: Dive Deep into AI's Secrets! 🔍 Open-source tool by AIShield for AI model insights & vulnerability scans. Secure your AI supply chain today! ⚙️🛡️

Code for our nips19 paper: You Only Propagate Once: Accelerating Adversarial Training Via Maximal Principle

Official TensorFlow Implementation of Adversarial Training for Free! which trains robust models at no extra cost compared to natural training.

Physical adversarial attack for fooling the Faster R-CNN object detector

Square Attack: a query-efficient black-box adversarial attack via random search [ECCV 2020]

Library containing PyTorch implementations of various adversarial attacks and resources

Python toolbox to evaluate graph vulnerability and robustness (CIKM 2021)

Attack to induce LLMs within hallucinations

A pytorch implementation of "Towards Deep Learning Models Resistant to Adversarial Attacks"

Implementation of the paper "Adversarial Attacks on Graph Neural Networks via Meta Learning".

A toolkit for detecting and protecting against vulnerabilities in Large Language Models (LLMs).

[NeurIPS 2020, Spotlight] Code for "Robust Deep Reinforcement Learning against Adversarial Perturbations on Observations"

Generative Adversarial Perturbations (CVPR 2018)

Code for the CVPR 2019 article "Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defenses"

A curated collection of adversarial attack and defense on recommender systems.

Real-time object detection is one of the key applications of deep neural networks (DNNs) for real-world mission-critical systems. While DNN-powered object detection systems celebrate many life-enriching opportunities, they also open doors for misuse and abuse. This project presents a suite of adversarial objectness gradient attacks, coined as TOG, which can cause the state-of-the-art deep object detection networks to suffer from untargeted random attacks or even targeted attacks with three types of specificity: (1) object-vanishing, (2) object-fabrication, and (3) object-mislabeling. Apart from tailoring an adversarial perturbation for each input image, we further demonstrate TOG as a universal attack, which trains a single adversarial perturbation that can be generalized to effectively craft an unseen input with a negligible attack time cost. Also, we apply TOG as an adversarial patch attack, a form of physical attacks, showing its ability to optimize a visually confined patch filled with malicious patterns, deceiving well-trained object detectors to misbehave purposefully.

A list of awesome resources for adversarial attack and defense method in deep learning

This repository contains PyTorch implementation for the baseline models from the paper Utterance-level Dialogue Understanding: An Empirical Study

The official implementation of the CCS'23 paper, Narcissus clean-label backdoor attack -- only takes THREE images to poison a face recognition dataset in a clean-label way and achieves a 99.89% attack success rate.

[CVPR 2020] When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks

A Paperlist of Adversarial Attack on Object Detection

[CVPR 2025] Official implementation of three papers "Certified Human Trajectory Prediction", "Vehicle trajectory prediction works, but not everywhere", and "Are socially-aware trajectory prediction models really socially-aware?".

Implementation of CVPR 2022 paper "On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles" https://arxiv.org/abs/2201.05057