This repository contains code to reproduce results from the paper:

TextHacker: Learning based Hybrid Local Search Algorithm for Text Hard-label Adversarial Attack

• torch == 1.7.1
• tensorflow-gpu == 1.15.0
• tensorflow-hub == 0.10.0
• numpy == 1.19.5
• nltk == 3.3
• language-tool-python == 2.5.3
• Pattern == 3.6

There are eight datasets used in our experiments including AG's News, IMDB, MR, Yelp, Yahoo! Answers, SNLI, MNLI and MNLIm. The sampled texts for evaluation are adopted from the github repo of HLBB. You could download and place the dataset into the directory ./data/dataset.

We adopt the pretrained models provided by HLBB, including BERT, WordCNN, WordLSTM. You could put these pretrained models BERT, WordCNN and WordLSTM into the directory ./data/model/bert, ./data/model/WordCNN, ./data/model/WordLSTM, respectively.

There are three dependencies for this project. Download and put glove.6B.200d.txt to the directory /data/embedding. And put counter-fitted-vectors.txt and the top synonym file mat.txt to the directory ./data/aux_files.

• GloVe vecors
• Counter fitted vectors
• Mat

• attack.py: Attack the target model for text classification with TextHacker.
• attack_nli.py: Attack the target model for textual entailment with TextHacker.
• config.py: Parameters of attack for all datasets.
• ./adv_method: Implementation for our TextHacker.
• ./data: Dataset, embedding matrix and various aux files.
• ./model_loader: Target model, including BERT, WordCNN and WordLSTM.
• ./utils: Helper functions for building dictionaries, loading data, and processing embedding matrix etc.
• ./parameter: All hyper-parameters of our TextHacker for various target models and datasets in our main experiments.
• ./scripts: Commands to run the attack.

Taking the TextHacker attack on BERT using MR dataset for example, you could run the following command:

sh scripts/bert_mr.sh

You could change the hyper-parameters of TextHacker in the ./parameter/bert_mr.yaml if necessary.

If you find this code and data useful, please consider citing the original work by authors: