Carview!

NIWT: Neuron-Importance aware Weight Transfer

Code for the ECCV'18 paper

[Choose-Your-Neuron: Incorporating Domain Knowledge into Deep Networks through Neuron-Importance]
Ramprasaath R. Selvaraju, Prithvijit Chattopadhyay, Mohammed Elhoseiny, Tilak Sharma, Dhruv Batra, Devi Parikh, Stefan Lee

Usage

This codebase assumes that you have installed Tensorflow. If not, please follow installation instructions from here.
Download data and pretrained checkpoints using sh download.sh and make sure the paths in the arg_config json files are correct. You may also need to create an imagenet_files.pkl which contains a list of (atleast) 3000 randomly sampled imagenet image paths.

Train a Generalized Zero Shot Learning model on AWA2 and CUB (class-level attributes)

python alpha2w.py --config_json arg_configs/vgg16_config_AWA.json
python alpha2w.py --config_json arg_configs/resnet_config_AWA.json
python alpha2w.py --config_json arg_configs/vgg16_config_CUB.json
python alpha2w.py --config_json arg_configs/resnet_config_CUB.json

Train a Generalized Zero Shot Learning model on CUB with captions (class-level)

python alpha2w.py --config_json arg_configs/vgg16_config_CUB_captions.json
python alpha2w.py --config_json arg_configs/resnet_config_CUB_captions.json

Train a GZSL classifier from scratch

Pretrain base model on dataset

To do this, we first finetune the base model (vgg16 or resnet_v1) on a seen class images.

cd seen_pretraining/
sh cnn_finetune.sh

Extract Neuron Importances (alphas)

Change the ckpt_path from the config_json files to the trained checkpoint (obtained from above) Extract Neuron-Importances (alphas) from the finetuned model.

sh alpha_extraction.sh

Domain knowledge to Neuron Importance:

Here we learn a transformation from domain knowledge (say attributes) to network neuron importances (alphas)

cd ..
python mod2alpha.py --config_json arg_configs/vgg16_config_AWA.json
python mod2alpha.py --config_json arg_configs/resnet_config_AWA.json
python mod2alpha.py --config_json arg_configs/vgg16_config_CUB.json
python mod2alpha.py --config_json arg_configs/resnet_config_CUB.json

Neuron Importance of unseen classes to classifier weights of unseen classes (training a GZSL model)

python alpha2w.py --config_json arg_configs/vgg16_config_AWA.json
python alpha2w.py --config_json arg_configs/resnet_config_AWA.json
python alpha2w.py --config_json arg_configs/vgg16_config_CUB.json
python alpha2w.py --config_json arg_configs/resnet_config_CUB.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

NIWT: Neuron-Importance aware Weight Transfer

Usage

Train a Generalized Zero Shot Learning model on AWA2 and CUB (class-level attributes)

Train a Generalized Zero Shot Learning model on CUB with captions (class-level)

Train a GZSL classifier from scratch

Pretrain base model on dataset

Extract Neuron Importances (alphas)

Domain knowledge to Neuron Importance:

Neuron Importance of unseen classes to classifier weights of unseen classes (training a GZSL model)

About

Uh oh!

Releases

Packages

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
arg_configs		arg_configs
seen_pretraining		seen_pretraining
README.md		README.md
alpha2w.py		alpha2w.py
download.sh		download.sh
mod2alpha.py		mod2alpha.py

ramprs/neuron-importance-zsl

Folders and files

Latest commit

History

Repository files navigation

NIWT: Neuron-Importance aware Weight Transfer

Usage

Train a Generalized Zero Shot Learning model on AWA2 and CUB (class-level attributes)

Train a Generalized Zero Shot Learning model on CUB with captions (class-level)

Train a GZSL classifier from scratch

Pretrain base model on dataset

Extract Neuron Importances (alphas)

Domain knowledge to Neuron Importance:

Neuron Importance of unseen classes to classifier weights of unseen classes (training a GZSL model)

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages