Carview!

Deep Learning on ImageNet using Torch ===================================== This is a complete training example for Deep Convolutional Networks on the ILSVRC classification task. Data is preprocessed and cached as a LMDB data-base for fast reading. A separate thread buffers images from the LMDB record in the background. Multiple GPUs are also supported by using nn.DataParallelTable (https://github.com/torch/cunn/blob/master/docs/cunnmodules.md). This code allows training at 4ms/sample with the AlexNet model and 2ms for testing on a single GPU (using Titan Z with 1 active gpu) ## Dependencies * Torch (https://torch.ch) * "eladtools" (https://github.com/eladhoffer/eladtools) for optimizer. * "lmdb.torch" (https://github.com/eladhoffer/lmdb.torch) for LMDB usage. * "DataProvider.torch" (https://github.com/eladhoffer/DataProvider.torch) for DataProvider class. * "cudnn.torch" (https://github.com/soumith/cudnn.torch) for faster training. Can be avoided by changing "cudnn" to "nn" in models. To install all dependencies (assuming torch is installed) use: ```bash luarocks install https://raw.githubusercontent.com/eladhoffer/eladtools/master/eladtools-scm-1.rockspec luarocks install https://raw.githubusercontent.com/eladhoffer/lmdb.torch/master/lmdb.torch-scm-1.rockspec luarocks install https://raw.githubusercontent.com/eladhoffer/DataProvider.torch/master/dataprovider-scm-1.rockspec ``` ## Data * To get the ILSVRC data, you should register on their site for access: https://www.image-net.org/ * Extract all archives and configure the data location and save dir in **Config.lua**. You can also change the saved image size by editing the default value `ImageMinSide=256`. * LMDB records for fast read access are created by running **CreateLMDBs.lua**. It defaults to saving the compressed jpgs (about ~24GB for training data, ~1GB for validation data when smallest image dimension is 256). * To validate the LMDB configuration and test its loading speed, you can run **TestLMDBs.lua**. * All data related functions used for training are available at **Data.lua**. ## Model configuration Network model is defined by writing a .lua file in `Models` folder, and selecting it using the `network` flag. The model file must return a trainable network. It can also specify additional training options such optimization regime, input size modifications. e.g for a model file: ```lua local model = nn.Sequential():add(...) return --optional: you can also simply return model { model = model, regime = { epoch = {1, 19, 30, 44, 53 }, learningRate = {1e-2, 5e-3, 1e-3, 5e-4, 1e-4}, weightDecay = {5e-4, 5e-4, 0, 0, 0 } } } ``` Currently available in `Models` folder are: `AlexNet`, `MattNet`, `OverFeat`, `GoogLeNet`, `CaffeRef`, `NiN`. Some are available with a batch normalized version (denoted with `_BN`) ## Training You can start training using **Main.lua** by typing: ```lua th Main.lua -network AlexNet -LR 0.01 ``` or if you have 2 gpus availiable, ```lua th Main.lua -network AlexNet -LR 0.01 -nGPU 2 -batchSize 256 ``` A more elaborate example continuing a pretrained network and saving intermediate results ```lua th Main.lua -network GoogLeNet_BN -batchSize 64 -nGPU 2 -save GoogLeNet_BN -bufferSize 9600 -LR 0.01 -checkpoint 320000 -weightDecay 1e-4 -load ./pretrainedNet.t7 ``` Buffer size should be adjusted to suit the used hardware and configuration. Default value is 5120 (40 batches of 128) which works well when using a non SSD drive and 16GB ram. Bigger buffer size allows better sample shuffling. ## Output Training output will be saved to folder defined with `save` flag. The complete netowork will be saved on each epoch as **Net_<#epoch>.t7** along with * A complete log **Log.txt** * Error rate summary **ErrorRate.log** and accompanying **ErrorRate.log.eps** graph ## Additional flags |Flag | Default Value |Description |:----------------|:--------------------:|:---------------------------------------------- |modelsFolder |./Models/ | Models Folder |network |AlexNet | Model file - must return valid network. |LR |0.01 | learning rate |LRDecay |0 | learning rate decay (in # samples) |weightDecay |5e-4 | L2 penalty on the weights |momentum |0.9 | momentum |batchSize |128, | batch size |optimization |'sgd' | optimization method |seed |123 | torch manual random number generator seed |epoch |-1 | number of epochs to train, -1 for unbounded |testonly |false | Just test loaded net on validation set |threads |8 | number of threads |type |'cuda' | float or cuda |bufferSize |5120 | buffer size |devid |1 | device ID (if using CUDA) |nGPU |1 | num of gpu devices used |constBatchSize |false | do not allow varying batch sizes - e.g for ccn2 kernel |load |'' | load existing net weights |save |time-identifier | save directory |optState |false | Save optimization state every epoch |checkpoint |0 | Save a weight check point every n samples. 0 for off |augment |1 | data augmentation level - {1 - simple mirror and crops, 2 +scales, 3 +rotations} |estMeanStd |preDef | estimate mean and std. Options: {preDef, simple, channel, image} |shuffle |true | shuffle training samples

Name		Name	Last commit message	Last commit date
Latest commit History 43 Commits
Models		Models
Config.lua		Config.lua
CreateLMDBs.lua		CreateLMDBs.lua
Data.lua		Data.lua
ImageNetClasses		ImageNetClasses
LICENSE		LICENSE
Main.lua		Main.lua
README.md		README.md
TestLMDB.lua		TestLMDB.lua
ValidationLabels		ValidationLabels

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

About

Uh oh!

Releases

Packages

Contributors 2

Uh oh!

Languages

License

eladhoffer/ImageNet-Training

Folders and files

Latest commit

History

Repository files navigation

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Contributors 2

Uh oh!

Languages

Packages