pipeline:window:sp_cryolo_train

This version (2019/07/15 17:04) was approved by twagner.

crYOLO - training: Training of crYOLO, a deep learning high accuracy particle picking procedure.


Usage in command line

sp_cryolo_train.py particle_diameter training_dir annot_dir --cryolo_train_path=CRYOLO_PATH --architecture=architecture --input_size=input_size --num_patches=num_patches --overlap_patches=overlap_patches --train_times=train_times --pretrained_weights_name=PRETRAINED_NAME --saved_weights_name=SAVE_WEIGHTS_NAME --batch_size=batch_size --learning_rate=learning_rate --np_epoch=np_epoch --object_scale=object_scale --no_object_scale=no_object_scale --coord_scale=coord_scale --valid_image_dir=valid_image_dir --valid_annot_dir=valid_annot_dir --warmup=warmup --gpu=gpu --fine_tune --gpu_fraction=GPU_FRACTION --num_cpu=NUM_CPU


To train crYOLO for a specific dataset, one have to specify the path to training data in the config file. Then the training typcial happens in two steps:


1. Warmup:

sp_cryolo_train.py particle_diameter training_dir annot_dir --architecture="YOLO" --warmup=5


2. Actual training:

sp_cryolo_train.py --conf=config_path --warmup=0 --gpu=0


Main Parameters

--cryolo_train_path
crYOLO train executeable
Path to the crYOLO executeable (default none)
particle_diameter
Particle diameter [Pixel]
Particle diameter in pixel. This size will be used for as box size for picking. Should be as small as possible. (default required int)
training_dir
Training image directory
Folder which contain all images. (default required string)
annot_dir
Annotation directory
Box or star files used for training. The should have the same name as the images. (default required string)


Advanced Parameters

--architecture
Network architecture: Type of network that is trained. (default PhosaurusNet)
--input_size
Input image dimension [Pixel]
Dimension of the image used as input to network. (default 1024)
--num_patches
Number of patches
The number of patches (e.g 2×2) the image is divided and classified separately. (default 1)
--overlap_patches: Patch overlap [Pixel]: The amount of overlap the patches will overlap (default 0)
--train_times
Repeat images
How often a images is augmented and repeadet in one epoch. (default 10)
--pretrained_weights_name: Pretrained weights name
Name of the pretrained model (default cryolo_model.h5)
--saved_weights_name: Saved weights name
Name of the model to save (default cryolo_model.h5)
--batch_size
Batch size
How many patches are processed in parallel. (default 5)
--fine_tune
Fine tune mode
Set it to true if you only want to use the fine tune mode. (default False)
--learning_rate
Learning rate
Learning rate used during training. (default 0.0001)
--np_epoch
Number of epochs
Maximum number of epochs. (default 100)
--object_scale
Object loss scale
Loss scale for object. (default 5.0)
--no_object_scale: Background loss scale: Loss scale for background. (default 1.0)
--coord_scale: Coordinates loss scale: Loss scale for coordinates. (default 1.0)
--valid_image_dir
Path to validation images
Images used (default none)
--valid_annot_dir
Path to validation annotations
Path to the validation box files (default none)
--warmup
Warm up epochs
Number of warmup epochs. (default 5)
--gpu: GPUs
List of GPUs to use. (default 0)
--gpu_fraction: GPU memory fraction
Specify the fraction of memory per GPU used by crYOLO during training. Only values between 0.0 and 1.0 are allowed. (default 1.0)
--num_cpu: Number of CPUs
Number of CPUs used during training. By default it will use half of the available CPUs. (default -1)


It will write a .h5 file (default yolo_model.h5) into your project directory.


The training is divided into two parts. 1. Warmup: It prepares the network with a few epochs of training without actually estimating the size of the particle. 2. Actual training: The training will stop when the loss on the validation data stops to improve.


See the reference below.


Training needs a GPU with ~8GB memory. Training on 20 micrographs typicall needs ~20 minutes.


2019/09/19 Thorsten Wagner

  • Initial creation of the document


Thorsten Wagner


Category 1:: APPLICATIONS


sparx/bin/sp_cryolo_train.py


Stable


None right now.


  • pipeline/window/sp_cryolo_train.txt
  • Last modified: 2019/04/02 11:40
  • by lusnig