CrYOLO is a fast and accurate particle picking procedure. It's based on convolutional neural networks and utilizes the popular You Only Look Once (YOLO) object detection system.

• crYOLO makes picking fast – On a modern GPU it will pick your particles at up to 6 micrographs per second.
• crYOLO makes picking smart – The network learns the context of particles (e.g. not to pick particles on carbon or within ice contamination )
• crYOLO makes training easy – You might use a general network model and skip training completely. However, if the general model doesn't give you satisfactory results or if you would like to improve them, you might want to train a specialized model specific for your data set by selecting particles (no selection of negative examples necessary) on a small number of micrographs.
• crYOLO makes training tolerant – Don't worry if you miss quite a lot particles during creation of your training set. crYOLO will still do the job.

In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information how to use crYOLO, about supported networks and about the config file in the following articles:

• crYOLO talk at SBGrid
• crYOLO networks
• crYOLO configuration file

You can find the download and installation instructions here: Download and Installation

Depending what you want to do, you can follow one of these self-contained Tutorials:

1. I would like to train a model from scratch for picking my particles
2. I would like to train a model from scratch for picking filaments.
3. I would like to refine a general model for my particles.

The first and the second tutorial are the most common use cases and well tested. The third tutorial is still experimental but might give you better results in less time and less training data.

Now you are ready to train the model. In case you have multiple GPUs, you should first select a free GPU. The following command will show the status of all GPUs:

nvidia-smi

For this tutorial, we assume that you have either a single GPU or want to use GPU 0. In the GUI you have to fille the mandatory fields:

The default number of warmup epochs is fine as long you don't want to refine an existing model. During the warmup training epochs it will not try to estimate the size of your particle, which helps crYOLO to converge.

Before you start the training, you might want to change the GPU. By default, crYOLO will use GPU 0. The following command will show the status of all GPUs:

nvidia-smi

In “Optional arguments” tab you can change the GPU that should be used by crYOLO. If you have multiple (e.g GPU 0 and GPU 1) you could also use both by set the GPU argument to '0 1'.

If you start the training, it will stop when the “loss” metric on the validation data does not improve 10 times in a row. This is typically enough. In case want to give the training more time to find the best model you might want to increase the “not changed in a row” parameter to, for example, 15 by setting the early argument in the “Optional arguments” to 15.

Now press the Start button to start the training. The final model will be written to disk as specified in saved_weights_name in your configuration file.

cryolo_train.py -c config.json -w 5 -g 0

cryolo_train.py -c config.json -w 3 -g 0 -e 15

Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: How to train your own general model

Our general models can be found and downloaded here: Download and Installation.

The next step is to create a configuration file. Type:

touch config.json

Open the file with your preferred editor.

There are two general Phosaurus networks available. One for cryo em images and one for negative stain data.

For the general Phosaurus network trained for low-pass filtered cryo images enter the following inside:

For the general model trained with neural-network denoised cryo images (with JANNI's general model) enter the following inside:

In all cases please set the value in the “anchors” field to your desired box size. It should be size of the minimum particle enclosing square in pixel.

For the general model for negative stain data please use:

Please set the value in the “anchors” field to your desired box size. It should be size of the minimum particle enclosing square in pixel.

Since crYOLO 1.3 you can train a model for your data by fine-tuning the general model.

What does fine-tuning mean?

The general model was trained on a lot of particles with a variety of shapes and therefore learned a very good set of generic features. The last layers, however, learn a pretty abstract representation of the particles and it might be that they do not perfectly fit for your particle at hand. Fine-tuning only traines the last two convolutional layers, but keep the others fixed. This adjusts the more abstract representation for your specific problem.

Why should I fine-tune my model instead of training from scratch?

1. From theory, using fine-tuning should reduce the risk of overfitting ⁴⁾ and the amount of training data.
2. The training is much faster, as not all layers have to be trained.
3. The training will need less GPU memory ⁵⁾ and therefore is usable with NVIDIA cards with less memory.

However, the fine tune mode is still somewhat experimental and we will update this section if see more advantages or disadvantages.

You can use almost the same configuration as used when training from scratch. You just have to tell crYOLO to use the latest general model⁷⁾ by pointing to it with the “pretrained_weights” options:

"train": {
    [...]
    "pretrained_weights":              "LATEST_GENERAL_MODEL.h5",
    [...]
    "saved_weights_name":   "my_refined_model.h5",
    [...]
}

In comparison to the training from scratch, you can skip the warm up training ( -w 0 ). Moreover you have to add the --fine_tune flag to tell crYOLO that it should do fine tuning. You can also tell crYOLO how many layers it should fine tune (default is two layers with -lft 2 ):

cryolo_train.py -c config.json -w 0 -g 0 --fine_tune -lft 2

Since version 1.1.0 crYOLO supports picking filaments.

Filament mode on Actin:

Filament mode on MAVS (EMPIAR-10031) :

The first step is to create the training data for your model. Right now, you have to use the e2helixboxer.py for this:

e2helixboxer.py --gui my_images/*.mrc

After tracing your training data in e2helixboxer, export them using File → Save. Make sure that you export particle coordinates as this the only format supported right now (see screenshot). In the following example, it is expected that you exported into a folder called “train_annotation”.

In principle, there is not much difference in training crYOLO for filament picking and particle picking. For project with roughly 20 filaments per image we successfully trained on 40 images (⇒ 800 filaments). However, in our experience the warm-up phase and training need a little bit more time:

Train your network with 10 warm up epochs:

cryolo_train.py -c config.json -w 10 -g 0 -e 10

The final model will be called model.h5

The biggest difference in picking filaments with crYOLO is during prediction. However, there are just three additional parameters needed:

• - -filament: Option that tells crYOLO that you want to predict filaments
• -fw: Filament width (pixels)
• -bd: Inter-Box distance (pixels).

Let's assume you want to pick a filament with a width of 100 pixels (-fw 100). The box size is 200×200 and you want a 90% overlap (-bd 20). Moreover, you wish that each filament has at least 6 boxes (-mn 6). The micrographs are in the full_data directory. Than the picking command would be:

cryolo_predict.py -c config.json -w model.h5 -i full_data --filament -fw 100 -bd 20 -o boxes/ -g 0 -mn 6

The directory boxes will be created and all results are saved there. The format is the eman2 helix format with particle coordinates. You can find a detailed description how to import crYOLO filament coordinates into Relion here.

The evaluation tool allows you, based on your validation data, to get statistics about your training. If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows:

cryolo_evaluation.py -c config.json -w model.h5 -r runfiles/run_YearMonthDay-HourMinuteSecond.json -g 0

The result looks like this:

The table contains several statistics:

• AUC: Area under curve of the precision-recall curve. Overall summary statistics. Perfect classifier = 1, Worst classifier = 0
• Topt: Optimal confidence threshold with respect to the F1 score. It might not be ideal for your picking, as the F1 score weighs recall and precision equally. However in SPA, recall is often more important than the precision.
• R (Topt): Recall using the optimal confidence threshold.
• R (0.3): Recall using a confidence threshold of 0.3.
• R (0.2): Recall using a confidence threshold of 0.2.
• P (Topt): Precision using the optimal confidence threshold.
• P (0.3): Precision using a confidence threshold of 0.3.
• P (0.2): Precision using a confidence threshold of 0.2.
• F1 (Topt): Harmonic mean of precision and recall using the optimal confidence threshold.
• F1 (0.3): Harmonic mean of precision and recall using a confidence threshold of 0.3.
• F1 (0.2): Harmonic mean of precision and recall using a confidence threshold of 0.2.
• IOU (Topt): Intersection over union of the auto-picked particles and the corresponding ground-truth boxes. The higher, the better – evaluated with the optimal confidence threshold.
• IOU (0.3): Intersection over union of the auto-picked particles and the corresponding ground-truth boxes. The higher, the better – evaluated with a confidence threshold of 0.3.
• IOU (0.2): Intersection over union of the auto-picked particles and the corresponding ground-truth boxes. The higher, the better – evaluated with a confidence threshold of 0.2.

If the training data consists of multiple folders, then evaluation will be done for each folder separately. Furthermore, crYOLO estimates the optimal picking threshold regarding the F1 Score and F2 Score. Both are basically average values of the recall and prediction, whereas the F2 score puts more weights on the recall, which is in the cryo-em often more important.

During training (cryolo_train), there are the following advanced parameters:

• --warm_restarts: With this option the learning rate is decreasing after each epoch and then reset after a couple of epochs.
• --num_cpu NUMBER_OF_CPUS: Number of CPU cores used during training
• --gpu_fraction FRACTION: Number between 0 - 1 quantifying the fraction of GPU memory that is reserved by crYOLO
• --skip_augmentation: Set this flaq, if crYOLO should skip the data augmentation (not recommended).
• --fine_tune: With this flag, crYOLO will only train the last layers (fine tune)
• -lft NUM_LAYER_FINETUNE: Numbers of layers to fine tune (default is 2).

During picking (cryolo_predict), there are these advanced parameters:

• -t CONFIDENCE_THRESHOLD: With the -t parameter, you can let the crYOLO pick more conservative (e.g by adding -t 0.4 to the picking command) or less conservative (e.g by adding -t 0.2 to the picking command). The valid parameter range is 0 to 1.
• -d DISTANCE_IN_PIXEL: With the -d parameter you can filter your picked particles. Boxes with a distance (pixel) less than this value will be removed.
• -pbs PREDICTION_BATCH_SIZE: With the -pbs parameter you can set the number of images picked as batch. Default is 3.
• --otf: Instead of saving the filtered images into an seperate directory, crYOLO will filter them on-the-fly and don't write them to disk.
• --num_cpu NUMBER_OF_CPUS: Number of CPU cores used during prediction
• --gpu_fraction FRACTION: Number between 0 -1 quantifying the fraction of GPU memory that is reserved by crYOLO.
• –monitor: With this flaq, crYOLO will monitor your input directory and pick images as they appear in the directory. The monitor mode can be stopped by writing the empty file STOP.CRYOLO ¹⁰⁾ into the input directory.
• -sr SEARCH_RANGE_FACTOR: (FILAMENT MODE) The search range for connecting boxes is the box size times this factor. Default is 1.41

Any questions? Problems? Suggestions?

Find help at our mailing list!