This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
pipeline:window:cryolo [2019/09/13 19:05] twagner [Start crYOLO] |
pipeline:window:cryolo [2019/09/17 13:40] twagner [Advanced parameters] |
||
---|---|---|---|
Line 17: | Line 17: | ||
< | < | ||
+ | |||
You can find more technical details in our paper: | You can find more technical details in our paper: | ||
Line 35: | Line 36: | ||
You can find the download and installation instructions here: [[howto: | You can find the download and installation instructions here: [[howto: | ||
+ | |||
+ | ===== Tutorials ===== | ||
+ | |||
+ | Depending what you want to do, you can follow one of these self-contained Tutorials: | ||
+ | |||
+ | - I would like to train a model from scratch for picking my particles | ||
+ | - I would like to train a model from scratch for picking filaments. | ||
+ | - I would like to refine a general model for my particles. | ||
+ | |||
+ | The **first and the second tutorial** are the most common use cases and well tested. The **third tutorial** is still experimental but might give you better results in less time and less training data. | ||
+ | |||
===== Picking particles - Using a model trained for your data ===== | ===== Picking particles - Using a model trained for your data ===== | ||
+ | This tutorial explains you how to train a model specific for you dataset. | ||
- | |||
- | ==== Data preparation ==== | ||
If you followed the installation instructions, | If you followed the installation instructions, | ||
Line 46: | Line 57: | ||
source activate cryolo | source activate cryolo | ||
</ | </ | ||
+ | ==== Data preparation ==== | ||
+ | {{page> | ||
- | In the following I will assume that your image data is in the folder '' | + | ==== Start crYOLO ==== |
+ | {{page> | ||
- | The next step is to create training data. To do so, we have to pick single particles manually in several micrographs. Ideally, the micrographs are picked to completion. [[: | + | ==== Configuration ==== |
- | One may ask how many micrographs have to be picked? It depends! Typically 10 micrographs are a good start. However, that number may increase / decrease due to several factors: | + | {{page> |
- | * A very heterogenous background could make it necessary to pick more micrographs. | + | |
- | * If your micrograph is only sparsely decorated, you may need to pick more micrographs. | + | |
- | We recommend that you start with 10 micrographs, | + | |
- | {{:pipeline:window:box_manager.png? | + | < |
- | To create your training data, crYOLO is shipped with a tool called | + | <div style=" |
+ | < | ||
+ | </ | ||
+ | </ | ||
- | Start the box manager with the following command: | ||
- | < | ||
- | cryolo_boxmanager.py | ||
- | </ | ||
- | Now press //File -> Open image folder// and the select the '' | + | {{page>pipeline:window: |
- | * LEFT MOUSE BUTTON: Place a box | + | ==== Training ==== |
- | * HOLD LEFT MOUSE BUTTON: Move a box | + | |
- | * CONTROL + LEFT MOUSE BUTTON: Remove a box | + | |
- | You can change the box size in the main window, by changing the number in the text field labeled //Box size://. Press //Set// to apply it to all picked particles. __For picking, you should the use minimum sized square which encloses your particle.__ | + | {{page> |
+ | ==== Picking ==== | ||
+ | {{page> | ||
- | If you finished picking from your micrographs, | ||
- | Create a new directory called '' | ||
- | Now create a third folder with the name '' | + | ==== Visualize |
+ | {{page> | ||
+ | |||
+ | ==== Evaluate | ||
+ | {{page> | ||
+ | ===== Picking particles - Without | ||
+ | Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: [[: | ||
+ | |||
+ | Our general models can be found and downloaded here: [[howto: | ||
+ | |||
+ | If you followed | ||
- | ==== Start crYOLO ==== | ||
- | You can use crYOLO either by command line or by using the GUI. The GUI should be easier for most users. You can start it with: | ||
< | < | ||
- | cryoloo.py | + | source activate cryolo |
</ | </ | ||
+ | ==== Start crYOLO ==== | ||
+ | {{page> | ||
- | SCREENSHOT HERE | + | ==== Configuration==== |
+ | In the GUI choose the //config// action. Fill in your target box size and leave the // | ||
- | The crYOLO GUI is basically a visualization of the commandline interface. On left side, you find all possible " | + | {{ :pipeline:window:cryolo_filter_options.png? |
- | * **conifg**: With this action you create the configuration file that you need to run crYOLO. | + | |
- | * **train**: This action let you train crYOLO from scratch or refine an existing model. | + | |
- | * **predict**: | + | |
- | * **evaluation**: This action helps you need to quantify the " | + | |
- | Each action has several parameters which are organized in tabs. Once you chosen your settings you can press "Start", the command will be applied and crYOLO shows you the outputÖ | + | [[: |
- | SCREENSHOT HERE | + | * General model trained for low-pass filtered images : Select //filter// " |
+ | * General model trained for JANNI-denoised images: Select //filter// " | ||
+ | * General model for negative stain images: Select filter " | ||
- | It will tell you when something went wrong. Pressing | + | < |
+ | <div style="background-color: | ||
+ | < | ||
+ | </ | ||
+ | </ | ||
- | ==== Configuration ==== | ||
- | You now have to create a configuration file your picking project. It contains all important constants and paths and helps you to reproduce your results later on. | ||
- | You can either use the commandline to create the configuration file or the GUI. | ||
- | **Using the command line:** | + | < |
+ | In the following I assume that you target box size is 220. Please adapt if necessary. | ||
- | To create an empty file do: | + | For the general **[[: |
< | < | ||
- | touch config.json | + | cryoloo.py |
</ | </ | ||
- | To use the [[:cryolo_nets#network_3_phosaurusnet|Phosaurus network]] copy the following lines into that file: | + | For the general model trained with **neural-network denoised cryo images** (with [[:janni_tutorial#download|JANNI' |
- | < | + | <code> |
- | { | + | cryoloo.py |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | "filter": | + | |
- | }, | + | |
- | + | ||
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | + | ||
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | }, | + | |
- | + | ||
- | " | + | |
- | " | + | |
- | " | + | |
- | + | ||
- | " | + | |
- | } | + | |
- | } | + | |
</ | </ | ||
- | // | ||
- | Please set the value in the //" | + | For the general model for **negative stain data** please run: |
< | < | ||
- | " | + | cryoloo.py config config_cryolo_.json 220 --filter NONE |
</ | </ | ||
- | crYOLO will automatically check if an image in full_data is available in the '' | + | </ |
- | <note tip> | + | ==== Picking ==== |
- | **Alternative: Using neural-network denoising with JANNI** | + | {{page> |
- | Since crYOLO 1.4 you can also use neural network denoising with [[:janni|JANNI]]. The easiest way is to use the JANNI' | + | ==== Visualize the results ==== |
+ | {{page> | ||
+ | ===== Picking particles - Using the general model refined | ||
- | To use JANNI' | ||
- | < | + | Since crYOLO 1.3 you can train a model for your data by // |
- | " | + | |
- | </code> | + | |
- | I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU) | + | What does //fine-tuning// mean? |
- | </ | + | The general model was trained on a lot of particles with a variety of shapes and therefore learned a very good set of generic features. The last layers, however, learn a pretty abstract representation of the particles and it might be that they do not perfectly fit for your particle at hand. Fine-tuning only traines the last two convolutional layers, but keep the others fixed. This adjusts the more abstract representation for your specific problem. |
- | Please note the wiki entry about the [[: | + | Why should I // |
+ | - From theory, using fine-tuning should reduce | ||
+ | - The training is much faster, as not all layers have to be trained. | ||
+ | - The training will need less GPU memory ((We are testing | ||
+ | |||
+ | However, the fine tune mode is still somewhat experimental and we will update this section | ||
- | **Using | + | If you followed |
- | ==== Training ==== | ||
- | |||
- | Now you are ready to train the model. In case you have multiple GPUs, you should first select a free GPU. The following command will show the status of all GPUs: | ||
< | < | ||
- | nvidia-smi | + | source activate cryolo |
</ | </ | ||
- | For this tutorial, we assume that you have either a single GPU or want to use GPU 0. Therefore we add '-g 0' after each command below. However, if you have multiple (e.g GPU 0 and GPU 1) you could also use both by adding '-g 0 1' after each command. | ||
- | Navigate to the folder with '' | + | ==== Data preparation ==== |
+ | {{page> | ||
- | **Train your network with 3 warmup epochs:** | + | ==== Start crYOLO ==== |
- | <code> | + | {{page>pipeline: |
- | cryolo_train.py -c config.json -w 3 -g 0 | + | ==== Configuration ==== |
- | </code> | + | {{page>pipeline: |
- | The final model will be called '' | + | {{ : |
+ | Furthermore, | ||
- | The training stops when the "loss" | + | < |
+ | <div style="background-color: | ||
+ | < | ||
+ | </div> | ||
+ | </html> | ||
- | <code> | + | <hidden **Create the configuration file using the command line:**> |
- | cryolo_train.py -c config.json | + | |
- | </ | + | I assume your box files for training are in the folder '' |
- | to the training command. | ||
- | ==== Picking ==== | ||
- | You can now use the model weights saved in '' | ||
< | < | ||
- | cryolo_predict.py -c config.json -w model.h5 | + | cryoloo.py config |
</ | </ | ||
- | You will find the picked particles in the directory '' | + | To get a full description of all available options type: |
- | + | ||
- | If you want to pick less conservatively or more conservatively you might want to change the selection threshold from the default of 0.3 to a less conservative value like 0.2 or more conservative value like 0.4 using the //-t// parameter: | + | |
< | < | ||
- | cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ -t 0.2 | + | cryoloo.py config -h |
</ | </ | ||
- | However, it is much easier to select the best threshold after picking using the '' | ||
- | ==== Visualize | + | If you want to specify seperate validation folders you can use the %%--%%valid_image_folder and %%--%%valid_annot_folder options: |
- | To visualize your results you can use the box manager: | ||
< | < | ||
- | cryolo_boxmanager.py | + | cryoloo.py config config_cryolo.json 160 --train_image_folder train_image --train_annot_folder train_annot --pretrained_weights gmodel_phosnet_20190516.h5 --valid_image_folder valid_img --valid_annot_folder valid_annot |
</ | </ | ||
- | Now press //File -> Open image// folder and the select the '' | ||
- | Since version 1.3.0 crYOLO writes cbox files in a separate '' | + | </ |
- | [{{ : | + | ==== Training ==== |
- | <note warning> | + | Now you are ready to train the model. In case you have multiple GPUs, you should first select a free GPU. The following command will show the status of all GPUs: |
- | Right now, **this filtering does not yet work for filaments**. | + | |
- | </ | + | |
- | |||
- | |||
- | ===== Picking particles - Without training using a general model ===== | ||
- | Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: [[: | ||
- | |||
- | Our general models can be found and downloaded here: [[howto: | ||
- | ==== Configuration==== | ||
- | The next step is to create a configuration file. Type: | ||
< | < | ||
- | touch config.json | + | nvidia-smi |
</ | </ | ||
- | Open the file with your preferred editor. | + | For this tutorial, we assume that you have either a single GPU or want to use GPU 0. |
- | There are two general **[[: | + | In the GUI choose |
- | === CryoEM images === | + | {{ :pipeline:window:cryolo_refine.png?600 |}} |
- | For the general **[[: | + | |
- | <hidden **config.json for low-pass filtered cryo-images**> | + | |
- | <code json config.json> | + | |
- | { | + | |
- | | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | } | + | |
- | } | + | |
- | </code> | + | |
- | </hidden> | + | |
- | < | + | |
- | For the general model trained with **neural-network denoised cryo images** (with JANNI' | + | |
- | <hidden **config.json for neural-network denoised cryo-images**> | + | |
- | <code json config.json> | + | |
- | | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | | + | |
- | | + | |
- | </ | + | |
- | You can download | + | In the //" |
- | </hidden> | + | {{ : |
- | < | + | <note important> |
- | In all cases please set the value in the //"anchors"// | + | The number of layers to fine tune (specified by layers_fine_tune |
+ | </ | ||
- | === Negative stain images === | ||
- | For the general model for **negative stain data** please use: | ||
- | <hidden **config.json for negative stain images**> | ||
- | <code json config.json> | ||
- | { | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | } | ||
- | } | ||
- | </ | ||
- | </ | ||
- | Please set the value in the //" | + | <note tip> |
- | ==== Picking ==== | + | **Training on CPU** |
- | Just follow the description given [[pipeline: | + | |
- | As for a direct trained model, | + | The fine tune mode is especially useful if you want to [[downloads: |
+ | </ | ||
- | + | <hidden **Run training with the command line**> | |
- | ===== Picking particles - Using the general model refined for your data ===== | + | In comparison to the training from scratch, |
- | + | ||
- | + | ||
- | Since crYOLO 1.3 you can train a model for your data by // | + | |
- | + | ||
- | What does //fine-tuning// mean? | + | |
- | + | ||
- | The general model was trained on a lot of particles with a variety of shapes and therefore learned a very good set of generic features. The last layers, however, learn a pretty abstract representation of the particles and it might be that they do not perfectly fit for your particle at hand. Fine-tuning only traines the last two convolutional layers, but keep the others fixed. This adjusts the more abstract representation for your specific problem. | + | |
- | + | ||
- | Why should I //fine-tune// my model instead of training from scratch? | + | |
- | - From theory, using fine-tuning should reduce the risk of overfitting ((Overfitting means, that the model works good on the training micrographs, | + | |
- | - The training is much faster, as not all layers have to be trained. | + | |
- | - The training will need less GPU memory ((We are testing | + | |
- | + | ||
- | However, the fine tune mode is still somewhat experimental and we will update this section if see more advantages or disadvantages. | + | |
- | + | ||
- | ==== Configuration ==== | + | |
- | + | ||
- | You can use almost the same configuration as used when [[pipeline: | + | |
< | < | ||
- | " | + | cryolo_train.py -c config.json -w 0 -g 0 --fine_tune -lft 2 |
- | [...] | + | |
- | " | + | |
- | [...] | + | |
- | " | + | |
- | [...] | + | |
- | } | + | |
</ | </ | ||
+ | </ | ||
- | ==== Training ==== | ||
- | In comparision to the training from scratch, you can skip the warm up training. Moreover you have to add the // | ||
- | |||
- | < | ||
- | cryolo_train.py -c config.json -w 0 -g 0 --fine_tune | ||
- | </ | ||
==== Picking ==== | ==== Picking ==== | ||
- | Picking is identical as with a model trained from scratch, so we will skip it here. Just follow the description given [[pipeline: | + | {{page>pipeline: |
- | ==== Training on CPU ==== | + | ==== Visualize the results |
+ | {{page> | ||
- | + | ==== Evaluate your results ==== | |
- | The fine tune mode is especially useful if you want to [[downloads:cryolo_1# | + | {{page> |
===== Picking filaments - Using a model trained for your data ===== | ===== Picking filaments - Using a model trained for your data ===== | ||
Since version 1.1.0 crYOLO supports picking filaments. | Since version 1.1.0 crYOLO supports picking filaments. | ||
Line 358: | Line 257: | ||
{{: | {{: | ||
+ | |||
+ | If you followed the installation instructions, | ||
+ | |||
+ | < | ||
+ | source activate cryolo | ||
+ | </ | ||
+ | |||
==== Data preparation ==== | ==== Data preparation ==== | ||
- | {{ : | + | {{ : |
- | After this is done, you have to prepare | + | The first step is to create the training data for your model. Right now, you have to use the e2helixboxer.py |
- | Right now, you have to use the e2helixboxer.py | + | |
< | < | ||
e2helixboxer.py --gui my_images/ | e2helixboxer.py --gui my_images/ | ||
Line 370: | Line 275: | ||
After tracing your training data in e2helixboxer, | After tracing your training data in e2helixboxer, | ||
- | ==== Configuration ==== | + | For projects with roughly 20 filaments per image we successfully trained on 40 images (=> 800 filaments). |
- | You can configure it the same way as for a " | + | |
- | <code json config.json> | + | ==== Start crYOLO ==== |
- | { | + | {{page> |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | | + | |
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | ||
- | " | + | ==== Configuration ==== |
- | " | + | {{page> |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | " | + | |
- | | + | |
- | " | ||
- | " | ||
- | " | ||
- | " | + | < |
- | } | + | <div style="background-color: #cfc ; padding: 10px; border: 1px solid green;"> |
- | } | + | < |
- | </code> | + | </ |
+ | </html> | ||
- | // | ||
- | |||
- | Just adapt the anchors accordingly to your box size. | ||
+ | {{page> | ||
==== Training ==== | ==== Training ==== | ||
- | In principle, there is not much difference in training | + | {{page> |
+ | ==== Picking ==== | ||
+ | Select | ||
+ | {{ : | ||
- | **Train your network with 10 warm up epochs:** | + | Now select the " |
- | < | + | {{ : |
- | cryolo_train.py -c config.json -w 10 -g 0 -e 10 | + | |
- | </ | + | |
- | The final model will be called | + | Press the start button to start the picking. |
- | ==== Picking ==== | + | |
- | The biggest difference in picking filaments with crYOLO | + | You can find a detailed description [[: |
- | + | ||
- | * //- -filament//: Option that tells crYOLO that you want to predict filaments | + | |
- | * //-fw//: Filament width (pixels) | + | |
- | * //-bd//: Inter-Box distance (pixels). | + | |
+ | <hidden **Run prediction in commmand line**> | ||
Let's assume you want to pick a filament with a width of 100 pixels (-fw 100). The box size is 200x200 and you want a 90% overlap (-bd 20). Moreover, you wish that each filament has at least 6 boxes (-mn 6). The micrographs are in the '' | Let's assume you want to pick a filament with a width of 100 pixels (-fw 100). The box size is 200x200 and you want a 90% overlap (-bd 20). Moreover, you wish that each filament has at least 6 boxes (-mn 6). The micrographs are in the '' | ||
< | < | ||
- | cryolo_predict.py -c config.json -w model.h5 -i full_data --filament -fw 100 -bd 20 -o boxes/ -g 0 -mn 6 | + | cryolo_predict.py -c cryolo_config.json -w cryolo_model.h5 -i full_data --filament -fw 100 -bd 20 -o boxes/ -g 0 -mn 6 |
</ | </ | ||
+ | </ | ||
+ | |||
- | The directory '' | ||
==== Visualize the results ==== | ==== Visualize the results ==== | ||
- | You can use the boxmanager as described [[pipeline: | + | {{page>pipeline: |
- | + | ||
- | ===== Evaluate your results ===== | + | |
- | <note warning> | + | |
- | Unfortunately, | + | |
- | </ | + | |
- | The evaluation tool allows you, based on your validation data, to get statistics about your training. | + | |
- | If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows: | + | |
- | < | + | |
- | cryolo_evaluation.py -c config.json -w model.h5 -r runfiles/ | + | |
- | </ | + | |
- | + | ||
- | The result looks like this: | + | |
- | {{: | + | |
- | + | ||
- | The table contains several statistics: | + | |
- | * AUC: Area under curve of the precision-recall curve. Overall summary statistics. Perfect classifier = 1, Worst classifier = 0 | + | |
- | * Topt: Optimal confidence threshold with respect to the F1 score. It might not be ideal for your picking, as the F1 score weighs recall and precision equally. However in SPA, recall is often more important than the precision. | + | |
- | * R (Topt): Recall using the optimal confidence threshold. | + | |
- | * R (0.3): Recall using a confidence threshold of 0.3. | + | |
- | * R (0.2): Recall using a confidence threshold of 0.2. | + | |
- | * P (Topt): Precision using the optimal confidence threshold. | + | |
- | * P (0.3): Precision using a confidence threshold of 0.3. | + | |
- | * P (0.2): Precision using a confidence threshold of 0.2. | + | |
- | * F1 (Topt): Harmonic mean of precision and recall using the optimal confidence threshold. | + | |
- | * F1 (0.3): Harmonic mean of precision and recall using a confidence threshold of 0.3. | + | |
- | * F1 (0.2): Harmonic mean of precision and recall using a confidence threshold of 0.2. | + | |
- | * IOU (Topt): Intersection over union of the auto-picked particles and the corresponding ground-truth boxes. The higher, the better -- evaluated with the optimal confidence threshold. | + | |
- | * IOU (0.3): Intersection over union of the auto-picked particles and the corresponding ground-truth boxes. The higher, the better -- evaluated with a confidence threshold of 0.3. | + | |
- | * IOU (0.2): Intersection over union of the auto-picked particles and the corresponding ground-truth boxes. The higher, the better -- evaluated with a confidence threshold of 0.2. | + | |
- | + | ||
- | If the training data consists of multiple folders, then evaluation will be done for each folder separately. | + | |
- | Furthermore, | + | |
- | ===== Advanced parameters ===== | ||
- | During **training** (// | ||
- | * // | ||
- | * // | ||
- | * // | ||
- | * // | ||
- | During **picking** (// | ||
- | * //-t CONFIDENCE_THRESHOLD//: | ||
- | * //-d DISTANCE_IN_PIXEL//: | ||
- | * //-pbs PREDICTION_BATCH_SIZE//: | ||
- | * // | ||
- | * // | ||
- | * // | ||
- | * //-sr SEARCH_RANGE_FACTOR//: | ||
===== Help ===== | ===== Help ===== |