This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
pipeline:window:cryolo [2019/03/08 14:17] twagner [Overview] |
pipeline:window:cryolo [2019/08/29 14:04] twagner |
||
---|---|---|---|
Line 7: | Line 7: | ||
* crYOLO makes picking **smart** -- The network learns the context of particles (e.g. not to pick particles on carbon or within ice contamination ) | * crYOLO makes picking **smart** -- The network learns the context of particles (e.g. not to pick particles on carbon or within ice contamination ) | ||
* crYOLO makes training **easy** -- You might use a general network model and skip training completely. However, if the general model doesn' | * crYOLO makes training **easy** -- You might use a general network model and skip training completely. However, if the general model doesn' | ||
+ | * crYOLO makes training **tolerant** -- Don't worry if you miss quite a lot particles during creation of your training set. [[: | ||
- | In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information about supported networks and about the config file in the following articles: | + | In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information |
+ | * [[https:// | ||
* [[: | * [[: | ||
* [[: | * [[: | ||
+ | |||
+ | |||
+ | < | ||
You can find more technical details in our paper: | You can find more technical details in our paper: | ||
- | [[https://www.biorxiv.org/content/10.1101/356584v1|SPHIRE-crYOLO: A fast and well-centering | + | [[https://doi.org/10.1038/s42003-019-0437-z|Wagner, T. et al. SPHIRE-crYOLO |
+ | ---- | ||
+ | |||
+ | We are also proud that crYOLO was recommended by F1000: | ||
+ | |||
+ | //" | ||
+ | < | ||
< | < | ||
- | <a href=" | + | <a href=" |
</ | </ | ||
+ | </ | ||
+ | |||
===== Installation ===== | ===== Installation ===== | ||
You can find the download and installation instructions here: [[howto: | You can find the download and installation instructions here: [[howto: | ||
- | ===== Picking - Using a model trained for your data ===== | + | ===== Picking |
==== Data preparation ==== | ==== Data preparation ==== | ||
CrYOLO supports MRC, TIF and JPG files. It can work with 32 bit data, 8 bit data and 16 bit data. | CrYOLO supports MRC, TIF and JPG files. It can work with 32 bit data, 8 bit data and 16 bit data. | ||
- | It will work on original MRC files, but it will probably improve when the data are filtered. Therefore you should low-pass filter them to a reasonable level. Since Version 1.2 crYOLO can automatically do that for you. You just have to add | + | It will work on original MRC files, but it will probably improve when the data are denoised. Therefore you should low-pass filter them to a reasonable level. Since Version 1.2 crYOLO can automatically do that for you. You just have to add |
< | < | ||
" | " | ||
</ | </ | ||
+ | |||
to the model section in your config file to filter your images down to an absolute frequency of 0.1. The filtered images are saved in folder '' | to the model section in your config file to filter your images down to an absolute frequency of 0.1. The filtered images are saved in folder '' | ||
+ | |||
+ | crYOLO will automatically check if an image in full_data is available in the '' | ||
+ | |||
+ | <hidden **Alternative: | ||
+ | < | ||
+ | Since crYOLO 1.4 you can also use neural network denoising with [[: | ||
+ | |||
+ | To use JANNI' | ||
+ | |||
+ | < | ||
+ | " | ||
+ | </ | ||
+ | |||
+ | I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU) | ||
+ | |||
+ | < | ||
+ | </ | ||
+ | < | ||
If you followed the installation instructions, | If you followed the installation instructions, | ||
Line 42: | Line 74: | ||
In the following I will assume that your image data is in the folder '' | In the following I will assume that your image data is in the folder '' | ||
- | The next step is to create training data. To do so, we have to pick single particles manually in several micrographs. Ideally, the micrographs are picked to completion. One may ask how many micrographs have to be picked? It depends! Typically 10 micrographs are a good start. However, that number may increase / decrease due to several factors: | + | The next step is to create training data. To do so, we have to pick single particles manually in several micrographs. Ideally, the micrographs are picked to completion. |
+ | One may ask how many micrographs have to be picked? It depends! Typically 10 micrographs are a good start. However, that number may increase / decrease due to several factors: | ||
* A very heterogenous background could make it necessary to pick more micrographs. | * A very heterogenous background could make it necessary to pick more micrographs. | ||
* If your micrograph is only sparsely decorated, you may need to pick more micrographs. | * If your micrograph is only sparsely decorated, you may need to pick more micrographs. | ||
Line 67: | Line 100: | ||
Create a new directory called '' | Create a new directory called '' | ||
- | Now create a third folder with the name '' | + | Now create a third folder with the name '' |
==== Configuration ==== | ==== Configuration ==== | ||
Line 92: | Line 125: | ||
" | " | ||
" | " | ||
- | " | + | " |
" | " | ||
" | " | ||
Line 116: | Line 149: | ||
// | // | ||
- | Please set the value in the //" | + | Please set the value in the //" |
< | < | ||
" | " | ||
</ | </ | ||
crYOLO will automatically check if an image in full_data is available in the '' | crYOLO will automatically check if an image in full_data is available in the '' | ||
+ | |||
+ | <note tip> | ||
+ | **Alternative: | ||
+ | |||
+ | Since crYOLO 1.4 you can also use neural network denoising with [[: | ||
+ | |||
+ | To use JANNI' | ||
+ | |||
+ | < | ||
+ | " | ||
+ | </ | ||
+ | |||
+ | I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU) | ||
+ | |||
+ | </ | ||
Please note the wiki entry about the [[: | Please note the wiki entry about the [[: | ||
Line 150: | Line 198: | ||
The final model will be called '' | The final model will be called '' | ||
- | The training stops when the " | + | The training stops when the " |
< | < | ||
- | cryolo_train.py -c config.json -w 0 -g 0 -e 10 | + | cryolo_train.py -c config.json -w 0 -g 0 -e 15 |
</ | </ | ||
to the training command. | to the training command. | ||
==== Picking ==== | ==== Picking ==== | ||
- | You can now use the model weights saved in '' | + | You can now use the model weights saved in '' |
< | < | ||
cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ | cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ | ||
</ | </ | ||
- | You will find the picked particles in the directory '' | + | You will find the picked particles in the directory '' |
If you want to pick less conservatively or more conservatively you might want to change the selection threshold from the default of 0.3 to a less conservative value like 0.2 or more conservative value like 0.4 using the //-t// parameter: | If you want to pick less conservatively or more conservatively you might want to change the selection threshold from the default of 0.3 to a less conservative value like 0.2 or more conservative value like 0.4 using the //-t// parameter: | ||
Line 167: | Line 215: | ||
cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ -t 0.2 | cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ -t 0.2 | ||
</ | </ | ||
+ | However, it is much easier to select the best threshold after picking using the '' | ||
==== Visualize the results ==== | ==== Visualize the results ==== | ||
Line 174: | Line 223: | ||
cryolo_boxmanager.py | cryolo_boxmanager.py | ||
</ | </ | ||
- | Now press //File -> Open image// folder and the select the '' | + | Now press //File -> Open image// folder and the select the '' |
+ | Since version 1.3.0 crYOLO writes cbox files in a separate '' | ||
+ | [{{ : | ||
- | ===== Picking - Without training using a general model ===== | + | <note warning> |
+ | Right now, **this filtering does not yet work for filaments**. | ||
+ | </ | ||
- | The general model can be found here: [[howto: | + | |
+ | |||
+ | ===== Picking particles - Without training using a general model ===== | ||
+ | Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: [[: | ||
+ | |||
+ | Our general models | ||
==== Configuration==== | ==== Configuration==== | ||
The next step is to create a configuration file. Type: | The next step is to create a configuration file. Type: | ||
Line 191: | Line 249: | ||
There are two general **[[: | There are two general **[[: | ||
=== CryoEM images === | === CryoEM images === | ||
- | For the general **[[: | + | For the general **[[: |
+ | <hidden **config.json for low-pass filtered cryo-images**> | ||
<code json config.json> | <code json config.json> | ||
{ | { | ||
Line 204: | Line 263: | ||
} | } | ||
</ | </ | ||
- | Please | + | </ |
+ | < | ||
+ | For the general model trained with **neural-network denoised cryo images** (with JANNI' | ||
+ | <hidden **config.json for neural-network denoised cryo-images**> | ||
+ | <code json config.json> | ||
+ | { | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | } | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | You can download the file '' | ||
+ | </ | ||
+ | < | ||
+ | In all cases please | ||
=== Negative stain images === | === Negative stain images === | ||
For the general model for **negative stain data** please use: | For the general model for **negative stain data** please use: | ||
+ | <hidden **config.json for negative stain images**> | ||
<code json config.json> | <code json config.json> | ||
{ | { | ||
Line 219: | Line 299: | ||
} | } | ||
</ | </ | ||
+ | </ | ||
Please set the value in the //" | Please set the value in the //" | ||
Line 225: | Line 306: | ||
Just follow the description given [[pipeline: | Just follow the description given [[pipeline: | ||
- | As for a direct trained model, you might want to play around with the -t parameter | + | As for a direct trained model, you might want to play around with the confidence threshold, either by using the '' |
+ | |||
+ | ===== Picking particles - Using the general model refined for your data ===== | ||
+ | |||
+ | |||
+ | Since crYOLO 1.3 you can train a model for your data by // | ||
+ | |||
+ | What does // | ||
+ | |||
+ | The general model was trained on a lot of particles with a variety of shapes and therefore learned a very good set of generic features. The last layers, however, learn a pretty abstract representation of the particles and it might be that they do not perfectly fit for your particle at hand. Fine-tuning only traines the last two convolutional layers, but keep the others fixed. This adjusts the more abstract representation for your specific problem. | ||
+ | |||
+ | Why should I // | ||
+ | - From theory, using fine-tuning should reduce the risk of overfitting ((Overfitting means, that the model works good on the training micrographs, | ||
+ | - The training is much faster, as not all layers have to be trained. | ||
+ | - The training will need less GPU memory ((We are testing crYOLO with its default configuration on graphic cards with >= 8 GB memory. Using the fine tune mode, it should also work with GPUs with 4 GB memory)) and therefore is usable with NVIDIA cards with less memory. | ||
+ | |||
+ | However, the fine tune mode is still somewhat experimental and we will update this section if see more advantages or disadvantages. | ||
+ | |||
+ | ==== Configuration ==== | ||
+ | |||
+ | You can use almost the same configuration as used when [[pipeline: | ||
+ | |||
+ | < | ||
+ | " | ||
+ | [...] | ||
+ | " | ||
+ | [...] | ||
+ | " | ||
+ | [...] | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | ==== Training ==== | ||
+ | In comparision to the training from scratch, you can skip the warm up training. Moreover you have to add the // | ||
+ | |||
+ | < | ||
+ | cryolo_train.py -c config.json -w 0 -g 0 --fine_tune | ||
+ | </ | ||
+ | ==== Picking ==== | ||
+ | Picking is identical as with a model trained from scratch, so we will skip it here. Just follow the description given [[pipeline: | ||
+ | |||
+ | ==== Training on CPU ==== | ||
+ | |||
+ | |||
+ | The fine tune mode is especially useful if you want to [[downloads: | ||
===== Picking filaments - Using a model trained for your data ===== | ===== Picking filaments - Using a model trained for your data ===== | ||
Since version 1.1.0 crYOLO supports picking filaments. | Since version 1.1.0 crYOLO supports picking filaments. | ||
Line 242: | Line 367: | ||
After this is done, you have to prepare training data for your model. | After this is done, you have to prepare training data for your model. | ||
- | Right now, you have to use the sxhelixboxer.py to generate the training data: | + | Right now, you have to use the e2helixboxer.py to generate the training data: |
< | < | ||
- | sxhelixboxer.py --gui my_images/ | + | e2helixboxer.py --gui my_images/ |
</ | </ | ||
- | After tracing your training data in sxhelixboxer, export them using //File -> Save//. Make sure that you export particle coordinates as this the only format supported right now (see screenshot). In the following example, it is expected that you exported into a folder called " | + | After tracing your training data in e2helixboxer, export them using //File -> Save//. Make sure that you export particle coordinates as this the only format supported right now (see screenshot). In the following example, it is expected that you exported into a folder called " |
==== Configuration ==== | ==== Configuration ==== | ||
- | You can configure it the same way as for a " | + | You can configure it the same way as for a " |
<code json config.json> | <code json config.json> | ||
{ | { | ||
" | " | ||
- | " | + | " |
- | " | + | " |
- | " | + | " |
" | " | ||
- | " | + | " |
- | " | + | " |
}, | }, | ||
Line 267: | Line 393: | ||
" | " | ||
" | " | ||
- | " | + | " |
" | " | ||
" | " | ||
Line 289: | Line 415: | ||
} | } | ||
</ | </ | ||
+ | |||
// | // | ||
Line 329: | Line 456: | ||
===== Evaluate your results ===== | ===== Evaluate your results ===== | ||
- | + | <note warning> | |
- | The evaluation tool allows you, based on your validation data, to get statistics about your training. Unfortunately, | + | Unfortunately, |
+ | </ | ||
+ | The evaluation tool allows you, based on your validation data, to get statistics about your training. | ||
If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows: | If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows: | ||
< | < | ||
Line 360: | Line 489: | ||
===== Advanced parameters ===== | ===== Advanced parameters ===== | ||
- | During **training** (// | + | During **training** (// |
- | * // | + | * //%%--%%warm_restarts//: |
+ | * // | ||
+ | * // | ||
+ | * // | ||
- | During **picking** (// | + | During **picking** (// |
- | * //-t confidence_threshold//: With the -t parameter, you can let the crYOLO pick more conservative (e.g by adding -t 0.4 to the picking command) or less conservative (e.g by adding -t 0.2 to the picking command). The valid parameter range is 0 to 1. | + | * //-t CONFIDENCE_THRESHOLD//: With the -t parameter, you can let the crYOLO pick more conservative (e.g by adding -t 0.4 to the picking command) or less conservative (e.g by adding -t 0.2 to the picking command). The valid parameter range is 0 to 1. |
- | * //-d distance_in_pixel//: With the -d parameter you can filter your picked particles. Boxes with a distance (pixel) less than this value will be removed. | + | * //-d DISTANCE_IN_PIXEL//: With the -d parameter you can filter your picked particles. Boxes with a distance (pixel) less than this value will be removed. |
- | * // | + | * // |
+ | * // | ||
+ | * // | ||
+ | * // | ||
+ | * //-sr SEARCH_RANGE_FACTOR//: | ||
===== Help ===== | ===== Help ===== |