This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
pipeline:window:cryolo [2019/04/03 14:53] twagner [Data preparation] |
pipeline:window:cryolo [2019/07/11 09:46] twagner [Data preparation] |
||
---|---|---|---|
Line 7: | Line 7: | ||
* crYOLO makes picking **smart** -- The network learns the context of particles (e.g. not to pick particles on carbon or within ice contamination ) | * crYOLO makes picking **smart** -- The network learns the context of particles (e.g. not to pick particles on carbon or within ice contamination ) | ||
* crYOLO makes training **easy** -- You might use a general network model and skip training completely. However, if the general model doesn' | * crYOLO makes training **easy** -- You might use a general network model and skip training completely. However, if the general model doesn' | ||
- | * crYOLO makes training **lenient** -- Don't worry if you miss quite a lot particles during creation of your training set. [[: | + | * crYOLO makes training **tolerant** -- Don't worry if you miss quite a lot particles during creation of your training set. [[: |
In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information about supported networks and about the config file in the following articles: | In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information about supported networks and about the config file in the following articles: | ||
Line 13: | Line 13: | ||
* [[: | * [[: | ||
+ | < | ||
You can find more technical details in our paper: | You can find more technical details in our paper: | ||
- | [[https://www.biorxiv.org/content/10.1101/356584v2|SPHIRE-crYOLO: A fast and accurate fully automated particle picker for cryo-EM | + | [[https://doi.org/10.1038/s42003-019-0437-z|Wagner, T. et al. SPHIRE-crYOLO |
- | ]] | + | |
+ | We are also proud that crYOLO was recommended by F1000: | ||
< | < | ||
<a href=" | <a href=" | ||
</ | </ | ||
+ | </ | ||
===== Installation ===== | ===== Installation ===== | ||
Line 30: | Line 33: | ||
==== Data preparation ==== | ==== Data preparation ==== | ||
CrYOLO supports MRC, TIF and JPG files. It can work with 32 bit data, 8 bit data and 16 bit data. | CrYOLO supports MRC, TIF and JPG files. It can work with 32 bit data, 8 bit data and 16 bit data. | ||
- | It will work on original MRC files, but it will probably improve when the data are filtered. Therefore you should low-pass filter them to a reasonable level. Since Version 1.2 crYOLO can automatically do that for you. You just have to add | + | It will work on original MRC files, but it will probably improve when the data are denoised. Therefore you should low-pass filter them to a reasonable level. Since Version 1.2 crYOLO can automatically do that for you. You just have to add |
< | < | ||
" | " | ||
</ | </ | ||
+ | |||
to the model section in your config file to filter your images down to an absolute frequency of 0.1. The filtered images are saved in folder '' | to the model section in your config file to filter your images down to an absolute frequency of 0.1. The filtered images are saved in folder '' | ||
+ | |||
+ | crYOLO will automatically check if an image in full_data is available in the '' | ||
+ | |||
+ | <hidden **Alternative: | ||
+ | < | ||
+ | Since crYOLO 1.4 you can also use neural network denoising with [[: | ||
+ | |||
+ | To use JANNI' | ||
+ | |||
+ | < | ||
+ | " | ||
+ | </ | ||
+ | |||
+ | I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU) | ||
+ | |||
+ | < | ||
+ | </ | ||
+ | < | ||
If you followed the installation instructions, | If you followed the installation instructions, | ||
Line 70: | Line 92: | ||
Create a new directory called '' | Create a new directory called '' | ||
- | Now create a third folder with the name '' | + | Now create a third folder with the name '' |
==== Configuration ==== | ==== Configuration ==== | ||
Line 95: | Line 117: | ||
" | " | ||
" | " | ||
- | " | + | " |
" | " | ||
" | " | ||
Line 124: | Line 146: | ||
</ | </ | ||
crYOLO will automatically check if an image in full_data is available in the '' | crYOLO will automatically check if an image in full_data is available in the '' | ||
+ | |||
+ | <note tip> | ||
+ | **Alternative: | ||
+ | |||
+ | Since crYOLO 1.4 you can also use neural network denoising with [[: | ||
+ | |||
+ | To use JANNI' | ||
+ | |||
+ | < | ||
+ | " | ||
+ | </ | ||
+ | |||
+ | I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU) | ||
+ | |||
+ | </ | ||
Please note the wiki entry about the [[: | Please note the wiki entry about the [[: | ||
Line 184: | Line 221: | ||
[{{ : | [{{ : | ||
+ | <note warning> | ||
Right now, **this filtering does not yet work for filaments**. | Right now, **this filtering does not yet work for filaments**. | ||
+ | </ | ||
+ | |||
===== Picking particles - Without training using a general model ===== | ===== Picking particles - Without training using a general model ===== | ||
+ | Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: [[: | ||
- | The general models can be found and downloaded here: [[howto: | + | Our general models can be found and downloaded here: [[howto: |
==== Configuration==== | ==== Configuration==== | ||
The next step is to create a configuration file. Type: | The next step is to create a configuration file. Type: | ||
Line 200: | Line 241: | ||
There are two general **[[: | There are two general **[[: | ||
=== CryoEM images === | === CryoEM images === | ||
- | For the general **[[: | + | For the general **[[: |
+ | <hidden **config.json for low-pass filtered cryo-images**> | ||
<code json config.json> | <code json config.json> | ||
{ | { | ||
Line 213: | Line 255: | ||
} | } | ||
</ | </ | ||
- | Please | + | </ |
+ | < | ||
+ | For the general model trained with **neural-network denoised cryo images** (with JANNI' | ||
+ | <hidden **config.json for neural-network denoised cryo-images**> | ||
+ | <code json config.json> | ||
+ | { | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | } | ||
+ | } | ||
+ | </ | ||
+ | </ | ||
+ | < | ||
+ | In all cases please | ||
=== Negative stain images === | === Negative stain images === | ||
Line 268: | Line 328: | ||
==== Training ==== | ==== Training ==== | ||
- | In comparision to the training from scratch, you can skip the warm up training. Moreover you have to add the // | + | In comparision to the training from scratch, you can skip the warm up training. Moreover you have to add the //%%--%%fine_tune// flag: |
< | < | ||
Line 321: | Line 381: | ||
" | " | ||
" | " | ||
- | " | + | " |
" | " | ||
" | " | ||
Line 384: | Line 444: | ||
===== Evaluate your results ===== | ===== Evaluate your results ===== | ||
- | + | <note warning> | |
- | The evaluation tool allows you, based on your validation data, to get statistics about your training. Unfortunately, | + | Unfortunately, |
+ | </ | ||
+ | The evaluation tool allows you, based on your validation data, to get statistics about your training. | ||
If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows: | If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows: | ||
< | < | ||
Line 428: | Line 490: | ||
* // | * // | ||
* // | * // | ||
+ | * //-sr SEARCH_RANGE_FACTOR//: | ||
===== Help ===== | ===== Help ===== |