User Tools

Site Tools


pipeline:window:cryolo

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
pipeline:window:cryolo [2019/05/17 08:25]
twagner [Advanced parameters]
pipeline:window:cryolo [2019/07/11 09:49]
twagner [Configuration]
Line 13: Line 13:
   * [[:cryolo_config|crYOLO configuration file]]   * [[:cryolo_config|crYOLO configuration file]]
  
 +<note>
 You can find more technical details in our paper: You can find more technical details in our paper:
  
-[[https://www.biorxiv.org/content/10.1101/356584v2|SPHIRE-crYOLO: A fast and accurate fully automated particle picker for cryo-EM +[[https://doi.org/10.1038/s42003-019-0437-z|Wagner, T. et al. SPHIRE-crYOLO is a fast and accurate fully automated particle picker for cryo-EM. Communications Biology 2, (2019). ]] 
-]]+ 
 +We are also proud that crYOLO was recommended by F1000:
  
 <html> <html>
 <a href="https://f1000.com/prime/733517098?bd=1" target="_blank"><img src="https://s3.amazonaws.com/cdn.f1000.com/images/badges/badgef1000.gif" alt="Access the recommendation on F1000Prime" id="bg" /></a> <a href="https://f1000.com/prime/733517098?bd=1" target="_blank"><img src="https://s3.amazonaws.com/cdn.f1000.com/images/badges/badgef1000.gif" alt="Access the recommendation on F1000Prime" id="bg" /></a>
 </html> </html>
 +</note>
 ===== Installation ===== ===== Installation =====
  
Line 30: Line 33:
 ==== Data preparation ==== ==== Data preparation ====
 CrYOLO supports MRC, TIF and JPG files. It can work with 32 bit data, 8 bit data and 16 bit data. CrYOLO supports MRC, TIF and JPG files. It can work with 32 bit data, 8 bit data and 16 bit data.
-It will work on original MRC files, but it will probably improve when the data are filtered. Therefore you should low-pass filter them to a reasonable level. Since Version 1.2 crYOLO can automatically do that for you. You just have to add+It will work on original MRC files, but it will probably improve when the data are denoised. Therefore you should low-pass filter them to a reasonable level. Since Version 1.2 crYOLO can automatically do that for you. You just have to add
 <code> <code>
 "filter":               [0.1,"filtered"] "filter":               [0.1,"filtered"]
 </code>  </code> 
 +
 to the model section in your config file to filter your images down to an absolute frequency of 0.1. The filtered images are saved in folder ''filtered''. to the model section in your config file to filter your images down to an absolute frequency of 0.1. The filtered images are saved in folder ''filtered''.
 +
 +crYOLO will automatically check if an image in full_data is available in the ''filtered'' directory. The filtering is done in parallel. If you don't want to use crYOLO's internal filtering, just remove the line and filter them manually. If you remove the line, don't forget to remove the comma at the end of the line above. 
 +
 +<hidden **Alternative: Using neural-network denoising with JANNI**>
 +<html><br></html>
 +Since crYOLO 1.4 you can also use neural network denoising with [[:janni|JANNI]]. The easiest way is to use the JANNI's general model ([[:janni#janni_general_model|Download here]]) but you can also [[:janni_tutorial#training_a_model_for_your_data|train JANNI for your data]]. crYOLO directly uses an interface to JANNI to filter your data, you just have to specify the path to your JANNI model, overlap of the batches (default 24), the batch size (default 3) and a path where the denoised images should be written. 
 +
 +To use JANNI's denoising you have to use following entry in your config.json:
 +
 +<code>
 +"filter":               ["path/to/janni_model.h5",24,3,"filtered"]
 +</code> 
 +
 +I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU)
 +
 +<html><br></html>
 +</hidden> 
 +<html><br></html>
  
 If you followed the installation instructions, you now have to activate the cryolo virtual environment with If you followed the installation instructions, you now have to activate the cryolo virtual environment with
Line 124: Line 146:
 </code> </code>
 crYOLO will automatically check if an image in full_data is available in the ''filtered'' directory. The filtering is done in parallel. If you don't want to use crYOLO's internal filtering, just remove the line and filter them manually. If you remove the line, don't forget to remove the comma at the end of the line above.  crYOLO will automatically check if an image in full_data is available in the ''filtered'' directory. The filtering is done in parallel. If you don't want to use crYOLO's internal filtering, just remove the line and filter them manually. If you remove the line, don't forget to remove the comma at the end of the line above. 
 +
 +<note tip>
 +**Alternative: Using neural-network denoising with JANNI**
 +
 +Since crYOLO 1.4 you can also use neural network denoising with [[:janni|JANNI]]. The easiest way is to use the JANNI's general model ([[:janni#janni_general_model|Download here]]) but you can also [[:janni_tutorial#training_a_model_for_your_data|train JANNI for your data]]. crYOLO directly uses an interface to JANNI to filter your data, you just have to specify the path to your JANNI model, overlap of the batches (default 24), the batch size (default 3) and a path where the denoised images should be written. 
 +
 +To use JANNI's denoising you have to use following entry in your config.json:
 +
 +<code>
 +"filter":               ["path/to/janni_model.h5",24,3,"filtered"]
 +</code> 
 +
 +I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU)
 +
 +</note>
  
 Please note the wiki entry about the [[:cryolo_config|crYOLO configuration file]] if you want to know more details. Please note the wiki entry about the [[:cryolo_config|crYOLO configuration file]] if you want to know more details.
Line 184: Line 221:
 [{{ :pipeline:window:ezgif-1-3b966b0324d1.gif?400 |This example shows how to filter particle boxes using the cryolo boxmanager. It is an animated gif. Click on it to see it playing.}}] [{{ :pipeline:window:ezgif-1-3b966b0324d1.gif?400 |This example shows how to filter particle boxes using the cryolo boxmanager. It is an animated gif. Click on it to see it playing.}}]
  
 +<note warning>
 Right now, **this filtering does not yet work for filaments**. Right now, **this filtering does not yet work for filaments**.
 +</note>
 +
  
  
Line 201: Line 241:
 There are two general **[[:cryolo_nets#network_3_phosaurusnet|Phosaurus networks]]** available. One for cryo em images and one for negative stain data. There are two general **[[:cryolo_nets#network_3_phosaurusnet|Phosaurus networks]]** available. One for cryo em images and one for negative stain data.
 === CryoEM images === === CryoEM images ===
-For the general **[[:cryolo_nets#network_3_phosaurusnet|Phosaurus network]]** trained for **cryo images** enter the following inside:+For the general **[[:cryolo_nets#network_3_phosaurusnet|Phosaurus network]]** trained for **low-pass filtered cryo images** enter the following inside: 
 +<hidden **config.json for low-pass filtered cryo-images**>
 <code json config.json> <code json config.json>
     {     {
Line 214: Line 255:
     }     }
 </code> </code>
-Please set the value in the //"anchors"// field to your desired box size. It should be size of the minimum particle enclosing square in pixel. +</hidden> 
 +<html><br></html> 
 +For the general model trained with **neural-network denoised cryo images** (with JANNI's general model) enter the following inside: 
 +<hidden **config.json for neural-network denoised cryo-images**> 
 +<code json config.json> 
 +    { 
 +    "model" : { 
 +        "architecture":         "PhosaurusNet", 
 +        "input_size":           1024, 
 +        "anchors":              [205,205], 
 +        "max_box_per_image":    700, 
 +        "num_patches":          1, 
 +        "filter":               ["gmodel_janni_20190703.h5",24,3,"tmp_filtered_nn"
 +      } 
 +    } 
 +</code> 
 + 
 +You can download the file ''gmodel_janni_20190703.h5'' [[https://github.com/MPI-Dortmund/sphire-janni/tree/master/janni_general_models|here]] 
 +</hidden> 
 +<html><br></html> 
 +In all cases please set the value in the //"anchors"// field to your desired box size. It should be size of the minimum particle enclosing square in pixel. 
  
 === Negative stain images === === Negative stain images ===
Line 385: Line 446:
  
 ===== Evaluate your results ===== ===== Evaluate your results =====
- +<note warning> 
-The evaluation tool allows you, based on your validation data, to get statistics about your training. Unfortunately, this script **does not work for filamental data**.+Unfortunately, this script **does not work for filamental data**. 
 +</note> 
 +The evaluation tool allows you, based on your validation data, to get statistics about your training.
 If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows: If you followed the tutorial, the validation data are selected randomly. With crYOLO 1.1.0 a run file for each training is created and saved into the folder runfiles/ in your project directory. This run file contains which files were selected for validation, and you can run your evaluation as follows:
 <code> <code>
pipeline/window/cryolo.txt · Last modified: 2021/02/19 10:00 by twagner