This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
pipeline:window:cryolo [2019/07/11 09:47] twagner [Configuration] |
pipeline:window:cryolo [2019/09/14 10:19] twagner [Visualize the results] |
||
---|---|---|---|
Line 9: | Line 9: | ||
* crYOLO makes training **tolerant** -- Don't worry if you miss quite a lot particles during creation of your training set. [[: | * crYOLO makes training **tolerant** -- Don't worry if you miss quite a lot particles during creation of your training set. [[: | ||
- | In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information about supported networks and about the config file in the following articles: | + | In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information |
+ | * [[https:// | ||
* [[: | * [[: | ||
* [[: | * [[: | ||
+ | |||
+ | |||
< | < | ||
Line 17: | Line 20: | ||
[[https:// | [[https:// | ||
+ | |||
+ | ---- | ||
We are also proud that crYOLO was recommended by F1000: | We are also proud that crYOLO was recommended by F1000: | ||
+ | //" | ||
+ | < | ||
< | < | ||
- | <a href=" | + | <a href=" |
</ | </ | ||
</ | </ | ||
+ | |||
===== Installation ===== | ===== Installation ===== | ||
You can find the download and installation instructions here: [[howto: | You can find the download and installation instructions here: [[howto: | ||
- | ===== Picking particles - Using a model trained for your data ===== | + | ===== Tutorials |
+ | Depending what you want to do, you can follow one of these Tutorials: | ||
- | ==== Data preparation ==== | + | - I would like to train a model from scratch for picking my particles |
- | CrYOLO supports MRC, TIF and JPG files. It can work with 32 bit data, 8 bit data and 16 bit data. | + | - I would like to train a model from scratch for picking filaments. |
- | It will work on original MRC files, but it will probably improve when the data are denoised. Therefore you should low-pass filter them to a reasonable level. Since Version 1.2 crYOLO can automatically do that for you. You just have to add | + | - I would like to refine |
- | < | + | |
- | " | + | |
- | </ | + | |
- | to the model section in your config file to filter your images down to an absolute frequency of 0.1. The filtered images are saved in folder '' | + | The **first and the second tutorial** are the most common use cases and well tested. The **third tutorial** is still experimental but might give you better results |
- | crYOLO will automatically check if an image in full_data is available in the '' | ||
- | <hidden **Alternative: | ||
- | < | ||
- | Since crYOLO 1.4 you can also use neural network denoising with [[: | ||
- | To use JANNI' | + | ===== Picking particles - Using a model trained for your data ===== |
- | < | ||
- | " | ||
- | </ | ||
- | |||
- | I recommend to use denoising with JANNI only together with a GPU as it is rather slow (~ 1-2 seconds per micrograph on the GPU and 10 seconds per micrograph on the CPU) | ||
- | |||
- | < | ||
- | </ | ||
- | < | ||
+ | ==== Data preparation ==== | ||
If you followed the installation instructions, | If you followed the installation instructions, | ||
Line 69: | Line 63: | ||
One may ask how many micrographs have to be picked? It depends! Typically 10 micrographs are a good start. However, that number may increase / decrease due to several factors: | One may ask how many micrographs have to be picked? It depends! Typically 10 micrographs are a good start. However, that number may increase / decrease due to several factors: | ||
* A very heterogenous background could make it necessary to pick more micrographs. | * A very heterogenous background could make it necessary to pick more micrographs. | ||
+ | * When you refine a general model, you might need to pick less micrographs. | ||
* If your micrograph is only sparsely decorated, you may need to pick more micrographs. | * If your micrograph is only sparsely decorated, you may need to pick more micrographs. | ||
- | We recommend that you start with 10 micrographs, | ||
+ | We recommend that you start with 10 micrographs, | ||
{{: | {{: | ||
Line 87: | Line 82: | ||
* CONTROL + LEFT MOUSE BUTTON: Remove a box | * CONTROL + LEFT MOUSE BUTTON: Remove a box | ||
- | You can change the box size in the main window, by changing the number in the text field labeled //Box size://. Press //Set// to apply it to all picked particles. | + | You can change the box size in the main window, by changing the number in the text field labeled //Box size://. Press //Set// to apply it to all picked particles. |
If you finished picking from your micrographs, | If you finished picking from your micrographs, | ||
Line 93: | Line 88: | ||
Now create a third folder with the name '' | Now create a third folder with the name '' | ||
+ | |||
+ | ==== Start crYOLO ==== | ||
+ | {{page> | ||
==== Configuration ==== | ==== Configuration ==== | ||
- | You now have to create a config | + | You now have to create a configuration |
+ | |||
+ | You can either use the commandline to create the configuration file or the GUI. | ||
+ | |||
+ | **Using the command line:** | ||
+ | |||
+ | To create an empty file do: | ||
< | < | ||
touch config.json | touch config.json | ||
Line 141: | Line 145: | ||
// | // | ||
- | Please set the value in the //" | + | Please set the value in the //" |
< | < | ||
" | " | ||
Line 150: | Line 154: | ||
**Alternative: | **Alternative: | ||
- | Since crYOLO 1.4 you can also use neural network denoising with [[: | + | Since crYOLO 1.4 you can also use neural network denoising with [[: |
To use JANNI' | To use JANNI' | ||
Line 164: | Line 168: | ||
Please note the wiki entry about the [[: | Please note the wiki entry about the [[: | ||
+ | **Using the GUI:** | ||
==== Training ==== | ==== Training ==== | ||
Line 176: | Line 180: | ||
Navigate to the folder with '' | Navigate to the folder with '' | ||
- | **1. Warm up your network** | + | **Train your network |
< | < | ||
cryolo_train.py -c config.json -w 3 -g 0 | cryolo_train.py -c config.json -w 3 -g 0 | ||
- | </ | ||
- | |||
- | **2. Train your network** | ||
- | |||
- | < | ||
- | cryolo_train.py -c config.json -w 0 -g 0 | ||
</ | </ | ||
Line 191: | Line 189: | ||
The training stops when the " | The training stops when the " | ||
+ | |||
< | < | ||
- | cryolo_train.py -c config.json -w 0 -g 0 -e 15 | + | cryolo_train.py -c config.json -w 3 -g 0 -e 15 |
</ | </ | ||
+ | |||
to the training command. | to the training command. | ||
==== Picking ==== | ==== Picking ==== | ||
- | You can now use the model weights saved in '' | + | {{page>pipeline: |
- | < | + | |
- | cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ | + | |
- | </code> | + | |
- | You will find the picked particles in the directory '' | ||
- | |||
- | If you want to pick less conservatively or more conservatively you might want to change the selection threshold from the default of 0.3 to a less conservative value like 0.2 or more conservative value like 0.4 using the //-t// parameter: | ||
- | < | ||
- | cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ -t 0.2 | ||
- | </ | ||
- | However, it is much easier to select the best threshold after picking using the '' | ||
==== Visualize the results ==== | ==== Visualize the results ==== | ||
- | + | {{page>pipeline: | |
- | To visualize your results you can use the box manager: | + | |
- | < | + | |
- | cryolo_boxmanager.py | + | |
- | </ | + | |
- | Now press //File -> Open image// folder and the select the '' | + | |
- | + | ||
- | Since version 1.3.0 crYOLO writes cbox files in a separate '' | + | |
- | + | ||
- | [{{ :pipeline: | + | |
- | + | ||
- | <note warning> | + | |
- | Right now, **this filtering does not yet work for filaments**. | + | |
- | </ | + | |
- | + | ||
- | + | ||
===== Picking particles - Without training using a general model ===== | ===== Picking particles - Without training using a general model ===== | ||
Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: [[: | Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: [[: | ||
Our general models can be found and downloaded here: [[howto: | Our general models can be found and downloaded here: [[howto: | ||
+ | |||
+ | ==== Start crYOLO ==== | ||
+ | {{page> | ||
+ | |||
==== Configuration==== | ==== Configuration==== | ||
The next step is to create a configuration file. Type: | The next step is to create a configuration file. Type: | ||
Line 267: | Line 245: | ||
" | " | ||
" | " | ||
- | " | + | " |
} | } | ||
} | } | ||
</ | </ | ||
+ | |||
+ | You can download the file '' | ||
</ | </ | ||
< | < | ||
Line 277: | Line 257: | ||
=== Negative stain images === | === Negative stain images === | ||
For the general model for **negative stain data** please use: | For the general model for **negative stain data** please use: | ||
+ | <hidden **config.json for negative stain images**> | ||
<code json config.json> | <code json config.json> | ||
{ | { | ||
Line 288: | Line 269: | ||
} | } | ||
</ | </ | ||
+ | </ | ||
Please set the value in the //" | Please set the value in the //" | ||
==== Picking ==== | ==== Picking ==== | ||
- | Just follow the description given [[pipeline: | + | {{page>pipeline: |
- | + | ||
- | As for a direct trained model, you might want to play around with the confidence threshold, either by using the '' | + | |
- | + | ||
===== Picking particles - Using the general model refined for your data ===== | ===== Picking particles - Using the general model refined for your data ===== | ||
Line 412: | Line 390: | ||
In principle, there is not much difference in training crYOLO for filament picking and particle picking. For project with roughly 20 filaments per image we successfully trained on 40 images (=> 800 filaments). However, in our experience the warm-up phase and training need a little bit more time: | In principle, there is not much difference in training crYOLO for filament picking and particle picking. For project with roughly 20 filaments per image we successfully trained on 40 images (=> 800 filaments). However, in our experience the warm-up phase and training need a little bit more time: | ||
- | **1. Warm up your network** | + | **Train your network |
- | + | ||
- | < | + | |
- | cryolo_train.py -c config.json -w 10 -g 0 | + | |
- | </ | + | |
- | + | ||
- | **2. Train your network** | + | |
< | < | ||
- | cryolo_train.py -c config.json -w 0 -g 0 -e 10 | + | cryolo_train.py -c config.json -w 10 -g 0 -e 10 |
</ | </ | ||