janni_tutorial

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
janni_tutorial [2019/07/10 13:31]
twagner [Training a model for your data]
janni_tutorial [2019/10/25 16:12] (current)
twagner [Denoise]
Line 4: Line 4:
 [[https://​arxiv.org/​abs/​1803.04189|Noise2Noise:​ Learning Image Restoration without Clean Data - arXiv]] [[https://​arxiv.org/​abs/​1803.04189|Noise2Noise:​ Learning Image Restoration without Clean Data - arXiv]]
  
-JANNI can be used command line tool but also provides an simple interface to be integrated into other programs.+Besides ​simple GUI and a commandline interface, JANNI also provides an simple ​python ​interface to be integrated into other programs.
  
 ==== Download and Installation ==== ==== Download and Installation ====
 You can find the download and installation instructions here: [[janni|Download and Installation]] You can find the download and installation instructions here: [[janni|Download and Installation]]
  
 +==== Start JANNI ====
 + If you followed the installation instructions,​ you now have to activate the JANNI virtual environment with 
 +<​code>​
 +source activate janni
 +</​code>​
 +
 +You can use JANNI either by command line or with the GUI. Typically, most users prefer to use the GUI (but we will also provide the command line commands in this tutorial). You can start the GUI with:
 +<​code>​
 +janni_denoise.py
 +</​code>​
 +
 +The GUI is essentially a visualization of the command line interface:
 +
 +{{ ::​start_01.png?​700 |}}
 +
 +On the left side, you can see all available actions:
 +  * **config**: To create a config file, which is needed to train JANNI.
 +  * **train**: This action let you train a model for your data.
 +  * **denoise**:​ Run this action to apply a model on new images.
 +
 +Each action has several parameters which are organized in tabs. Once you chosen your settings you can press "​Start,"​ and the command will be applied and JANNI shows you the output: ​
 +
 +{{ :​janni_start_02.png?​700 |}}
 +
 +It will tell you if something goes wrong. Pressing "​Edit"​ brings you back to your settings, where you can either edit the settings (in case something went wrong) or go to the next action. ​
 ==== Training a model for your data ==== ==== Training a model for your data ====
-In case you want to use the general model ([[janni#​Download|Download here]]) you can skip this part and directly [[janni_tutorial#​denoise|denoise]] your images. ​+In case you want to use the general model ([[janni#​Download|Download here]])you can skip this part and directly [[janni_tutorial#​denoise|denoise]] your images. ​
  
-In case you would like to train a model for your data, the first thing you have to do is to create a configuration file for JANNI. ​In the following I assume you named it //​config.json//​. It should look like this: +In case you would like to train a model for your data, you need to copy a few movie files into a separate directory. We typically use at least 30 movies (unaligned) to train the model. Fewer might also work, but more often work much better. ​ The first thing you have to do is to create a configuration file for JANNI.
-<code json config.json>​ +
-+
-    "​model"​ : { +
-        "​architecture": ​        "​unet",​ +
-        "​patch_size": ​          ​1024 +
-    },+
  
-    ​"train": { +=== Configuration === 
-        "movie_dir": ​           ​"/path/to/movie/directory/",​ +In the GUI choose the action ​"config" ​and fill out the required arguments: 
-        "​even_dir"​            "​even_averages/",​ +{{ ::​jann_config_02.png?​700 |}} 
-        "​odd_dir": ​             "​odd_averages/",​ + 
-        "​batch_size": ​          4, +Press "Start," ​and the config file will be written in the specified ​//config_out_path//. You should see the following output
-        "​learning_rate"​       1e-3, +<hidden **Generate the configuration file with the command line**> 
-        "​nb_epoch": ​            100, +If you would like to use the command lineyou can get a description of all parameters with
-        "​saved_weights_name": ​  "​mymodel.h5" +<​code>​ 
-    } +janni_denoise.py config -h
-}+
 </​code>​ </​code>​
  
-The fields have the following ​meaning: +The following ​command ​will create ​the same config file as with the GUI: 
-  * **architecture**:​ CNN Architecture. Right now, only unet is supported. +<​code>​ 
-  * **patch_size**:​ The image will be denoised in patches. This field defines ​the patch size. +janni_denoise.py config ~/​example/​config_janni.json --movie_dir ​~/example/​movies/​ --even_dir ~/example/​even/ ​--odd_dir ​~/example/odd/ 
-  * **movie_dir**:​ Path to the directory ​with the movie filesIf an average exists already in even_dir or odd_dir (see below), it will be skipped. +</code> 
-  * **even_dir**:​ For each movie in movie_dir, an average based on the //even// frames is calculated and saved in even_dir. +</​hidden>​ 
-  * **odd_dir**: For each movie in movie_dir, an average based on the //​odd// ​frames is calculated and saved in even_dir. + 
-  * **batch_size**:​ How many patches are in one mini-batch. If you have memory problems (e.g with cards < 8GB memory), you can try to reduce this value+=== Training === 
-  * **learning_rate**Learning rate, should not be changed. +In principle, ​you simply ​have to specify the config fileHowever, you might want to specify the GPU ID as wellYou find the GPU option in the //Optional arguments// tab. 
-  * **nb_epoch**Number of epochs to trainMore epochs seems to only slightly improve ​the results. + 
-  * **saved_weights_name**:​ Filename ​of your model. +{{:jann_train_01.png?​420|}}{{:janni_train_02.png?​420|}} 
-In principle you only have to adapt the paths. The other could keep as they are. + 
-We typically use at least 30 movies ​to train the model. Less might also work, more work often much better+Press "​Start" ​to run the training and wait for finishing ​of JANNIAfter that, press //Edit// (where ​the "​Start"​ button used to be) to prepare for the next step.
  
 +<hidden **Run the training with the command line**>
 To run the training on gpu 0: To run the training on gpu 0:
 <​code>​ <​code>​
 janni_denoise.py train config.json -g 0 janni_denoise.py train config.json -g 0
 </​code>​ </​code>​
 +</​hidden>​
 ==== Denoise ==== ==== Denoise ====
  
-With a trained model (either trained by you or the general model) you can directly denoise either your movies or averages. In our experience, denoising the motion corrected averages works better.+With a trained model (either ​a model trained by you or the general model ([[janni#​Download|Download here]]), you can directly denoise either your movies or averages. In our experience, denoising the motion corrected averages works better. ​In the GUI select the action //denoise// and fill the required parameters:​ 
 +{{ ::​janni_denoise.png?​700 |}}
  
-To denoise a set if images you have to tell JANNI three **mandatory** arguments:​ +You might also want to change the GPU ID in //Optional arguments// tabThen, press the //Start// buttonJANNI will denoise your images at roughly 1s per micrograph.
-  - //input_path//: This path points to the directory with your images. +
-  - //​ouput_path//:​ This path points to a directory where the results will be written +
-  - //model_path//: This path points to the model you want to use (the .h5 file).+
  
-As model you can either use the model you trained for your data or the general model ([[janni#​Download|Download here]]). +<​hidden ​**Run prediction in the command line**> 
- +In case you need description ​of all available parameterstype
-There are couple of **optional** parameters that you use: +<​code>​ 
-  * **%%-ol%%**:​ The image is denoised in patches and these patches have to overlap to avoid artifacts. This is the amount of overlap in pixel and the default value is 24. If you observe ​grid like pattern in your images, increase this value. +janni_denoise.py denoise ​-h 
-  * **%%-bs%%**:​ Number ​of patches predicted in parallel. Default is 4. If you have memory problemsdecrease this value. Increase should speed up the things a bit. +</code>
-  * **%%-g%%**GPU ID to run JANNI on. Multiple GPUs are not supported yet. +
- +
-Here is now how you do the actual denoising:​ +
-It is assumed that you run the command in a directory with your model file ''​mymodel.h5''​ (might have a different name in case of the [[janni#​Download|general model]]). Furthermore,​ it is assumed that your would like denoise ​averages in the folder ///​my/​averages///​ and want to write results in the folder ///​my/​outputdir/​denoised//​/+
  
-The following command will run the denoising on GPU 0:+The following command will run the denoise the images in ''/​my/​averages/''​ and save the denoised images in ''/​my/​outputdir/​denoised/''​. The denoising ​here will run on GPU 0:
 <​code>​ <​code>​
-janni_denoise.py ​predict ​/​my/​averages/​ /​my/​outputdir/​denoised/ ​mymodel.h5 -g 0+janni_denoise.py ​denoise ​/​my/​averages/​ /​my/​outputdir/​denoised/ ​janni_imodel.h5 -g 0
 </​code>​ </​code>​
 +</​hidden>​
 +
 +
  
  
  • janni_tutorial.1562758302.txt.gz
  • Last modified: 2019/07/10 13:31
  • by twagner