Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
pipeline:window:cryolo [2019/03/17 20:00]
twagner [Training on CPU]
pipeline:window:cryolo [2019/05/17 08:25] (current)
twagner [Advanced parameters]
Line 7: Line 7:
   * crYOLO makes picking **smart** -- The network learns the context of particles (e.g. not to pick particles on carbon or within ice contamination )   * crYOLO makes picking **smart** -- The network learns the context of particles (e.g. not to pick particles on carbon or within ice contamination )
   * crYOLO makes training **easy** -- You might use a general network model and skip training completely. However, if the general model doesn'​t give you satisfactory results or if you would like to improve them, you might want to train a specialized model specific for your data set by selecting __particles__ (no selection of negative examples necessary) on a small number of micrographs.   * crYOLO makes training **easy** -- You might use a general network model and skip training completely. However, if the general model doesn'​t give you satisfactory results or if you would like to improve them, you might want to train a specialized model specific for your data set by selecting __particles__ (no selection of negative examples necessary) on a small number of micrographs.
-  * crYOLO makes training **lenient** -- Don't worry if you miss quite a lot particles during creation of your training set. [[:​cryolo_picking_unlabeled|crYOLO will still do the job.]]+  * crYOLO makes training **tolerant** -- Don't worry if you miss quite a lot particles during creation of your training set. [[:​cryolo_picking_unlabeled|crYOLO will still do the job.]]
  
 In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information about supported networks and about the config file in the following articles: In this tutorial we explain our recommended configurations for single particle and filament projects. You can find more information about supported networks and about the config file in the following articles:
Line 70: Line 70:
 Create a new directory called ''​train_annotation''​ and save it there. Close boxmanager. Create a new directory called ''​train_annotation''​ and save it there. Close boxmanager.
  
-Now create a third folder with the name ''​train_image''​. Now for each box file, copy the corresponding image from ''​full_data''​ into ''​train_image''​. crYOLO will detect image / box file pairs by search taking the box file an searching for an image filename which contains the box filename.+Now create a third folder with the name ''​train_image''​. Now for each box file, copy the corresponding image from ''​full_data''​ into ''​train_image''​((While it is nice to keep the things organized, you don't have to copy your training images in a separate folder. In the configuration file (see below) you can also simply specify the full_data directory as "//​train_image_folder//"​. crYOLO will find the correct images using the box files.)). crYOLO will detect image / box file pairs by search taking the box file an searching for an image filename which contains the box filename.
  
 ==== Configuration ==== ==== Configuration ====
Line 95: Line 95:
         "​train_times": ​         10,         "​train_times": ​         10,
         "​pretrained_weights": ​  "​model.h5",​         "​pretrained_weights": ​  "​model.h5",​
-        "​batch_size": ​          6,+        "​batch_size": ​          4,
         "​learning_rate": ​       1e-4,         "​learning_rate": ​       1e-4,
         "​nb_epoch": ​            50,         "​nb_epoch": ​            50,
Line 159: Line 159:
 to the training command. to the training command.
 ==== Picking ==== ==== Picking ====
-You can now use the model weights saved in ''​model.h5''​ to pick all your images in the directory ''​full_data''​. To do this, run: +You can now use the model weights saved in ''​model.h5'' ​(//if you come to this section from another point of the tutorial, this filename might be different like ''​gmodel_phosnet_X_Y.h5''//​) ​to pick all your images in the directory ''​full_data''​. To do this, run: 
 <​code>​ <​code>​
 cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/ cryolo_predict.py -c config.json -w model.h5 -i full_data/ -g 0 -o boxfiles/
 </​code>​ </​code>​
  
-You will find the picked particles in the directory ''​boxfiles''​+You will find the picked particles in the directory ''​boxfiles''​.
  
 If you want to pick less conservatively or more conservatively you might want to change the selection threshold from the default of 0.3 to a less conservative value like 0.2 or more conservative value like 0.4 using the //-t// parameter: If you want to pick less conservatively or more conservatively you might want to change the selection threshold from the default of 0.3 to a less conservative value like 0.2 or more conservative value like 0.4 using the //-t// parameter:
Line 188: Line 188:
  
 ===== Picking particles - Without training using a general model ===== ===== Picking particles - Without training using a general model =====
 +Here you can find how to apply the general models we trained for you. If you would like to train your own general model, please see our extra wiki page: [[:​cryolo_train_general_model|How to train your own general model]]
  
-The general ​model can be found here: [[howto:​download_latest_cryolo|Download and Installation]]. ​+Our general ​models ​can be found and downloaded ​here: [[howto:​download_latest_cryolo|Download and Installation]]. ​
 ==== Configuration==== ==== Configuration====
 The next step is to create a configuration file. Type: The next step is to create a configuration file. Type:
Line 268: Line 269:
  
 ==== Training ==== ==== Training ====
-In comparision to the training from scratch, you can skip the warm up training. Moreover you have to add the //​--fine_tune//​ flag:+In comparision to the training from scratch, you can skip the warm up training. Moreover you have to add the //%%--%%fine_tune// flag:
  
 <​code>​ <​code>​
Line 303: Line 304:
  
 ==== Configuration ==== ==== Configuration ====
-You can configure it the same way as for a "​normal"​ project. ​We recommend to use [[:​cryolo_nets#​network_2_yolo_with_patches|YOLO in "patch mode" with 3x3 patches]]:+You can configure it the same way as for a "​normal"​ project. 
 <code json config.json>​ <code json config.json>​
 { {
     "​model"​ : {     "​model"​ : {
-        "​architecture": ​        "​YOLO", +        "​architecture": ​        "​PhosaurusNet", 
-        "​input_size": ​          768+        "​input_size": ​          1024
-        "​anchors": ​             [200,200],+        "​anchors": ​             [160,160],
         "​max_box_per_image": ​   600,         "​max_box_per_image": ​   600,
-        "​num_patches": ​         ​3+        "​num_patches": ​         ​1
-        "​filter": ​              ​[0.1,"​tmp_filtered"]+        "​filter": ​              ​[0.1,"​filtered"]
     },     },
  
Line 320: Line 322:
         "​train_times": ​         10,         "​train_times": ​         10,
         "​pretrained_weights": ​  "​model.h5",​         "​pretrained_weights": ​  "​model.h5",​
-        "​batch_size": ​          6,+        "​batch_size": ​          4,
         "​learning_rate": ​       1e-4,         "​learning_rate": ​       1e-4,
         "​nb_epoch": ​            50,         "​nb_epoch": ​            50,
Line 342: Line 344:
 } }
 </​code>​ </​code>​
 +
 //​[[:​cryolo_config|Click here to get more information about the configuration file]]// //​[[:​cryolo_config|Click here to get more information about the configuration file]]//
  
Line 426: Line 429:
   * //​%%--%%num_cpu NUMBER_OF_CPUS//:​ Number of CPU cores used during prediction   * //​%%--%%num_cpu NUMBER_OF_CPUS//:​ Number of CPU cores used during prediction
   * //​%%--%%gpu_fraction FRACTION//: Number between 0 -1 quantifying the fraction of GPU memory that is reserved by crYOLO   * //​%%--%%gpu_fraction FRACTION//: Number between 0 -1 quantifying the fraction of GPU memory that is reserved by crYOLO
 +  * //-sr SEARCH_RANGE_FACTOR//:​ (FILAMENT MODE) The search range for connecting boxes is the box size times this factor. Default is 1.41
  
 ===== Help ===== ===== Help =====