Differences

This shows you the differences between two versions of the page.

--- pipeline:utilities:sxrelion2sphire [2018/06/20 13:12]
127.0.0.1 external edit
+++ pipeline:utilities:sxrelion2sphire [2019/04/02 10:57]
lusnig [Output]
@@ Line 1: / Line 1: @@
 ~~NOTOC~~
-===== sxrelion2sphire =====
+===== sp_relion2sphire =====
-RELION to SPHIRE Conversion: Create several types of parameter text files and a particle image stack in SPHIRE format from parameters stored in a RELION STAR file
+RELION to SPHIRE Conversion: Create several types of parameter text files and per-micrograph virtual stacks of particle images in bdb format from parameters stored in a RELION STAR file
 \\
@@ Line 9: / Line 9: @@
 Usage in command line
-  sxrelion2sphire.py  input_star_file  output_directory  --star_section=STAR_SECTION  --box_size=BOX_SIZE  --create_stack
+  sp_relion2sphire.py   input_star_file output_directory --relion_project_dir=DIR_PATH --star_section=SECTION_STRING --outputs_root=ROOT_NAME_STRING --box_size=BOX_SIZE --do_not_create_stack
 \\
 ===== Typical usage =====
-sxrelion2sphire does not support MPI.
+sp_relion2sphire does not support MPI.
-  sxrelion2sphire.py  Particles.star  sphire_dir  --box_size=256  --create_stack
+  sp_relion2sphire.py  'Particles.star'  'Sphire_Project'  --box_size=256  --do_not_create_stack
-  sxrelion2sphire.py  Refine3D/run1_data.star sphire_dir  --star_section=data_images  --box_size=100  --create_stack
+  sp_relion2sphire.py  'Refine3D/run1_data.star' 'Sphire_Project'  --star_section='data_images'  --box_size=100  --do_not_create_stack
+  sp_relion2sphire.py  'Relion_Project/Refine3D/run1_data.star'  'Sphire_Project'  --relion_project_dir='Relion_Project' --outputs_root='my_protein'  --do_not_create_stack
 \\
 ===== Input =====
 === Main Parameters ===
-  ; input_star_file : Input RELION STAR file: Specify a STAR file generated by RELION. The file should contain parameters related to Micrographs, CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset. Entries for Micrographs category are required. If it does not contain some entries associated with CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset, then the script does not produced the related output file(s). (default required string)
+  ; input_star_file : Input RELION STAR file: Specify a STAR file generated by RELION. The file should contain parameters related to Micrographs, CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset. Entries for Micrographs category are required. If some entries associated with CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset are missing, then the script does not produced the related output file(s). (default required string)
-  ; output_directory : Output directory: All the results will be written in here. This directory will be created automatically and it must not exist previously. (default required string)
+  ; output_directory : Output directory: All the results will be written in here. It cannot be an existing one. (default required string)
+  ; %%--%%relion_project_dir : RELION project directory: Path to RELION project directory associated with the RELION STAR file. By default, the program assume the current directory is the RELION project directory. (default none)
   ; %%--%%star_section : Section title in STAR file: The section title in the RELION star file where the data should be extracted. (default data_)
-  ; %%--%%box_size : Box size: Box size for particle extraction.  It also controls the saved coordinates file format. If the given value is > 0, store the eman1 format. coordinate file. The coordinates of eman1 format is particle box corner associated with this box size. The coordinates of sphire format is particle center. By default, use sphire format. (default 0)
+  ; %%--%%outputs_root : Root name of outputs: Specify the root name of all outputs. It cannot be empty string or only white spaces. (default sphire)
-  ; %%--%%create_stack : Create particle stack: By default, the program does not generate the stack of particle images because it takes a long time and the file size is large. (default False)
+  ; %%--%%box_size : Box size: Box size for particle extraction. It also controls the saved coordinates file format. If the given value is > 0, store the eman1 format. coordinate file. The coordinates of eman1 format is particle box corner associated with this box size. The coordinates of SPHIRE format is particle center. By default, use SPHIRE format. (default 0)
 \\
 === Advanced Parameters ===
+  ;%%--%%do_not_create_stack : Skip virtual stack creation: Create per-micrograph virtual stacks without the actual particle meta information in BDB format. By default, the program does generate the stack of particle meta information. (default False))
 \\
@@ Line 38: / Line 42: @@
 \\
 === List of Output Files ===
+The program copies the subdirectory structure under the RELION project directory based on the information stored in the input RELION STAR file.
+For example, if two sets of micrographs are stored in two different subdirectories (e.g. "EM_Session01" and "EM_Session02"  under a root micrograph directory (e.g. "Micrograph"), the program creates two subdirectories under the user-specified output directory (e.g. "Sphire_Project/EM_Session01" and "Sphire_Project/EM_Session02").
+Each output subdirectory will contain the files in the table below.
 || **File Name** || **Discription** ||
-|| sphire_micrographs.txt || Text file containing a list of micrograph names/paths (Micrograph selection file). It can be used as input of [[pipeline:cter:sxcter|sxcter]], [[pipeline:window:sxwindow|sxwindow]], [[pipeline:movie:sxunblur|sxunblur]], [[pipeline:movie:sxsummovie|sxsummovie]], and [[pipeline:utilities:sxpipe_organize_micrographs|sxpipe organize_micrographs]]. (requires Micrographs Category) ||
+|| *_micrographs.txt || Text file containing a list of micrograph names/paths (Micrograph selection file). It can be used as input of [[pipeline:cter:sxcter|sp_cter]], [[pipeline:window:sxwindow|sp_window]], [[pipeline:movie:sxunblur|sp_unblur]], [[pipeline:movie:sxsummovie|sp_summovie]], and [[pipeline:utilities:sxpipe_organize_micrographs|sp_pipe organize_micrographs]]. (requires Micrographs Category) ||
-|| sphire_cter_partres.txt || Text file containing a list of CTF parameters in CTER format (CTER partres file). It can be used as an input of [[pipeline:cter:sxgui_cter|sxgui_cter]] and [[pipeline:window:sxwindow|sxwindow]]. (requires CTF Estimation Category) ||
+|| *_cter_partres.txt || Text file containing a list of CTF parameters in CTER format (CTER partres file). It can be used as an input of [[pipeline:cter:sxgui_cter|sp_gui_cter]] and [[pipeline:window:sxwindow|sp_window]]. (requires CTF Estimation Category) ||
-|| sphire_stack_ctf.txt || Text file containing a list of CTF parameters. It can be used as an input of [[pipeline:utilities:sxheader|sxheader]] %%--%%params=ctf. (requires CTF Estimation Category) ||
+|| Coordinates/*.box || Coordinates files. One for each micrograph. These files can be used as inputs of [[pipeline:window:sxwindow|sp_window]]. (requires Particle Extraction Category) ||
-|| Coordinates/*.box || Coordinates files. One for each micrograph. It can be used as an input of [[pipeline:window:sxwindow|sxwindow]]. (requires Particle Extraction Category) ||
+|| Rebox/*.rbx || SPHIRE rebox files. One for each micrograph. These files can be used as inputs of [[pipeline:window:sxrewindow|sp_rewindow]]. (requires Particle Extraction Category) ||
-|| sphire_stack_proj3d.txt || Text file containing a list of 3D projection parameters. It can be used as an input of [[pipeline:utilities:sxheader|sxheader]] %%--%%params=xform.projection. (requires 3D Alignment Category) ||
+|| EMAN2DB/*_stack.bdb || Per-micrograph virtual stacks of particle meta information in bdb format (only when %%--%%do_not_create_stack is not used) containing the header entries associated with the extracted information. (requires Particle Extraction Category) ||
-|| sphire_stack_chunk*.txt || Text file containing a list of particle IDs for the subset (typically #0 and #1). It can be used as an input of [[pipeline:utilities:sxheader|sxheader]] %%--%%params=chunk_id. (requires Random Subset Category) ||
-|| sphire_stack.hdf || Particle image stack (only when %%--%%create_stack option is used) containing the header entries associated with the extracted information. (requires Particle Extraction Category) ||
 \\
 ===== Description =====
-Run the script to create several types of parameter text files and particle image stack in SPHIRE format from parameters stored in a RELION STAR file. The RELION STAR should contain entries related to Micrographs, CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset. The entries for Micrographs category are required. If it does not contain some parameters associated with CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset, then the command does not produce the related output file(s). Make sure the current directory is the project folder of the RELION (where you run RELION GUI).
+Run the script to create several types of parameter text files and particle image stack in SPHIRE format from parameters stored in a RELION STAR file. The RELION STAR should contain entries related to Micrographs, CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset. The entries for Micrographs category are required. If it does not contain some parameters associated with CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset, then the command does not produce the related output file(s).
 \\
@@ Line 64: / Line 70: @@
 \\
 === Related Header Entries ===
-The script adds or sets the following header entries of sphire_stack.hdf.
+The script sets the following header entries of EMAN2DB/*_stack.bdb or *_stack.hdf.
 || **Header Entry** || **Discription** ||
 || ptcl_source_relion || RELION's image name (_rlnImageName) which the script extracted this particle image. ||
 || ptcl_source_image || Micrograph path (relative path) where RELION extracted this particle image. ||
-|| ptcl_source_coord || Coordinates of this particle in the Micrograph in sphire format. ||
+|| ptcl_source_coord || Coordinates of this particle in the Micrograph in SPHIRE format. ||
 || ptcl_source_coord_id || Coordinates ID in the original coordinate file RELION used. ||
+|| data_path || Relative path to the original particle image location. ||
 || data_n || Serial particle image ID.  Here, it is same as ptcl_source_coord_id. ||
 || resample_ratio || Resampling ratio of dimensions or pixel size of extracted particle images relative to micrograph. Here, it is always set to 1.0. ||
@@ Line 76: / Line 83: @@
 || ptcl_source_apix || Pixel size of the associated micrograph. ||
 || xform.projection || 3D projection parameters. ||
+|| relion_max_prob_dist || Maximum probability distribution value computed by RELION. ||
+|| relion_norm_correct || Normalisation correction value computed by RELION. ||
 || chunk_id || Chunk ID (or Subset ID) where this particle belongs. ||
-NOTE: If you run [[pipeline:window:sxwindow|sxwindow]] using the coordinates files generated by this script, ptcl_source_coord_id will be reassigned. It can be different from the RELION's local particle ID (the number before '@' in ptcl_source_relion).
+NOTE: If you run [[pipeline:window:sp_window|sp_window]] using the coordinates files generated by this script, ptcl_source_coord_id will be reassigned. It can be different from the RELION's local particle ID (the number before '@' in ptcl_source_relion).
 \\
@@ Line 85: / Line 94: @@
 \\
 ==== Reference ====
+\\
+==== Developer Notes ====
+=== 2018/09/17 Markus Stabrin ===
+  - Remove image stack creation.
+  - Created stacks only contain meta data.
+  - Write bdb meta data stacks by default (Change %%--%%create_stack to %%--%%do_not_create_stack).
+  - Remove %%--%%cs_save_as_hdf option.
+=== 2018/07/11 Toshio Moriya ===
+  - Change default stack format from HDF to bdb (%%--%%cs_save_as_hdf).
+    * By default, the script creates a local particle image stack for each micrograph like sp_window.py.
+    * Likewise, because of synchronisation problem of subprocess execution, user must run particle stack command to create a virtual stack of all particles.
+  - Accept arbitrary output file name (%%--%%outputs_root).
+  - Added SPHIRE rebox file output support. The rebox file contains box coordinates, CTF parameters, and projection parameters.
+    * This modification solves the problem associated with per-particle defocus estimation.
+  - Add relion project directory option so that the script can be run from any directory (%%--%%relion_project_dir).
+    * That is, you don't anymore need to make sure the current directory is the project folder of the RELION (where you run RELION GUI).
+    * The program appends the path of relion project directory to micrograph path extracted from the star file.
+These changes require user not to break the relative path whenever they move or copy the micrographs because sp_relion2sparx.py use full name including the path stored in the input RELION STAR file.
+The program does not generate the following output files anymore.
+|| **File Name** || **Discription** ||
+|| sphire_stack_ctf.txt || Text file containing a list of CTF parameters. It can be used as an input of [[pipeline:utilities:sp_header|sp_header]] %%--%%params=ctf. (requires CTF Estimation Category) ||
+|| sphire_stack_proj3d.txt || Text file containing a list of 3D projection parameters. It can be used as an input of [[pipeline:utilities:sp_header|sp_header]] %%--%%params=xform.projection. (requires 3D Alignment Category) ||
+|| sphire_stack_chunk*.txt || Text file containing a list of particle IDs for the subset (#0 and #1). It can be used as an input of [[pipeline:utilities:sp_header|sp_header]] %%--%%params=chunk_id. (requires Random Subset Category) ||
+To create these files, please use [[pipeline:utilities:sp_header|sp_header]] to extract information from the output stack or rewindowed stack with the output SPHIRE rebox files using [[pipeline:window:sp_rewindow|sp_rewindow]].
 \\
 ==== Author / Maintainer ====
 Toshio Moriya
+Markus Stabrin
 \\
@@ Line 96: / Line 138: @@
 \\
 ==== Files ====
-sparx/bin/sxrelion2sphire.py
+sparx/bin/sp_relion2sphire.py
 \\
 ==== See also ====
-[[pipeline:cter:sxcter|sxcter]], [[pipeline:cter:sxgui_cter|sxgui_cter]], [[pipeline:window:sxwindow|sxwindow]], [[pipeline:utilities:sxheader|sxheader]], [[pipeline:utilities:sxpipe_organize_micrographs|sxpipe organize_micrographs]], [[pipeline:movie:sxunblur|sxunblur]], and [[pipeline:movie:sxsummovie|sxsummovie]].
+[[pipeline:cter:sp_cter|sp_cter]], [[pipeline:cter:sp_gui_cter|sp_gui_cter]], [[pipeline:window:sp_window|sp_window]], [[pipeline:window:sp_rewindow|sp_rewindow]], [[pipeline:utilities:sp_header|sp_header]], [[pipeline:utilities:sp_pipe_organize_micrographs|sp_pipe organize_micrographs]], [[pipeline:movie:sp_unblur|sp_unblur]], and [[pipeline:movie:sp_summovie|sp_summovie]].
 \\
@@ Line 111: / Line 153: @@
 \\

User Tools

Site Tools

Differences

Page Tools