User Tools

Site Tools


pipeline:utilities:sxrelion2sphire
The most recent version of this page is a draft.DiffThis version (2018/08/22 12:02) is a draft.
Approvals: 0/1

This is an old revision of the document!


sxrelion2sphire

RELION to SPHIRE Conversion: Create several types of parameter text files and per-micrograph virtual stacks of particle images in bdb format from parameters stored in a RELION STAR file


Usage

Usage in command line

sxrelion2sphire.py  input_star_file  output_directory  --relion_project_dir=DIR_PATH  --star_section=STAR_SECTION  --outputs_root=ROOT_NAME_STRING   --box_size=BOX_SIZE  --create_stack  --cs_save_as_hdf


Typical usage

sxrelion2sphire does not support MPI.

sxrelion2sphire.py  'Particles.star'  'Sphire_Project'  --box_size=256  --create_stack

sxrelion2sphire.py  'Refine3D/run1_data.star' 'Sphire_Project'  --star_section='data_images'  --box_size=100  --create_stack

sxrelion2sphire.py  'Relion_Project/Refine3D/run1_data.star'  'Sphire_Project'  --relion_project_dir='Relion_Project' --outputs_root='my_protein'  --create_stack


Input

Main Parameters

input_star_file
Input RELION STAR file: Specify a STAR file generated by RELION. The file should contain parameters related to Micrographs, CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset. Entries for Micrographs category are required. If some entries associated with CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset are missing, then the script does not produced the related output file(s). (default required string)
output_directory
Output directory: All the results will be written in here. It cannot be an existing one. (default required string)
--relion_project_dir
RELION project directory: Path to RELION project directory associated with the RELION STAR file. By default, the program assume the current directory is the RELION project directory. (default none)
--star_section
Section title in STAR file: The section title in the RELION star file where the data should be extracted. (default data_)
--outputs_root
Root name of outputs: Specify the root name of all outputs. It cannot be empty string or only white spaces. (default sphire)
--box_size
Box size: Box size for particle extraction. It also controls the saved coordinates file format. If the given value is > 0, store the eman1 format. coordinate file. The coordinates of eman1 format is particle box corner associated with this box size. The coordinates of SPHIRE format is particle center. By default, use SPHIRE format. (default 0)
--create_stack
Create virtual stacks: Create per-micrograph virtual stacks of particle images in bdb format. By default, the program does not generate the stack of particle images because it takes a long time and the file size is large. (default False)


Advanced Parameters

--cs_save_as_hdf
Save stack as HDF file: Save a stack file in HDF file format instead in bdb format. In this case, the stack file will contain all particle images in the input RELION STAR file. Effective only with --create_stack. (default False)


Output


List of Output Files

The program copies the subdirectory structure under the RELION project directory based on the information stored in the input RELION STAR file. For example, if two sets of micrographs are stored in two different subdirectories (e.g. “EM_Session01” and “EM_Session02” under a root micrograph directory (e.g. “Micrograph”), the program creates two subdirectories under the user-specified output directory (e.g. “Sphire_Project/EM_Session01” and “Sphire_Project/EM_Session02”). Each output subdirectory will contain the files in the table below.

File Name Discription
*_micrographs.txt Text file containing a list of micrograph names/paths (Micrograph selection file). It can be used as input of sxcter, sxwindow, sxunblur, sxsummovie, and sxpipe organize_micrographs. (requires Micrographs Category)
*_cter_partres.txt Text file containing a list of CTF parameters in CTER format (CTER partres file). It can be used as an input of sxgui_cter and sxwindow. (requires CTF Estimation Category)
Coordinates/*.box Coordinates files. One for each micrograph. These files can be used as inputs of sxwindow. (requires Particle Extraction Category)
Rebox/*.rbx SPHIRE rebox files. One for each micrograph. These files can be used as inputs of sxrewindow. (requires Particle Extraction Category)
EMAN2DB/*_stack.bdb Per-micrograph virtual stacks of particle images in bdb format (only when --create_stack option without --cs_save_as_hdf is used) containing the header entries associated with the extracted information. (requires Particle Extraction Category)
*_stack.hdf Stack of all particle images in HDF format (only when --create_stack option with --cs_save_as_hdf is used) containing the header entries associated with the extracted information. (requires Particle Extraction Category)


Description

Run the script to create several types of parameter text files and particle image stack in SPHIRE format from parameters stored in a RELION STAR file. The RELION STAR should contain entries related to Micrographs, CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset. The entries for Micrographs category are required. If it does not contain some parameters associated with CTF Estimation, Particle Extraction, 3D Alignment, or/and Random Subset, then the command does not produce the related output file(s).


RELION STAR Entries

The following RELION STAR file entries are necessary to extract information for certain categories. The entries can exist in the file but won't be used by this command.

Category Required entries Optional entries Dependent category
Micrographs (Required) _rlnMicrographName None None
CTF Estimation _rlnDefocusU
_rlnDefocusV
_rlnDefocusAngle
_rlnAmplitudeContrast
_rlnVoltage
_rlnSphericalAberration
_rlnMagnification
_rlnPhaseShift
_rlnCtfFigureOfMerit
_rlnCtfMaxResolution
_rlnCtfImage
Micrographs
Particle Extraction _rlnImageName
_rlnCoordinateX
_rlnCoordinateY
None Micrographs
3D Alignment _rlnOriginX
_rlnOriginY
_rlnAngleRot
_rlnAngleTilt
_rlnAnglePsi
_rlnMaxValueProbDistribution
_rlnNormCorrection
None None
Random Subset _rlnRandomSubset None None


The script sets the following header entries of EMAN2DB/*_stack.bdb or *_stack.hdf.

Header Entry Discription
ptcl_source_relion RELION's image name (_rlnImageName) which the script extracted this particle image.
ptcl_source_image Micrograph path (relative path) where RELION extracted this particle image.
ptcl_source_coord Coordinates of this particle in the Micrograph in SPHIRE format.
ptcl_source_coord_id Coordinates ID in the original coordinate file RELION used.
data_n Serial particle image ID. Here, it is same as ptcl_source_coord_id.
resample_ratio Resampling ratio of dimensions or pixel size of extracted particle images relative to micrograph. Here, it is always set to 1.0.
ctf CTF parameters.
ctf_applied Flag indicating if CTF parameters are applied to image or not.
ptcl_source_apix Pixel size of the associated micrograph.
xform.projection 3D projection parameters.
relion_max_prob_dist Maximum probability distribution value computed by RELION.
relion_norm_correct Normalisation correction value computed by RELION.
chunk_id Chunk ID (or Subset ID) where this particle belongs.

NOTE: If you run sxwindow using the coordinates files generated by this script, ptcl_source_coord_id will be reassigned. It can be different from the RELION's local particle ID (the number before '@' in ptcl_source_relion).


Method


Reference


Developer Notes

2018/07/11 Toshio Moriya

  1. Change default stack format from HDF to bdb (--cs_save_as_hdf).
    • By default, the script creates a local particle image stack for each micrograph like sxwindow.py.
    • Likewise, because of synchronisation problem of subprocess execution, user must run particle stack command to create a virtual stack of all particles.
  2. Accept arbitrary output file name (--outputs_root).
  1. Added SPHIRE rebox file output support. The rebox file contains box coordinates, CTF parameters, and projection parameters.
    • This modification solves the problem associated with per-particle defocus estimation.
  1. Add relion project directory option so that the script can be run from any directory (--relion_project_dir).
  • That is, you don't anymore need to make sure the current directory is the project folder of the RELION (where you run RELION GUI).
  • The program appends the path of relion project directory to micrograph path extracted from the star file.

These changes require user not to break the relative path whenever they move or copy the micrographs because sxrelion2sparx.py use full name including the path stored in the input RELION STAR file.

The program does not generate the following output files anymore.

File Name Discription
sphire_stack_ctf.txt Text file containing a list of CTF parameters. It can be used as an input of sxheader --params=ctf. (requires CTF Estimation Category)
sphire_stack_proj3d.txt Text file containing a list of 3D projection parameters. It can be used as an input of sxheader --params=xform.projection. (requires 3D Alignment Category)
sphire_stack_chunk*.txt Text file containing a list of particle IDs for the subset (#0 and #1). It can be used as an input of sxheader --params=chunk_id. (requires Random Subset Category)

To create these files, please use sxheader to extract information from the output stack or rewindowed stack with the output SPHIRE rebox files using sxrewindow.


Author / Maintainer

Toshio Moriya


Keywords

Category 1:: APPLICATIONS


Files

sparx/bin/sxrelion2sphire.py


See also

Maturity

Beta:: Under evaluation and testing. Please let us know if there are any bugs.


Bugs

There are no known bugs so far.


pipeline/utilities/sxrelion2sphire.1534932155.txt.gz · Last modified: 2018/08/22 12:02 by fmerino