This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
pipeline:isac:sxisac2 [2018/02/21 16:59] moriya |
pipeline:isac:sxisac2 [2020/08/19 16:40] (current) fschoenfeld |
||
---|---|---|---|
Line 1: | Line 1: | ||
~~NOTOC~~ | ~~NOTOC~~ | ||
- | ===== sxisac2 | + | ===== sp_isac2 |
ISAC2 - 2D Clustering: Iterative Stable Alignment and Clustering (ISAC) of a 2D image stack. | ISAC2 - 2D Clustering: Iterative Stable Alignment and Clustering (ISAC) of a 2D image stack. | ||
Line 9: | Line 9: | ||
Usage in command line | Usage in command line | ||
- | | + | |
\\ | \\ | ||
===== Typical usage ===== | ===== Typical usage ===== | ||
- | sxisac2 | + | sp_isac2 |
\\ __1. Conventional TEM dataset__: | \\ __1. Conventional TEM dataset__: | ||
- | mpirun -np 96 sxisac2.py bdb:stack isac2_outdir --radius=120 --CTF | + | mpirun -np 96 sp_isac2.py bdb:stack isac2_outdir --radius=120 --CTF |
Note: ISAC2 will change the size of input data such that they fit into box size 76x76 by default (see Description below). | Note: ISAC2 will change the size of input data such that they fit into box size 76x76 by default (see Description below). | ||
Line 24: | Line 24: | ||
\\ __2. Phase Plate TEM dataset__: | \\ __2. Phase Plate TEM dataset__: | ||
- | mpirun -np 96 sxisac2.py bdb:stack isac2_outdir --radius=120 --VPP | + | mpirun -np 96 sp_isac2.py bdb:stack isac2_outdir --radius=120 --VPP |
Note: CTF and VPP options cannot be used together. | Note: CTF and VPP options cannot be used together. | ||
Line 31: | Line 31: | ||
===== Input ===== | ===== Input ===== | ||
=== Main Parameters === | === Main Parameters === | ||
- | ; stack_file : Input image stack: | + | ; stack_file : Input image stack: |
- | ; output_directory : Output directory: | + | ; output_directory : Output directory: |
- | ; %%--%%radius : Particle radius [Pixels]: Radius of the particle | + | ; %%--%%radius : Particle radius [Pixels] : Radius of the particle |
- | ; %%--%%img_per_grp : Images per class: | + | ; %%--%%img_per_grp : Images per class: |
- | ; %%--%%CTF : CTF phase flipping: If set, the data will be phase-flipped using CTF information included in the image headers. (default False) | + | ; %%--%%CTF : CTF phase flipping : Use for cryo datasets. |
- | ; %%--%%xr : Translation search range [Pixels]: The translational search range. Set by the program by default. (default 1) | + | ; %%--%%VPP : Phase Plate data : Use this option if the dataset is taken with phase plate. Cannot be used togehter with the CTF option. (default |
- | ; %%--%%thld_err | + | ; %%--%%filament_width |
- | ; %%--%%target_radius : Target particle radius [Pixels]: Particle radius used by ISAC2 to process the data. The images will be resized to fit this radius (default 29) | + | |
- | ; %%--%%target_nx | + | |
\\ | \\ | ||
=== Advanced Parameters === | === Advanced Parameters === | ||
- | ; %%--%%VPP : Phase Plate data: Please use this option if the dataset | + | ; %%--%%xr : Translation search range [Pixels] |
- | ; %%--%%ir : Inner ring [Pixels]: | + | ; %%--%%thld_err : Pixel error threshold [Pixels]: Used as a threshold value when checking cluster stability. The pixel error is defined as the root mean square of distances between corresponding pixels from set of found transformations and their average transformation; |
- | ; %%--%%rs : Ring step [Pixels]: | + | ; %%--%%target_radius : Target particle radius [Pixels] : Particle radius used by ISAC2 to process the data. All particle images will be re-scaled to match their particle radius |
- | ; %%--%%yr : Y search range [Pixels]: The translational search range in the y direction. Set as //xr// by default. (default -1) | + | ; %%--%%target_nx : Target particle image size [Pixels] : Image size used by ISAC2 to process the data. particle images will first be resized according to target particle radius (see above) and then cropped or padded to achieve the target image size. When //xr// > 0, the final image size for ISAC2 processing is // |
- | ; %%--%%ts : Search step [Pixels]: Translational search step. (default 1.0) | + | ; %%--%%ir : Inner ring [Pixels] : Radius |
- | ; %%--%%maxit : Reference-free alignment iterations: The number of iterations for reference-free | + | ; %%--%%rs : Ring step [Pixels] : Radius step size when resampling |
- | ; %%--%%center_method : Centering method: Method to center global 2D average during the initial prealignment of the data (0: no centering; -1: average shift method; please see center_2D in utilities.py for methods 1-7). (default -1) | + | ; %%--%%yr : Y search range [Pixels] : The translational search range in the y direction. Set to the value of //xr// by default. (default -1) |
- | ; %%--%%dst : Discrete angle used for within-group alignment: Discrete angle used for within-group alignment. (default 90.0) | + | ; %%--%%ts : Search step [Pixels] : Translational search step. (default 1.0) |
- | ; %%--%%FL : Lowest filter frequency [1/Pixel]: Lowest frequency used for the tangent filter. (default 0.2) | + | ; %%--%%maxit : Reference-free alignment iterations: The number of iterations for reference-free |
- | ; %%--%%FH : Highest filter frequency [1/Pixel]: Highest frequency used for the tangent filter. (default 0.45) | + | ; %%--%%center_method : Centering method : Method to center global 2D average during the initial prealignment of the data (0: no centering; -1: average shift method; please see center_2D in utilities.py for methods 1-7). (default -1) |
- | ; %%--%%FF : Tangent filter fall-off: The fall-off of the tangent filter. (default 0.2) | + | ; %%--%%dst : Discrete angle used for within-group alignment : Discrete angle used for within-group alignment. (default 90.0) |
- | ; %%--%%init_iter : Maximum generations: | + | ; %%--%%FL : Lowest filter frequency [1/Pixel] : Lowest frequency used for the tangent filter. (default 0.2) |
- | ; %%--%%iter_reali : SAC stability check interval: Defines every how many iterations the SAC stability checking is performed. (default 1) | + | ; %%--%%FH : Highest filter frequency [1/Pixel] : Highest frequency used for the tangent filter. (default 0.45) |
- | ; %%--%%stab_ali : Number of alignments for stability check: The number of alignments | + | ; %%--%%FF : Tangent filter fall-off : The fall-off of the tangent filter. (default 0.2) |
- | ; %%--%%minimum_grp_size : Minimum size of reproducible | + | ; %%--%%init_iter : Maximum generations : Maximum number of generation iterations performed for a given subset. (default 7) |
+ | ; %%--%%iter_reali : SAC stability check interval : Defines every how many iterations the SAC stability checking is performed. (default 1) | ||
+ | ; %%--%%stab_ali : Number of alignments for stability check : The number of alignment runs when checking stability. (default 5) | ||
+ | ; %%--%%minimum_grp_size : Minimum size of reproducible | ||
; %%--%%rand_seed : Seed: Random seed set before calculations. Useful for testing purposes. By default, ISAC2 sets a random seed number. (default none) | ; %%--%%rand_seed : Seed: Random seed set before calculations. Useful for testing purposes. By default, ISAC2 sets a random seed number. (default none) | ||
; %%--%%skip_prealignment : Do pre-alignment: | ; %%--%%skip_prealignment : Do pre-alignment: | ||
; %%--%%restart : Restart run: 0: Restart ISAC2 after the last completed main iteration (i.e. the directory must contain // | ; %%--%%restart : Restart run: 0: Restart ISAC2 after the last completed main iteration (i.e. the directory must contain // | ||
+ | ; %%--%%filament_mask_ignore : Ignore filament masking (filament use only): ONLY RELEVANT IF parameter filament_width is set to a non-default value. When processing helical particle images rectangular masking is used (A) to normalize and (B) to mask the actual particle images. The latter can be disabled by setting this flag to True. (default True question reversed in GUI) | ||
\\ | \\ | ||
===== Output ===== | ===== Output ===== | ||
- | For each generation of running the program, there are two phases. The first phase is an exploratory phase. In this phase, we set the criteria to be very loose and try to find as much candidate class averages as possible. (OBSOLETE? This phase typically should have 10 to 20 rounds). The candidate class averages are stored in class_averages_candidate_generation_n.hdf. | + | For each generation of running the program, there are two phases. The first phase is an exploratory phase. In this phase, we set the criteria to be very loose and try to find as much candidate class averages as possible. The candidate class averages are stored in class_averages_candidate_generation_n.hdf. |
- | The second phase is where the actual class averages are generated | + | The second phase is where the actual class averages are generated. The candidate groups |
\\ | \\ | ||
Line 90: | Line 92: | ||
- The alignment shift parameters will be applied to the input data. | - The alignment shift parameters will be applied to the input data. | ||
- **IMPORTANT**: | - **IMPORTANT**: | ||
- | * The pseudo-code for adjusting the size of the radius and the size of the images is as follows: | + | * The pseudo-code for adjusting the size of the radius and the size of the images is as follows: |
- | | + | shrink_ratio |
- | | + | new_pixel_size = original_pixel_size |
- | * if shrink_ratio | + | new_box_size |
- | * if new_pixel_size | + | IF shrink_ratio =/= 1.0 THEN: resample images using shrink_ratio |
- | * if new_pixel_size | + | IF new_box_size |
+ | IF new_box_size | ||
* The target_radius and target_nx options allow the user to finely adjust the image so that it contains enough background information. | * The target_radius and target_nx options allow the user to finely adjust the image so that it contains enough background information. | ||
- The program will iterate through generations of ISAC2 by alternating two steps. The outcome of these two steps is in subdirectory generation_*** (stars replaced by the current generation number). | - The program will iterate through generations of ISAC2 by alternating two steps. The outcome of these two steps is in subdirectory generation_*** (stars replaced by the current generation number). | ||
Line 108: | Line 111: | ||
==== Time and Memory ==== | ==== Time and Memory ==== | ||
- | Unfortunately, | + | ISAC2 is very time- and memory-consuming. |
\\ | \\ | ||
Line 115: | Line 118: | ||
- Delete averages whose member particles should not be included in the output. | - Delete averages whose member particles should not be included in the output. | ||
- Save the selected subset under a new name,say select1.hdf | - Save the selected subset under a new name,say select1.hdf | ||
- | - Retrieve IDs of member particles and store them in a text file ohk.txt: | + | - Retrieve IDs of member particles and store them in a text file ok.txt: |
- | * $ sxprocess.py --isacselect class_averages.hdf ok.txt | + | * $ sp_process.py --isacselect class_averages.hdf ok.txt |
- Create a vritual stack containng selected particles: | - Create a vritual stack containng selected particles: | ||
- | * $ e2bdb.py bdb:data --makevstack: | + | * $ e2bdb.py bdb:data --makevstack: |
The same steps can be performed on files containing candidate class averages. | The same steps can be performed on files containing candidate class averages. | ||
Line 128: | Line 131: | ||
- Retrieve original image numbers in the selected ISAC2 group. | - Retrieve original image numbers in the selected ISAC2 group. | ||
* $ cd generation_0003 | * $ cd generation_0003 | ||
- | * $ sxprocess.py bdb:../data class_averages_generation_3.hdf | + | * $ sp_process.py bdb:../data class_averages_generation_3.hdf |
- Extract the identified images from the main stack (into subdirectory RCT, has to be created): | - Extract the identified images from the main stack (into subdirectory RCT, has to be created): | ||
* $ e2bdb.py bdb: | * $ e2bdb.py bdb: | ||
Line 134: | Line 137: | ||
* $ e2proc2d.py --split=12 --first=12 --last=12 class_averages_generation3.hdf | * $ e2proc2d.py --split=12 --first=12 --last=12 class_averages_generation3.hdf | ||
- Align particles using the corresponding class average from ISAC2 as a template (please adjust the parameters): | - Align particles using the corresponding class average from ISAC2 as a template (please adjust the parameters): | ||
- | * $ sxali2d.py bdb: | + | * $ sp_ali2d.py bdb: |
- | - Extract the needed alignment parameters. | + | - Extract the needed alignment parameters. |
- | * $ sxheader.py group3_12.12.hdf | + | * $ sp_header.py group3_12.12.hdf |
\\ | \\ | ||
Line 160: | Line 163: | ||
\\ | \\ | ||
==== Files ==== | ==== Files ==== | ||
- | sparx/bin/sxisac2.py, sparx/bin/sxisac.py, sparx/ | + | sparx/bin/sp_isac2.py, sparx/bin/sp_isac.py, sparx/ |
\\ | \\ | ||
==== See also ==== | ==== See also ==== | ||
- | [[pipeline: | + | [[pipeline: |
\\ | \\ |