User Tools

Site Tools


pipeline:utilities:sxpipe_moon_eliminator

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Last revision Both sides next revision
pipeline:utilities:sxpipe_moon_eliminator [2018/08/22 11:15]
fmerino
pipeline:utilities:sxpipe_moon_eliminator [2019/04/02 09:30]
lusnig
Line 1: Line 1:
 ~~NOTOC~~ ~~NOTOC~~
  
-===== sxpipe moon_eliminator ===== +===== sp_pipe moon_eliminator ===== 
-Moon eliminator: Eliminate moons or remove dusts from the background of a 3D density map based on the expected molecular mass. Optionally, create 3D mask also.+Volume Adjustment: Eliminate moons or remove dusts from the background of a 3D density map based on the expected molecular mass.
  
 \\ \\
Line 9: Line 9:
 Usage in command line Usage in command line
  
-  sxpipe.py  moon_eliminator  input_volume_path  input_volume_path_2nd  output_directory  --pixel_size=PIXEL_SIZE  --mol_mass=KILODALTON  --use_density_threshold=THRESHOLD  --moon_distance=PIXEL_DISTANCE  --dilation=DILATION_PIXEL_WIDTH  --edge_sigma=SIGMA_PIXEL_WIDTH  --resample_ratio=RATIO_OR_DIR_PATH  --box_size=BOX_SIZE  --resampled_shift3d  --shift3d_x=SHIFT3D_X  --shift3d_y=SHIFT3D_Y  --shift3d_z=SHIFT3D_Z  --invert_handedness  --fl=LPF_CUTOFF_FREQ  --aa=LPF_FALLOFF_WIDTH  --generate_mask  --gm_dilation=DILATION_PIXEL_WIDTH  --gm_edge_width=MASK_EDGE_PIXEL_WIDTH  --gm_edge_sigma=SIGMA_PIXEL_WIDTH  --outputs_root=FILE_ROOT  --allow_disconnect  --edge_type=SOFT_EDGE_TYPE  --debug+  sp_pipe.py  moon_eliminator  input_volume_path  input_volume_path_2nd  output_directory  --pixel_size=PIXEL_SIZE  --mol_mass=KILODALTON  --use_density_threshold=THRESHOLD  --moon_distance=PIXEL_DISTANCE  --ndilation=DILATION_PIXEL_WIDTH  --edge_width=SIGMA_PIXEL_WIDTH  --resample_ratio=RATIO_OR_DIR_PATH  --box_size=BOX_SIZE  --resampled_shift3d  --shift3d_x=SHIFT3D_X  --shift3d_y=SHIFT3D_Y  --shift3d_z=SHIFT3D_Z  --invert_handedness  --fl=LPF_CUTOFF_FREQ  --aa=LPF_FALLOFF_WIDTH   --outputs_root=FILE_ROOT   --edge_type=SOFT_EDGE_TYPE  --debug
    
 \\ \\
 ===== Typical usage ===== ===== Typical usage =====
  
-sxpipe moon_eliminator does not support MPI. +sp_pipe moon_eliminator does not support MPI. 
  
 There are two modes to run the program: There are two modes to run the program:
Line 23: Line 23:
 Create reference 3D structure and 3D mask from R-VIPER 3D model with the resample ratio used in ISAC2 using an expected molecular mass [kDa]. Create reference 3D structure and 3D mask from R-VIPER 3D model with the resample ratio used in ISAC2 using an expected molecular mass [kDa].
  
-  sxpipe.py moon_eliminator 'outdir_rviper/main001/average_volume.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12 --resample_ratio='outdir_isac2' --box_size=352 --generate_mask +  sp_pipe.py moon_eliminator 'outdir_rviper/main001/average_volume.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12 --resample_ratio='outdir_isac2' --box_size=352
  
 Create reference 3D structure and 3D mask from R-VIPER 3D model with the resample ratio used in ISAC2 using ad-hoc density threshold instead of the expected molecular mass [kDa]. Create reference 3D structure and 3D mask from R-VIPER 3D model with the resample ratio used in ISAC2 using ad-hoc density threshold instead of the expected molecular mass [kDa].
  
-  sxpipe.py moon_eliminator 'outdir_rviper/main001/average_volume.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12 --use_density_threshold=13.2 --resample_ratio='outdir_isac2' --box_size=352 --generate_mask +  sp_pipe.py moon_eliminator 'outdir_rviper/main001/average_volume.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12 --use_density_threshold=13.2 --resample_ratio='outdir_isac2' --box_size=352
  
 Create reference 3D structure and 3D mask from post-refined MERIDIEN 3D model using the expected molecular mass [kDa]. Create reference 3D structure and 3D mask from post-refined MERIDIEN 3D model using the expected molecular mass [kDa].
  
-  sxpipe.py moon_eliminator 'outdir_postrefiner/postrefine3d.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12 --generate_mask +  sp_pipe.py moon_eliminator 'outdir_postrefiner/postrefine3d.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12
  
 \\ \\
Line 38: Line 38:
 Create reference 3D structure and 3D mask from halfset unfiltered maps produced by MERIDIEN, using the expected molecular mass [kDa]. Create reference 3D structure and 3D mask from halfset unfiltered maps produced by MERIDIEN, using the expected molecular mass [kDa].
  
-  sxpipe.py moon_eliminator 'outdir_meridien/vol_0_unfil_025.hdf' 'outdir_meridien/vol_0_unfil_025.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12 --generate_mask +  sp_pipe.py moon_eliminator 'outdir_meridien/vol_0_unfil_025.hdf' 'outdir_meridien/vol_0_unfil_025.hdf' 'outdir_pipe_moon_eliminator' --mol_mass=1400 --pixel_size=1.12
  
  
Line 45: Line 45:
 === Main Parameters === === Main Parameters ===
   ; input_volume_path : Input volume path: Path to input volume file containing the 3D density map. (default required string)   ; input_volume_path : Input volume path: Path to input volume file containing the 3D density map. (default required string)
-  ; input_volume_path_2nd : Second input volume path: Path to second input volume file containing the 3D density map. Use this option to create a mask from the sum of two MERIDIEN half-set maps. (default none) 
   ; output_directory : Output directory: The results will be written here. It cannot be an existing one. (default required string)   ; output_directory : Output directory: The results will be written here. It cannot be an existing one. (default required string)
  
   ; %%--%%pixel_size : Output pixel size [A]: The original pixel size of dataset. This must be the pixel size after resampling when resample_ratio != 1.0. That is, it will be the pixel size of the output map. (default required float)   ; %%--%%pixel_size : Output pixel size [A]: The original pixel size of dataset. This must be the pixel size after resampling when resample_ratio != 1.0. That is, it will be the pixel size of the output map. (default required float)
-  ; %%--%%mol_mass : Molecular mass [kDa]: The estimated molecular mass of the target particle in kilodalton. (default required float) +  ; %%--%%use_mol_mass : Use molecular mass : GUI OPTION ONLY - Define if one want to use the molecular mass option as a masking threshold. (default False) : %%--%%use_density_threshold==none 
-  ; %%--%%use_density_threshold : Use ad-hoc density threshold: Use user-provided ad-hoc density threshold, instead of computing the value from the molecular mass. Below this density value, the data is assumed not to belong to the main body of the particle density. (default none)+  ; %%--%%mol_mass : Molecular mass [kDa]: The estimated molecular mass of the target particle in kilodalton. (default required float) : %%--%%use_mol_mass==True 
 +  ; %%--%%use_density_threshold : Use ad-hoc density threshold: Use user-provided ad-hoc density threshold, instead of computing the value from the molecular mass. Below this density value, the data is assumed not to belong to the main body of the particle density. (default none) : %%--%%use_mol_mass==False
   ; %%--%%moon_distance : Distance to the nearest moon [Pixels]: The moons further than this distance from the density surface will be elminated. The value smaller than the default is not recommended because it is difficult to avoid the stair-like gray level change at the edge of the density surface. (default 3.0)   ; %%--%%moon_distance : Distance to the nearest moon [Pixels]: The moons further than this distance from the density surface will be elminated. The value smaller than the default is not recommended because it is difficult to avoid the stair-like gray level change at the edge of the density surface. (default 3.0)
-  ; %%--%%dilation : Dilation width [Pixels]: The pixel width to dilate the 3D binary map corresponding to the specified molecular mass or density threshold prior to softening the edge. By default, it is set to half of %%--%%moon_distance so that the voxels with 1.0 values in the mask are same as the hard-edged molecular-mass binary map. (default -1.0) 
-  ; %%--%%edge_sigma : Edge sigma [Pixels]: The Gaussian sigma of transition area for soft-edge of the moon eliminator 3D mask. This value controls the falloff speed of soft-edge; The samller the sigma, the faster the falloff. Effective only with %%--%%edge_type='gauss'. (default 1.0) 
   ; %%--%%resample_ratio : Resample ratio: Specify a value larger than 0.0. By default, the program does not resample the input map (i.e. resample ratio is 1.0). Use this option maily to restore the original dimensions or pixel size of VIPER or R-VIPER model. Alternatively, specify the path to the output directory of an ISAC2 run. The program automatically extracts the resampling ratio used by the ISAC2 run. (default '1.0')   ; %%--%%resample_ratio : Resample ratio: Specify a value larger than 0.0. By default, the program does not resample the input map (i.e. resample ratio is 1.0). Use this option maily to restore the original dimensions or pixel size of VIPER or R-VIPER model. Alternatively, specify the path to the output directory of an ISAC2 run. The program automatically extracts the resampling ratio used by the ISAC2 run. (default '1.0')
   ; %%--%%box_size : Output box size [Pixels]: The x, y, and z dimensions of cubic area to be windowed from input 3D volume for output 3D volumes. This must be the box size after resampling when resample_ratio != 1.0. (default none)   ; %%--%%box_size : Output box size [Pixels]: The x, y, and z dimensions of cubic area to be windowed from input 3D volume for output 3D volumes. This must be the box size after resampling when resample_ratio != 1.0. (default none)
-  ; %%--%%resampled_shift3d : Providing resampled 3D shifts: Use this option when you are providing the resampled 3D shifts (using pixel size of outputs) when %%--%%resample_ratio!=1.0. By default, the program assums the provided shifts are not resampled. (default False) 
-  ; %%--%%shift3d_x : 3D x-shift [Pixels]: 3D x-shift value. (default 0) 
-  ; %%--%%shift3d_y : 3D y-shift [Pixels]: 3D y-shift value. (default 0) 
-  ; %%--%%shift3d_z : 3D z-shift [Pixels]: 3D z-shift value. (default 0) 
   ; %%--%%invert_handedness : Invert handedness: Invert the handedness of the 3D map. (default False)   ; %%--%%invert_handedness : Invert handedness: Invert the handedness of the 3D map. (default False)
   ; %%--%%fl : Low-pass filter resolution [A]: >0.0: low-pass filter to the value in Angstrom; =-1.0: no low-pass filter. The program applies this low-pass filter before the moon elimination. (default -1.0)   ; %%--%%fl : Low-pass filter resolution [A]: >0.0: low-pass filter to the value in Angstrom; =-1.0: no low-pass filter. The program applies this low-pass filter before the moon elimination. (default -1.0)
-  ; %%--%%aa : Low-pass filter fall-off [1/Pixels]: Low-pass filter fall-off in absolute frequency. The program applies this low-pass filter before the moon elimination. Effective only when %%--%%fl > 0.0. (default 0.1) 
-  ; %%--%%generate_mask : Generate soft-edged mask: Generate soft-edged mask with specified method from the 3D binary map corresponding to the specified molecular mass or density threshold, using the values provided through %%--%%gm_* options. Available methods are (1) 'gauss' for gaussian soft-edge and (2) 'cosine' for cosine soft-edged (used in PostRefiner). (default False) 
  
 \\ \\
 === Advanced Parameters === === Advanced Parameters ===
-  ; %%--%%gm_dilation Soft-edge dilation [Pixels]: The pixel width to dilate the 3D binary map corresponding to the specified molecular mass or density threshold prior to softening the edge. By default, it is set to half of %%--%%gm_edge_width so that the voxels with 1.0 values in the mask are same as the hard-edged binary map with the threadhold. Effective only with %%--%%enerate_mask option. (default -1.0) +  ; input_volume_path_2nd : Second input volume path: Path to second input volume file containing the 3D density map. Use this option to create a mask from the sum of two MERIDIEN half-set maps. (default none) 
-  ; %%--%%gm_edge_width : Soft-edge width [Pixels]: The pixel width of transition area for soft-edged masking. Effective only with %%--%%generate_mask option. (default 6.0) +  ; %%--%%ndilation Dilation width [Pixels]: The pixel width to dilate the 3D binary volume corresponding to the specified molecular mass or density threshold prior to softening the edge. By default, it is set to half of --moon_distance so that the voxels with 1.0 values in the mask are same as the hard-edged molecular-mass binary volume. (default -1.0) 
-  ; %%--%%gm_edge_sigma : Soft-edge sigma [Pixels]: The Gaussian sigma of transition area for soft-edged masking. This value controls the fall-speed of soft-edge; The samller the sigma, the faster the falloff. Effective only with %%--%%generate_mask and %%--%%generate_mask='gauss'. (default 2.0) +  ; %%--%%edge_width : Soft-edge width [Pixels]: The pixel width of transition area for soft-edged masking.(default 1)
-  ; %%--%%outputs_root : Root name of outputs: Specify the root name of all outputs. It cannot be empty string or only white spaces. (default vol3d) +
-  ; %%--%%allow_disconnect : Allow disconnection: Allow disconnection of density maps. Only for very special cases. (default False)+
   ; %%--%%edge_type : Soft-edge type: The type of soft-edge for moon-eliminator 3D mask and a moon-eliminated soft-edged 3D mask. Available methods are (1) 'cosine' for cosine soft-edged (used in PostRefiner) and (2) 'gauss' for gaussian soft-edge. (default cosine)   ; %%--%%edge_type : Soft-edge type: The type of soft-edge for moon-eliminator 3D mask and a moon-eliminated soft-edged 3D mask. Available methods are (1) 'cosine' for cosine soft-edged (used in PostRefiner) and (2) 'gauss' for gaussian soft-edge. (default cosine)
 +  ; %%--%%outputs_root : Root name of outputs: Specify the root name of all outputs. It cannot be empty string or only white spaces. (default vol3d)
 +  ; %%--%%resampled_shift3d : Providing resampled 3D shifts: Use this option when you are providing the resampled 3D shifts (using pixel size of outputs) when %%--%%resample_ratio!=1.0. By default, the program assums the provided shifts are not resampled. (default False)
 +  ; %%--%%shift3d_x : 3D x-shift [Pixels]: 3D x-shift value. (default 0)
 +  ; %%--%%shift3d_y : 3D y-shift [Pixels]: 3D y-shift value. (default 0)
 +  ; %%--%%shift3d_z : 3D z-shift [Pixels]: 3D z-shift value. (default 0)
 +  ; %%--%%aa : Low-pass filter fall-off [1/Pixels]: Low-pass filter fall-off in absolute frequency. The program applies this low-pass filter before the moon elimination. Effective only when %%--%%fl > 0.0. (default 0.1) : %%--%%fl!=-1.0
   ; %%--%%debug : Run with debug mode: Mainly for developer. (default False)   ; %%--%%debug : Run with debug mode: Mainly for developer. (default False)
  
Line 117: Line 113:
     * If moon_distance is too long, the moons will be connected and creates strange low density shape at the edge of the moon_distance.     * If moon_distance is too long, the moons will be connected and creates strange low density shape at the edge of the moon_distance.
     * On the other hand, if it is too short, soft-edge will have the stair-like gray level change because of quantization or digitization.     * On the other hand, if it is too short, soft-edge will have the stair-like gray level change because of quantization or digitization.
-  * dilation+  * ndilation
     * Setting dilation to half of moon_distance generates mask where the voxels with 1.0 values are same as the hard-edged molecular-mass binary map (default behaviour).     * Setting dilation to half of moon_distance generates mask where the voxels with 1.0 values are same as the hard-edged molecular-mass binary map (default behaviour).
     * Setting dilation to smaller than half of moon_distance generates mask where the voxels with 1.0 values are smaller than the hard-edged molecular-mass binary map.     * Setting dilation to smaller than half of moon_distance generates mask where the voxels with 1.0 values are smaller than the hard-edged molecular-mass binary map.
Line 123: Line 119:
     * In principle, smaller edge_sigma is better.     * In principle, smaller edge_sigma is better.
     * However, edge_sigma must be at least larger than 1[pixel].      * However, edge_sigma must be at least larger than 1[pixel]. 
-    * If not, the density distribution of moon eliminator 3D mask won't be smooth (spiky) because of quantization or digitization. 
-    * In addition, moon-eliminated reference 3D map will have a strange dent near zero. 
- 
-Tips about balancing settings of gm_dilation, gm_edge_width, and gm_edge_sigma options for Gaussian soft-edge. 
-  * gm_dilation 
-    * Setting dilation to half of gm_edge_width generates mask where the voxels with 1.0 values are same as the hard-edged molecular-mass binary map (default behaviour). 
-    * Setting dilation to smaller than half of gm_edge_width generates mask where the voxels with 1.0 values are smaller than the hard-edged molecular-mass binary map. 
-  * gm_edge_width  
-    * A wider gm_edge_width is better. If it is too narrow, soft-edge will have the stair-like gray level change because of quantization or digitization. 
-    * However, if it is too wide, the soft-edge might touch the box boundary and create strange low density shape there. 
-  * gm_edge_sigma 
-    * In principle, smaller gm_edge_sigma is better. 
-    * However, gm_edge_sigma must be at least larger than 1[pixel].  
     * If not, the density distribution of moon eliminator 3D mask won't be smooth (spiky) because of quantization or digitization.     * If not, the density distribution of moon eliminator 3D mask won't be smooth (spiky) because of quantization or digitization.
     * In addition, moon-eliminated reference 3D map will have a strange dent near zero.     * In addition, moon-eliminated reference 3D map will have a strange dent near zero.
Line 149: Line 132:
 \\ \\
 ==== Files ==== ==== Files ====
-sparx/bin/sxpipe.py+sparx/bin/sp_pipe.py
  
 \\ \\
 ==== See also ==== ==== See also ====
-[[pipeline:viper:sxrviper|sxrviper]], [[pipeline:viper:sxviper|sxviper]], [[pipeline:isac:sxisac2|sxisac2]], [[pipeline:utilities:sxprocess|sxprocess]], [[pipeline:meridien:sxmeridien|sxmeridien]], [[pipeline:sort3d:sxsort3d_depth|sxsort3d_depth]], and [[pipeline:localres:sxfilterlocal|sxfilterlocal]].+[[pipeline:viper:sp_rviper|sp_rviper]], [[pipeline:viper:sp_viper|sp_viper]], [[pipeline:isac:sp_isac2|sp_isac2]], [[pipeline:utilities:sp_process|sp_process]], [[pipeline:meridien:sp_meridien|sp_meridien]], [[pipeline:sort3d:sp_sort3d_depth|sp_sort3d_depth]], and [[pipeline:localres:sp_filterlocal|sp_filterlocal]].
  
 \\ \\
Line 164: Line 147:
  
 \\ \\
 +
pipeline/utilities/sxpipe_moon_eliminator.txt · Last modified: 2019/04/02 10:42 by lusnig