gpu_isac_chimera []

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{ :gpu_isac:gpu_isac_chimera_logo.png?600 |}}

===== Overview =====

**ISAC** (Iterative Stable Alignment and Clustering) is a 2D classification algorithm to sort cryo-EM particles into classes depicting the same view of a target protein. It is based around iterations of 
alternating equal size k-means clustering and repeated 2D alignment routines. 

<note>
You can find the details of the ISAC algorithm in [[https://www.cell.com/fulltext/S0969-2126(11)00467-9|this paper]]. To cite ISAC, use the following:
''Yang, Z., Fang, J., Chittuluru, J., Asturias, F. J. and Penczek, P. A. (2012) Iterative stable alignment and clustering of 2D transmission electron microscope images. Structure 20, 237–247.''
</note>

**ISAC2** is an improved version of ISAC, and the default tool to produce 2D class averages in the **[[http://sphire.mpg.de/wiki/doku.php?id=downloads:sphire_1_3|SPHIRE]]** ([[https://github.com/cryoem/eman2|git]]) software package and the **[[https://github.com/MPI-Dortmund/transphire|TranSPHIRE]]** automated pipeline for processing cryo-EM data. ISAC2 is a CPU-only implementation that is usually run on a computer cluster.

**GPU ISAC** is designed to run ISAC2 on a single workstation by outsourcing its computationally expensive calculations to any available GPUs.

<note important>
**Beta version:** The currently available version of GPU ISAC is a beta version. A significantly improved and significantly faster version is being tested right now and will be released shortly.
</note>

===== Download & Installation =====

<note important>
  * **CUDA required:** The following installation instructions assume that CUDA is already installed on your system.

  * **SPHIRE required:** In order to use GPU ISAC, SPHIRE needs to be installed first and the following instructions assume that this is the case. You can find the SPHIRE download and installation instructions [[http://sphire.mpg.de/wiki/doku.php?id=downloads:sphire_1_3#download|here]].

//**Note:** To confirm a working version of SPHIRE 1.3 (or later), use //''which sphire''// in your terminal. It should return to you the path to your SPHIRE installation.//
</note>

=== Download ===

  * The GPU ISAC beta version is currently distributed as a .tar file that can be found {{:gpu_isac:gpu_isac_beta_chimera_.tar|here}}.
  * A printout of the installation notes below can be found {{:gpu_isac:gpu_isac_installation_notes.pdf|here}}.

=== Installation ===

  - **Extract the archive** to your chosen GPU ISAC installation folder.
  - **Open a terminal** and navigate to your installation folder.
  - **Untar the archive**: <code>tar -xf GPU_ISAC_CHIMERA.tar</code>
  - **Check CUDA path variables** using:
    - <code>echo $PATH</code> Confirm your ''PATH'' variable contains the path to your **cuda/bin** folder.
    - <code>echo $LD_LIBRARY_PATH</code> Confirm your ''LD_LIBRARY_PATH'' variable contain the path to your **cuda/lib64** folder.
    - If the path variables do **not** contain these path variables, you can add them like so: <code>export PATH=/path/to/cuda/bin:${PATH}
export LD_LIBRARY_PATH=/path/to/cuda/lib64:${LD_LIBRARY_PATH}</code> Where ''path/to/cuda/bin'' and ''path/to/cuda/lib64'' need to be replaced with the real paths to the respective folders. If you do not know where to find them, by default they should be located in ''/usr/local/cuda''.
  - **Compile the GPU ISAC C++/CUDA library:**
    - <code>cd vChimera/cuda</code>//**Note:** This assumes you did not change directories after unpacking the .tar archive.//
    - <code>nvcc gpu_aln_common.cu gpu_aln_noref.cu -o gpu_aln_pack.so -shared -Xcompiler -fPIC -lcufft -std=c++11</code>
  - **Adjust sparx libraries** to work with the C++/CUDA library we just compiled:
    - <code>cd ../eman2/sparx/libpy</code>
    - <code>sed -i.bkp "s|/home/schoenf/work/code/cuISAC/cuda|$(realpath ../../../cuda)|g" applications.py</code>
    - <code>sed -i.bkp2 's|statistics.sum_oe( data, "a", CTF, EMData(), myid=myid|statistics.sum_oe( data, "a", CTF, EMData()|g' applications.py</code>
  - **Set the correct libraries and environment:**
    - <code>cd ../bin</code> //**Note:** We are now in the **/eman2/sparx/bin** folder of your GPU ISAC installation folder.//
    - <code>ln -rs ../libpy/* .</code> //**Note:** Don't forget the dot at the end!//
    - <code>sed -i.bkp "s|/home/schoenf/applications/sphire/v1.1/envs/sphire_1.3/bin|$(dirname $(which sphire))|g" sxisac2_gpu.py</code>
    - <code>sed -i.bkp2 "s/^\(.*options, args.*\)$/\1\n    os.environ['CUDA_VISIBLE_DEVICES'] = options.gpu_devices\n    options.gpu_devices = ','.join(map(str, range(len(options.gpu_devices.split(',')))))/g" sxisac2_gpu.py</code>
  - **Bugfix for the Chimera beta:**
    - <code>sed -i.bkp3 "s/output_text = \"\n/output_text = \"/g" sxisac2_gpu.py</code>


===== Usage =====

An example call to use GPU ISAC looks as follows:

<code>
mpirun -np 6 /path/to/sxisac2_gpu.py bdb:path/to/stack out_dir --CTF -–radius=160 --target_radius=29 --target_nx=76 --img_per_grp=100 --minimum_grp_size=60 --thld_err=0.7 --center_method=0 --gpu_devices=0,1
</code>

More readable:

<code>
mpirun -np 6 /path/to/sxisac2_gpu.py
bdb:path/to/stack out_dir
--CTF 
-–radius=160
--target_radius=29
--target_nx=76
--img_per_grp=100
--minimum_grp_size=60
--thld_err=0.7
--center_method=0
--gpu_devices=0,1
</code>

**[ ! ] - Mandatory** parameters in the GPU ISAC call:

  * Replace ''/path/to/sxisac2_gpu.py'' with the path to your **sxisac2_gpu.py** file.
  * Replace ''path/to/stack'' with the path to your **input .bdb stack**. If you are using an **.hdf** stack, you need to remove the ''bdb:'' prefix.
  * Replace ''out_dir'' with the path to your preferred **output directory**.
  * Adjust the number in ''<nowiki>--</nowiki>radius=160'' to the **radius of your particle** (in pixels).

**[?] - Optional** parameters in the GPU ISAC call:

  * In ''mpirun –np 6'' the number can be set to the number of your **CPU processors** (e.g., if you have a quad core CPU, you would use 4 here). 
  * Using ''<nowiki>--</nowiki>gpu_devices'' you can set **what GPUs to use**. This example uses two GPUs with id values 0 and 1, respectively. You can check the id values of your available GPUs by executing ''nvidia-smi'' in your terminal (GPUs are sorted by capability, with 0 being your strongest GPU).
  * You can also use ''<nowiki>--</nowiki>img_per_grp'' to limit the **maximum size of individual classes**. Empirically, a class size of 100-200 (30-50 for negative stain) particles has been proven successful when dealing with around 100,000 particles.
  * Similarly, you can also use ''<nowiki>--</nowiki>minimum_grp_size'' to limit the **minimum size of individual classes**. In general, this value should be around 50-60% of your maximum class size.

<note>
  * The full list of **ISAC2 parameters** can be found [[http://sphire.mpg.de/wiki/doku.php?id=pipeline:isac:sxisac2|here]].
  
  * **Additional utilities** that are helpful when using any version of ISAC can be found [[http://sphire.mpg.de/wiki/doku.php?id=pipeline:isac:start|here]].

  * More information about **using ISAC for 2D classification** can also be found in the ISAC chapter of the official [[ftp://ftp.gwdg.de/pub/misc/sphire/sphire_1_3_tutorial/sphire_1_3.pdf|SPHIRE tutorial]] (link to .pdf file).
</note>
User Tools

Site Tools

Page Tools
