This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
gpu_isac [2020/05/29 18:47] fschoenfeld |
gpu_isac [2021/02/22 13:33] fschoenfeld |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | {{ : | + | {{ : |
===== Overview ===== | ===== Overview ===== | ||
- | **ISAC** (Iterative Stable Alignment and Clustering) is a 2D classification algorithm | + | **ISAC** (//Iterative Stable Alignment and Clustering//) is a **2D classification algorithm**. It sorts a given stack of cryo-EM particles into different |
- | alternating equal size k-means clustering and repeated 2D alignment routines. | + | |
- | < | + | < |
- | You can find the details of the ISAC algorithm in [[https:// | + | ===== ISAC versions ===== |
- | '' | + | |
- | </ | + | |
- | + | ||
- | **ISAC2** is an improved version of ISAC, and the default tool to produce 2D class averages in the **[[http:// | + | |
- | + | ||
- | **GPU ISAC** is designed to run ISAC2 on a single workstation by outsourcing its computationally expensive calculations to any available GPUs. | + | |
- | + | ||
- | <note important> | + | |
- | **Beta version:** The currently available version of GPU ISAC is a beta version. A significantly improved and significantly faster version is being tested right now and will be released shortly. | + | |
- | </ | + | |
+ | * **ISAC** | ||
+ | * **ISAC2** | ||
+ | * **GPU ISAC** | ||
===== Download & Installation ===== | ===== Download & Installation ===== | ||
- | <note important> | + | <note important> |
- | | + | |
- | * **SPHIRE required:** In order to use GPU ISAC, SPHIRE needs to be installed first and the following instructions assume that this is the case. You can find the SPHIRE download and installation instructions [[http:// | + | ---- |
- | //**Note:** To confirm | + | * **CUDA:** These installation instructions assume that CUDA is already installed on your system. You can confirm |
+ | * **SPHIRE: | ||
</ | </ | ||
+ | |||
+ | ---- | ||
=== Download === | === Download === | ||
- | * The GPU ISAC beta version | + | * GPU ISAC is currently |
- | * A printout of the installation notes below can be found {{: | + | |
+ | ---- | ||
=== Installation === | === Installation === | ||
- | - **Extract the archive** to your chosen GPU ISAC installation folder. | + | Before |
- | - **Open a terminal** and navigate to your installation folder. | + | |
- | - **Untar the archive** using '' | + | |
- | - **Check CUDA path variables** using: | + | |
- | - '' | + | |
- | - '' | + | |
- | - If the path variables do **not** contain these path variables, | + | |
- | * '' | + | |
- | * '' | + | |
- | * Here '' | + | |
- | - **Compile the GPU ISAC C++/CUDA library: | + | |
- | - '' | + | |
- | - '' | + | |
- | - **Adjust sparx libraries** | + | |
- | - '' | + | |
- | - '' | + | |
- | - '' | + | |
- | - **Set the correct libraries and environment: | + | |
- | - '' | + | |
- | - '' | + | |
- | - '' | + | |
- | - '' | + | |
- | ===== Usage ===== | + | * During the SPHIRE installation, |
+ | < | ||
- | An example call to use GPU ISAC looks as follows: | + | conda env list |
+ | |||
+ | </ | ||
+ | |||
+ | * Look for your SPHIRE environment and activate it using either: | ||
< | < | ||
- | mpirun -np 6 / | + | conda activate NAME_OF_YOUR_ENVIRONMENT |
</ | </ | ||
- | More readable: | + | or |
< | < | ||
- | mpirun -np 6 /path/to/sxisac2_gpu.py | + | source activate NAME_OF_YOUR_ENVIRONMENT |
- | bdb: | + | |
- | --CTF | + | </ |
+ | |||
+ | It will depend on your system and Anaconda installation which one of these you will have to use. | ||
+ | |||
+ | </ | ||
+ | |||
+ | GPU ISAC comes with a handy installation script that can be used as follows: | ||
+ | |||
+ | - **Extract the archive** | ||
+ | - **Open a terminal** | ||
+ | - **Run the installation script**: | ||
+ | < | ||
+ | ./ | ||
+ | |||
+ | </ | ||
+ | |||
+ | All done! | ||
+ | |||
+ | < | ||
+ | |||
+ | ===== Running GPU ISAC ===== | ||
+ | |||
+ | When calling GPU ISAC from the terminal, an example call looks as follows: | ||
+ | |||
+ | < | ||
+ | mpirun | ||
+ | |||
+ | </ | ||
+ | |||
+ | Using the following mix of both mandatory and optional parameters (see below to learn which is which): | ||
+ | |||
+ | < | ||
+ | mpirun python | ||
+ | bdb: | ||
+ | path/ | ||
+ | --CTF | ||
-–radius=160 | -–radius=160 | ||
- | --target_radius=29 | ||
- | --target_nx=76 | ||
--img_per_grp=100 | --img_per_grp=100 | ||
--minimum_grp_size=60 | --minimum_grp_size=60 | ||
- | --thld_err=0.7 | ||
- | --center_method=0 | ||
--gpu_devices=0, | --gpu_devices=0, | ||
+ | |||
</ | </ | ||
- | **[ ! ] - Mandatory** parameters in the GPU ISAC call: | + | **[ ! ] - Mandatory** |
- | * Replace | + | * '' |
- | * Replace | + | * ''/ |
- | * Replace | + | * '' |
- | * Adjust the number in '' | + | * '' |
+ | * '' | ||
+ | * ''< | ||
+ | <hidden What GPUs do I have and what are their system id values?> You can use '' | ||
- | **[?] - Optional** parameters in the GPU ISAC call: | + | {{ |
- | * In '' | + | Above: Example output of '' |
- | * Using '' | + | |
- | * You can also use ''< | + | **[?] - Optional** parameters recommended |
- | * Similarly, | + | |
+ | | ||
+ | * Use ''< | ||
+ | * Use '' | ||
< | < | ||
- | * The full list of **ISAC2 parameters** can be found [[http:// | ||
- | | ||
- | * **Additional utilities** that are helpful when using any version of ISAC can be found [[http:// | ||
- | * More information about **using ISAC for 2D classification** can also be found in the ISAC chapter of the official [[ftp:// | + | |
+ | < | ||
+ | |||
+ | mpirun python / | ||
+ | |||
+ | </ | ||
+ | |||
+ | or simply | ||
+ | |||
+ | < | ||
+ | python / | ||
+ | |||
+ | </ | ||
+ | |||
+ | * The online documentation of **ISAC2 parameters** | ||
+ | |||
+ | * **Additional utilities** | ||
+ | | ||
</ | </ | ||
+ | |||
+ | ===== Examples ===== | ||
+ | |||
+ | **EXAMPLE 01: Test run** | ||
+ | |||
+ | This example is a test run that can be used to confirm GPU ISAC was installed successfully. It is a small stack that contains 64 artificial faces and is already included in the GPU ISAC installation package. You can process it using GPU ISAC as follows: | ||
+ | |||
+ | - In your terminal, navigate to your GPU ISAC installation folder: | ||
+ | < | ||
+ | |||
+ | cd / | ||
+ | |||
+ | </ | ||
+ | |||
+ | - Run GPU ISAC: | ||
+ | |||
+ | < | ||
+ | mpirun python bin/ | ||
+ | |||
+ | </ | ||
+ | |||
+ | Note that we don't care about the quality of any produced averages here; this test is used to make sure there are no runtime issues before a more time consuming run is executed. | ||
+ | |||
+ | ---- | ||
+ | |||
+ | **EXAMPLE 02: TcdA1 toxin data** | ||
+ | |||
+ | This example uses the [[https:// | ||
+ | |||
+ | After downloading the data you'll notice that the extracted folder contains a multitude of subfolders. For the purposes of this example we are only interested in the '' | ||
+ | |||
+ | You can process this stack using GPU ISAC as follows: | ||
+ | |||
+ | - In your terminal, navigate to your GPU ISAC installation folder: | ||
+ | < | ||
+ | |||
+ | cd / | ||
+ | |||
+ | </ | ||
+ | |||
+ | - Run GPU ISAC: | ||
+ | |||
+ | < | ||
+ | mpirun python bin/ | ||
+ | |||
+ | </ | ||
+ | |||
+ | * Replace ''/ | ||
+ | * Optional: Replace ''< | ||
+ | |||
+ | The final averages can then be found in '' | ||
+ | |||
+ | {{: | ||
+ | |||
+ | ===== Usage ===== | ||
+ | |||
+ | Next to producing high quality 2D class averages, GPU ISAC is also an excellent tool to screen your data which allows you to: | ||
+ | |||
+ | * Quickly generate **2D class averages**. | ||
+ | * Quickly identify **suitable parameters** | ||
+ | * Quickly gauge the **quality of your data** | ||
+ | |||
+ | <hidden Well, " | ||
+ | |||
+ | * **Class size: | ||
+ | * **Threshold error: | ||
+ | Since GPU ISAC processes small stacks of about 10,000 to 20,000 particles fairly quickly, you can try several runs with different values for ''< | ||
+ | |||
+ | ===== GPU ISAC output files ===== | ||
+ | |||
+ | GPU ISAC produces a multitude of output files that can be used to analyze the success of running the program, even while it is still ongoing. These include the following: | ||
+ | |||
+ | * **Main iteration folders: | ||
+ | * In both the main iteration folders and the base output folder you will find '' | ||
+ | * **The final averages** | ||
+ | ===== Release notes ===== | ||
+ | |||
+ | **GPU ISAC limitations** | ||
+ | |||
+ | * The current develpoment goal of GPU ISAC is to run as fast as possible on a single machine. Because of this priority, **GPU ISAC does not yet run on multiple nodes**. This is planned to change as soon as the currently known bottlenecks have all been converted to run on the available GPUs. | ||
+ | |||
+ | ---- | ||
+ | |||
+ | **Known issues** | ||
+ | |||
+ | * In some cases **when using CUDA version 11, GPU ISAC receives a kill signal interrupt**. We're investigating the issue but recommend to use a lower version (confirmed working when using CUDA 9 and 10) until it is resolved. You can use '' | ||
+ | |||
+ | ---- | ||
+ | |||
+ | **GPU ISAC v2.3.4** | ||
+ | |||
+ | * Updated the installer to automatically link GPU ISAC to the SPHIRE GUI. | ||
+ | |||
+ | **GPU ISAC v2.3.3** | ||
+ | |||
+ | * Internal changes only. | ||
+ | |||
+ | **GPU ISAC v2.3.1 & v2.3.2 (hotfix releases)** | ||
+ | |||
+ | * Changed data handling, which results in a massive reduction in overall memory usage and an increased pre-alignment performance. | ||
+ | * Fixed use of '' | ||
+ | * Fixed error in the pre-alignment progress bar that made it seem as if it did not run to completion. | ||
+ | * Minimum class size is now automatically set to 60% of the full class size, if no minimum class size was specified by the user. | ||
+ | |||
+ | **GPU ISAC v2.3** | ||
+ | |||
+ | * Updated GPU ISAC install package for version 2.3. | ||
+ | * Fixed multiple issues occurring when handling larger data sets (200k and upwards). | ||
+ | * Added memory checks to the output to document GPU ISAC memory use. | ||
+ | * Batched input read, part I: Input is now read in batches with pre-processing already applied during reading. This means that we can now deal with data sets that do not fit into system RAM (as long as the compressed data does). | ||
+ | * Batched input read, part II: Input reading and processing is spread across all MPI processes, while actual processing (pre-alignment) is spread only across GPU processes. This ensures data is processed using all CPU & GPU resources available to the used machine. | ||
+ | * Multiple bug fixes for increased stability and functionality. | ||
+ | * Cleanup of old code for increased readability and maintainability. | ||
+ | |||
+ | **GPU ISAC v2.2** | ||
+ | |||
+ | * Created an easy installation package for GPU ISAC including a quick test to confirm a successful installation. | ||
+ | * Added more sanity checks to catch invalid parameter combinations and abort the run immediately. | ||
+ | * Updated the transformation stack in GPU alignment functions for higher quality averages. | ||
+ | * Multiple bug fixes for increased stability and functionality. | ||
+ | |||
+ | **GPU ISAC " | ||
+ | |||
+ | * Initially released beta version of GPU ISAC. | ||
+ | |||
+ |