auto_2d_class_selection

This version (2019/07/29 15:32) was approved by twagner.The Previously approved version (2019/07/11 14:03) is available.Diff

Cinderella: Automatic 2D class selection


"The good ones go into the pot, the bad ones go into your crop."

                                                        From the fairy tale Cinderella


Our automatic 2d class selection tool (Cinderella) is based on a deep learning network to seperate 2D classes from .hdf / .mrcs files into good and bad classes. It uses the same deep neural network as crYOLO and was pretrained on a set good / bad classes. Cinderella was written to automate cryo-em data processing. It's open source and easy to use (see tutorial). You can easily train it with your own set of classes.

Here are a couple of examples for good / bad classes in Cinderella:

The Model

Our model was trained on a set of 2D classes from ISAC. During the creation of the training dataset, I tried to ask myself “Which class would I select If I would not know the particle?” to decide which is a good class.

The training dataset does not contain any Relion classes, so it might be that Cinderella will not work with well them.

You can easily contribute your own classes!

Right now our model is trained on 19 datasets. But we will increase the number often!

Download

Version: 0.3.1

Uploaded: 29. July 2019

DOWNLOAD

Uploaded: 11. July 2019, Datasets: 19

DOWNLOAD

Valid configuration file

Old version of cinderella and the pretrained model can be found in the archive

Version 0.3.1

  • Downgrade to tensorflow 1.10.1 again, as user report long initialization times
  • Only report the number of good / bad classes + their fraction.

Version 0.3.0

  • More data augmentation (add rotation)
  • Better sampling of validation data. It is now ensured that each file contributes some validation data.
  • Updated tensorflow to 1.12.3 to make it compatible to the crYOLO environment

Contribute

Here is the repository of our training data:

Download the public training data

Unfortunately, we cannot upload the complete training dataset, as some classes are from projects that are not yet published.

If you want to contribute with your own classes, please upload them here:

Contribute good / bad classes

Ideally, please upload separate HDF/mrcs files for good and bad classes. You can do this separation with EMAN2's e2display. However, you can also upload the classes without separation and we will try to do it.

Installation

The following instructions assume that pip and anaconda or miniconda are available. In case you have a old cinderella environment installed, first remove the old one with:

conda env remove --name cinderella

After that, create a new virtual environment:

conda create -n cinderella -c anaconda python=3.6 pyqt=5 cudnn=7.1.2

Activate the environment:

source activate cinderella

Install fast numpy:

conda install numpy==1.15.4

Install Cinderella for GPU:

pip install cinderella[gpu]

… or CPU:

pip install cinderella[cpu]

Tutorial

  • auto_2d_class_selection.txt
  • Last modified: 2019/07/29 14:48
  • by twagner