Research in nanoscopy (e.g. electron microscopy) and imaging involves large datasets of images. Not only is there a data storage challenge, data management and processing requires high performance computing facilities and connectivity for access to data.
The main projects involves building a pipeline to construct 3D electron microscopy images in collaboaration with NECEN.
The e-BioGrid team is open to take dedicated Nanoscopy and imaging projects on board. Contact us
if you are involved in Nanoscopy or Imaging life science research and you
have a need for support in software of hardware infrastructure.
With powerful machines available at the Netherlands Centre for Electron Microscopy various forms of 3D reconstruction in EM can be realized. Reconstruction methods range from near-atomic resolution to the level of ultra-structure. These methods are computationally intensive. Parallelization and Grid computing have only been partially adopted in this field. Particle-reconstruction has been implemented on a cluster using the IMAGIC software. X-ray crystallography uses the CCP4-suite and this suite is partially made suitable for the Grid. Computerized EM tomography invokes computer clusters using the IMOD software. Here, special attention need be given to enhancement of the 3D reconstruction. The goal of this project is to bundle 3D-reconstruction tools into an e-science problem solving environment for nanoscopy that is Grid enabled.
Fons Verbeek, Leiden Institute of Advanced Computer Science
Infrastructure developed. Software for Single Particle Analysis (SPA) and well as eTomography is used on powerfull workstations. It has turned out the implementation that has been designed on the small cluster had considerable increase in performance. However, as the software could not be ported to a GRID the solution for a cloud was elaborated. This is a scalable solution in which a cluster is created in the cloud according to the size the user considers fit for the job at hand. The HPC infrastructure provided by SURFsara was used to test the specifications that were elaborated for this specific project. The implementation of the specifications is based on a Debian Linux OS. A head node is defined from which a command line interface is provided. The user interface provided facilities for starting batch jobs that are parallelized over the core nodes. The number of core nodes is chosen such to prevent CPU overload and overhead spill. From the Head node the process is started with the parameters the user has chosen. From the command line an graphical X11 based interface can be invoked to start a particular interface. In addition to the CL interface a web interface is provided to manage the data in a comfortable manner. Basically, the web interface acts like a unix shell. Both interfaces require some basic knowledge of the unix cl structure. Next to that knowledge, users do have good insights in the software that is being ported to the platform.
Knowledge. In this project a considerable knowledge has been acquired to create a flexible and powerful computational environment for Electron Nanoscopy. In the next years this knowledge should be consolidated on computational platforms, used on the largest possible scale and expanded with new insights when required. The platform will support researchers in the field of Electron Nanoscopy to execute complex tasks that are emerging from their research. It takes into account the increasing size of the data volumes and can adapt to new situations. In the next few years we hope to extend our knowledge on these types of computational platforms.
Software. The software in fitted in a middle layer for the OS basically starting a head node that distributes to client nodes. The middle layer was specifically designed and implemented for this platform architecture.
Other. The achieved results are suitable for use in the field of Electron Nanoscopy. In particular researchers involved in NeCEN will be future users. Amongst the testers were researchers that are involved in the NeCEN. They have been selected for their knowledge on particular processes and specific software that we have ported to the cloud.
Publications. Cloud and Cluster Computing for eNanoscopy, in preparation (FJ Sicking & FJ Verbeek)
Fons Verbeek, Floris Sicking, N. Pannu, Jan-Pieter Abrahams, Bram Koster
Advanced analyses, easier Electron microscopy is of invaluable importance for the study of the complex organization and the architecture of cellular structures. In recent years new and (mostly automated) electron microscopical techniques have been developed (like electron tomography (ET) and focused ion beam scanning electron microscopy (FIB-SEM) that provide us with 3-dimensional (3D) information of the cell. This added dimension gave us new insights in cellular structures, and the interrelatedness between organelles and cellular processes. Automated 3D analysis methods are still in their infancy. Data extraction is still mainly depending on manual segmentation techniques, and therefore time consuming and subjective.
Structured storage Modern 3D electron microscopic recording methods and techniques (not only (S)TEM Tomography, but also FIB-SEM, ILEM and SEM) will have to cope with ever growing amounts of data. This amount of data needs to be stored in a structured and well-organized manner in order to be and remain accessible to its users. Currently data-storage is done by the individual researcher in many different ways and forms. A better structured and more uniform way of storing data and organizing data-management is a precondition for the primary science case, but it also creates the opportunity for long term use and thus enabling future reuse of electron microscopy information by research institutes and their distant collaborators.
The primary objective of this BiG Grid project is, to improve computer intensive analysis methods - including 3D template matching of 3D electron microscopy tomography data. In addition, the improvements include providing broad access of these methods to the 3D electron microscopy community. The secondary objective is to implement a data storage system for 3D electron microscopy data.
In order to achieve these objectives, an intensive collaboration of expert partners is needed to establish an infrastructure for the analysis of 3D electron microscopy data.
This project contains 5 main activities 1. Reduce the total compute time and improve the compute capacity by implementation of existing 3D analysis algorithms on GPU's 2. Improvement of the 3D analysis process by implementation of additional analysis and information management tools, objectifying and improving reliability of 3D data mining of electron tomography volumes 3. Improvement of the accessibility by creating an intuitive user environment 4. Improvement of storage, retrieval, archival and the controlled sharing of 3D data for electron microscopy data-analysis 5. Ensuring continuity and availability of the created solutions during and after the duration of the project
This project stems from a previous collaboration (IOP: IGE03012/VL-E: CellTom) in a IOP genomics programme. The Sara e-science support team has assisted in creating the project proposal and has established the project organization.
The foundation of this project is a proposal by the 3D electron microscopy group of the Utrecht University. This proposal is supported by the Leiden University Medical Center.