An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics

By Ronald C. Taylor. BMC Bioinformatics, 2010

512 citations recorded by the source corpus.

BackgroundBioinformatics researchers are now confronted with analysis of ultra large-scale data sets, a problem that will only increase at an alarming rate in coming years. Recent developments in open source software, that is, the Hadoop project and associated software, provide a foundation for scaling to petabyte scale data warehouses on Linux clusters, providing fault-tolerant parallelized analysis on such data using a programming style named MapReduce.DescriptionAn overview is given of the current usage within the bioinformatics community of Hadoop, a top-level Apache Software Foundation project, and of associated open source software projects. The concepts behind Hadoop and the associated HBase project are defined, and current bioinformatics software that employ Hadoop is described. The focus is on next-generation sequencing, as the leading application area to date.ConclusionsHadoop and the MapReduce programming paradigm already have a substantial base in the bioinformatics community, especially in the field of next-generation sequencing analysis, and such use is increasing. This is due to the cost-effectiveness of Hadoop-based analysis on commodity Linux clusters, and in the cloud via data upload to cloud vendors who have implemented Hadoop/HBase; and due to the effectiveness and ease-of-use of the MapReduce method in parallelization of many data analysis algorithms.

Subfields: Molecular Biology

View Paper: https://www.semanticscholar.org/paper/b84cc17984f6f13bfadd2b61a605dabf9d9bfa8b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

Clinical decision support systems for improving diagnostic accuracy and achieving precision medicine (2015, methodology): “Alternative analytic systems such as Hadoop, an open source software framework for large dataset analysis, are already being used in BLAST multiple sequence alignment and in the topological analysis of immense genotype datasets [57].”
The rise of "big data" on cloud computing: Review and open research issues (2015, background): “Other examples of key-value stores are Apache Hbase [23], Apache Cassandra [24], and Voldemort.”
Parallel data processing with MapReduce: a survey (2012, methodology): “The use of MapReduce for data intensive scientific analysis and bioinformatics are well studied in [41, 80].”
'Big data', Hadoop and cloud computing in genomics (2013, methodology): “MapReduce is Google’s solution for processing big data [38] and was developed as large Internet search engine providers were the first to truly face the ‘‘big data tsunami’’, indexing billions of webpages in a quick and meaningful way.”
Fractal MapReduce decomposition of sequence alignment (2012, methodology): “Accordingly, the use the MapReduce functional pattern now underlie many of the leading genomic analysis packages such as GATK [12] and CloudBurst [13] and is the key cloud computing abstraction for large scale data management and analysis [11,14].”
Next-generation transcriptome assembly (2011, background): “Alternatively, cloud computing 67 is an attractive framework for parallel computing, since computing resources can be rented as a service on an as-needed basis.”
Big Data Application in Biomedical Research and Health Care: A Literature Review (2016, background): “29 This section classifies big data technologies/tools into four categories: (1) data storage and retrieval, (2) error identification, (3) data analysis, and (4) platform integration deployment.”