Connect with us

Algorithms

SparkBLAST: Introduction

Dr. Muniba Faiza

Published

on

The basic local alignment search tool (BLAST) [1,2] is known for its speed and results, which is also a primary step in sequence analysis. The ever-increasing demand for processing huge amount of genomic data has led to the development of new scalable and highly efficient computational tools/algorithms. For example, MapReduce is the most widely accepted framework which supports design patterns representing general reusable solutions to some problems including biological assembly [3] and is highly efficient to handle large datasets running over hundreds to thousands of processing nodes [4]. But the implementation frameworks of MapReduce (such as Hadoop) limits its capability to process smaller data.

Cloud computing has emerged as a powerful tool to process these dynamic tasks over the last decade. Recently, a framework called Apache Spark emerged as a promising framework to implement highly scalable parallel applications [5,6]. A new parallelization of BLAST is developed by de Castro et al., 2017 employs cloud computing and Apache Spark as the coordination framework [7]. The SparkBLAST reduces the number of local input/output operations resulting in highly efficient and superior performance [7].

Working:

It requires two input files: a) a target database consisting of bacterial genomic sequences, and b) a query file consisting of a set of query genome sequences to be compared with the sequences of the target’s database. The basic concept of the working of SparkBLAST is that as soon as it takes the input, it replicates the entire input database on every computing node, then it split the query file into fragments which are later evenly distributed to every node, thus, rendering each node with the local deployment of the BLAST application, a copy of the target database, and a fragment of the query sequences [7]. After that, the whole computation is partitioned into tasks by the (Spark’s scheduler), which is assigned to the computing nodes depending on the data locality [8]. The replicated target database and the fragment of the query file are loaded into the memory for the execution of each task. The SparkBLAST then install the NCBI BLAST2 locally at each node with the help of Spark Pipe to execute multiple parallels and distributed tasks in the cluster. SparkBLAST uses YARN [9] as the resource manager to execute the tasks as it can be uniformly used by the Spark and Hadoop.

 

The processing of SparkBLAST is divided into three categories [7]:

1. Pre-processing

In this first stage, the query file is evenly partitioned into splits which are then distributed among the computing nodes.

2. Main-processing

After the data transferred to each processing node, the tasks are then scheduled on each computing node to be executed according to the data locality. At this stage, the NCBI BLAST2 has been installed locally at each node. As a computing core finishes a task, it starts working in a loop for all the tasks assigned to it and keeps doing until the available core executes all tasks of the assigned job.

3. Post-processing

During the second stage of processing, each individual task produces small output files, which are merged into a single output file at this stage.

Several other tools combine cloud and Hadoop technologies such as CloudBLAST [10], BioDoop [11], and Crossbow [12]. The SparkBLAST was found to outperform the CloudBLAST which is also a cloud platform [7,10].

For further details about SparkBLAST, click here.

References

  1. Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990). Basic local alignment search tool. Journal of molecular biology215(3), 403-410.
  2. Camacho, C., Coulouris, G., Avagyan, V., Ma, N., Papadopoulos, J., Bealer, K., & Madden, T. L. (2009). BLAST+: architecture and applications. BMC bioinformatics10(1), 421.
  3. O’Driscoll, A., Daugelaite, J., & Sleator, R. D. (2013). ‘Big data’, Hadoop and cloud computing in genomics. Journal of biomedical informatics46(5), 774-781.
  4. Senger, H., Gil‐Costa, V., Arantes, L., Marcondes, C. A., Marín, M., Sato, L. M., & Silva, F. A. (2016). BSP cost and scalability analysis for MapReduce operations. Concurrency and Computation: Practice and Experience28(8), 2503-2527.
  5. Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2010). Spark: Cluster computing with working sets. HotCloud10(10-10), 95.
  6. Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., … & Stoica, I. (2012, April). Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation (pp. 2-2). USENIX Association.
  7. de Castro, M. R., dos Santos Tostes, C., Dávila, A. M., Senger, H., & da Silva, F. A. (2017). SparkBLAST: scalable BLAST processing using in-memory operations. BMC Bioinformatics18(1), 318.
  8. Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., & Stoica, I. (2010, April). Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In Proceedings of the 5th European conference on Computer systems (pp. 265-278). ACM.
  9. Vavilapalli, V. K., Murthy, A. C., Douglas, C., Agarwal, S., Konar, M., Evans, R., … & Saha, B. (2013, October). Apache hadoop yarn: Yet another resource negotiator. In Proceedings of the 4th annual Symposium on Cloud Computing (p. 5). ACM.
  10. Matsunaga, A., Tsugawa, M., & Fortes, J. (2008, December). Cloudblast: Combining mapreduce and virtualization on distributed resources for bioinformatics applications. In eScience, 2008. eScience’08. IEEE Fourth International Conference on (pp. 222-229). IEEE.
  11. Leo, S., Santoni, F., & Zanetti, G. (2009, September). Biodoop: bioinformatics on hadoop. In Parallel Processing Workshops, 2009. ICPPW’09. International Conference on (pp. 415-422). IEEE.
  12. Langmead, B., Hansen, K. D., & Leek, J. T. (2010). Cloud-scale RNA-sequencing differential expression analysis with Myrna. Genome biology11(8), R83.

Dr. Muniba is a Bioinformatician based in New Delhi, India. She has completed her PhD in Bioinformatics from South China University of Technology, Guangzhou, China. She has cutting edge knowledge of bioinformatics tools, algorithms, and drug designing. When she is not reading she is found enjoying with the family. Know more about Muniba

Advertisement
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Algorithms

MOCCA- A New Suite to Model cis- regulatory Elements for Motif Occurrence Combinatorics

Published

on

MOCCA- A New Suite to Model cis- regulatory Elements for Motif Occurrence Combinatorics

cis-regulatory elements are DNA sequence segments that regulate gene expression. cis-regulatory elements consist of some regions such as promoters, enhancers, and so on. These regions consist of specific sequence motifs. (more…)

Continue Reading

Algorithms

vs_Analysis.py: A Python Script to Analyze Virtual Screening Results of Autodock Vina

Dr. Muniba Faiza

Published

on

VS-Analysis: A Python Script to Analyze Virtual Screening Results of Autodock Vina

The output files obtained as a result of virtual screening (VS) using Autodock Vina may be large in number. It is difficult or quite impossible to analyze them manually. Therefore, we are providing a Python script to fetch top results (i.e., compounds showing low binding affinities). (more…)

Continue Reading

Algorithms

How to search motif pattern in FASTA sequences using Perl hash?

Dr. Muniba Faiza

Published

on

Here is a simple Perl script to search for motif patterns in a large FASTA file with multiple sequences.

(more…)

Continue Reading

Algorithms

How to read fasta sequences from a file using PHP?

Published

on

Here is a simple function in PHP to read fasta sequences from a file. (more…)

Continue Reading

Algorithms

How to read fasta sequences as hash using perl?

Published

on

This is a simple Perl script to read a multifasta file as a hash. (more…)

Continue Reading

Algorithms

BETSY: A new backward-chaining expert system for automated development of pipelines in Bioinformatics

Dr. Muniba Faiza

Published

on

Bioinformatics analyses have become long and difficult as it involves a large number of steps implemented for data processing. Bioinformatics pipelines are developed to make this process easier, which on one hand automate a specific analysis, while on the other hand, are still limited for investigative analyses requiring changes to the parameters used in the process. (more…)

Continue Reading

Algorithms

Algorithm and workflow of miRDB

Published

on

As mentioned in the previous article, Micro RNAs (miRNAs) are the short endogenous RNAs (~22 nucleotides) and originate from the non-coding RNAs [1], produced in single-celled eukaryotes, viruses, plants, and animals [2]. They play significant roles in various biological processes such as degradation of mRNA [3]. Several databases exist storing a large amount of information about miRNAs, one of such databases miRBase [4] was explained in the previous article, today we will explain the algorithm of miRDB [5,6], another database for miRNA target prediction. (more…)

Continue Reading

Algorithms

miRBase: Explained

Dr. Muniba Faiza

Published

on

Micro RNAs (miRNAs) are the short endogenous RNAs (~22 nucleotides) and originate from the non-coding RNAs [1], produced in single-celled eukaryotes, viruses, plants, and animals [2]. miRNAs are capable of controlling homeostasis [2] and play significant roles in various biological processes such as degradation of mRNA and post-translational inhibition through complementary base pairing [3].  (more…)

Continue Reading

Algorithms

Prediction of biochemical reactions catalyzed by enzymes in humans

Dr. Muniba Faiza

Published

on

There are many biological important enzymes which exist in the human body, one of them is Cytochrome P450 (CyP450) enzymes which are mostly considered in drug discovery due to their involvement in the majority (75%) of drug metabolism [1]. Therefore, various in-silico methods have been applied to predict the possible substrates of CyP 450 enzymes [2-4]. Recently, an in-silico model has been developed to predict the potential chemical reactions mediated by the enzymes present in humans including CyP450 enzymes [5]. (more…)

Continue Reading

Algorithms

A new high-level Python interface for MD simulation using GROMACS

Published

on

The roots of the molecular simulation application can be traced back to physics where it was applied to simplified hard-sphere systems [1]. This field of molecular simulation study has gained a lot of interest since then and applied to perform simulations to fold small protein at multi-microsecond scale [2-4], predict functional properties of receptors and to capture the intermediate transitions of the complex [5], and to study the movement and behavior of ligand in a binding pocket and also to predict interactions between receptors and ligands [6,7]. (more…)

Continue Reading

Algorithms

Machine learning in prediction of ageing-related genes/proteins

Dr. Muniba Faiza

Published

on

Ageing has a great impact on human health, when people’s age advance towards 80 years, approximately half of the proteins in the body get damaged through oxidation. The chemical degradations occurring in our body produce energy by the consumed food via oxidation in the presence of oxygen. (more…)

Continue Reading

Algorithms

Simulated sequence alignment software: An alternative to MSA benchmarks

Dr. Muniba Faiza

Published

on

In our previous article, we discussed different multiple sequence alignment (MSA) benchmarks to compare and assess the available MSA programs. However, since the last decade, several sequence simulation software have been introduced and are gaining more interest. In this article, we will be discussing various sequence simulating software being used as alternatives to MSA benchmarks. (more…)

Continue Reading

Algorithms

Benchmark databases for multiple sequence alignment: An overview

Dr. Muniba Faiza

Published

on

Multiple sequence alignment (MSA) is a very crucial step in most of the molecular analyses and evolutionary studies. Many MSA programs have been developed so far based on different approaches which attempt to provide optimal alignment with high accuracy. Basic algorithms employed to develop MSA programs include progressive algorithm [1], iterative-based [2], and consistency-based algorithm [3]. Some of the programs incorporate several other methods into the process of creating an optimal alignment such as M-COFFEE [4] and PCMA [5]. (more…)

Continue Reading

Algorithms

ab-initio prediction of protein structure: An introduction

Dr. Muniba Faiza

Published

on

We have heard a lot about the ab-initio term in Bioinformatics, which could be difficult to understand for newbies in the field of bioinformatics. Today, we will discuss in detail what ab-initio is and what are the applicable methods for it. (more…)

Continue Reading

Algorithms

Intrinsically disordered proteins’ predictors and databases: An overview

Published

on

Intrinsically unstructured proteins (IUPs) are the natively unfolded proteins which must be unfolded or disordered in order to perform their functions.  They are commonly referred to as intrinsically disordered proteins (IDPs) and play significant roles in regulating and signaling biological networks [1]. IDPs are also involved in the assembly of signaling complexes and in the dynamic self-assembly of membrane-less nuclear and cytoplasmic organelles [1]. The disordered regions in a protein can be highly conserved among the species in respect of both the composition and the sequence [2]. (more…)

Continue Reading

Algorithms

An introduction to the predictors of pathogenic point mutations

Dr. Muniba Faiza

Published

on

Single nucleotide variation is a change in a single nucleotide in a sequence irrespective of the frequency of the variation. Single nucleotide variants (SNVs) play a very important role in causing several diseases such as the tumor, cancer, etc. Many efforts have been made to identify the SNVs which were initially based on identifying non-synonymous mutations in coding regions of the genomes. (more…)

Continue Reading

Algorithms

Role of Information Theory, Chaos Theory, and Linear Algebra and Statistics in the development of alignment-free sequence analysis

Published

on

By

Sequence alignment is customary to not only find similar regions among a pair of sequences but also to study the structural, functional and evolutionary relationship between organisms. Many tools have been discovered to achieve the goal of alignment of a pair of sequences, separately for nucleotide sequence and amino acid sequence, BLOSSUM & PAM [1] are a few to name. (more…)

Continue Reading

Algorithms

Bioinformatics Challenges and Advances in RNA interference

Published

on

By

 

 

RNA interference is a post-transcriptional gene regulatory mechanism to down-regulate the gene expression either by mRNA degradation or by mRNA translation inhibition. The mechanism involves a small partially complementary RNA against the target gene. To perform the action, it also requires a class of dedicated proteins to process these primary RNAs into mature microRNAs. The guide sequence determines the specificity of the miRNA. Therefore, the knowledge of the guide sequence is crucial for predicting its targets and also exploiting the sequence to create a new regulatory circuit. In this short review, we will briefly discuss the role and challenges in miRNA research for unveiling the target prediction by bioinformatics and to foster our understanding and applications of RNA interference. (more…)

Continue Reading

Algorithms

Systems pharmacology and drug development

Dr. Muniba Faiza

Published

on

Systems pharmacology is an emerging area in the field of medicinal chemistry and pharmacology which utilizes systems network to understand drug action at the organ and organism level. It applies the computational and experimental systems biology approaches to pharmacology, which includes network analyzes at multiple biological organization levels facilitating the understanding of both therapeutic and adverse effects of the drugs. Nearly a decade ago, the term systems pharmacology was used to define the drug action in a specific organ system such as reproductive pharmacology [1], but to date, it has been expanded to different organ and organism levels [2]. (more…)

Continue Reading

Algorithms

Recent advances in in-silico approaches for enzyme engineering

Dr. Muniba Faiza

Published

on

Enzymes are natural biocatalysts and an attractive alternative to chemicals providing improved efficiency for biochemical reactions. They are widely utilized in industrial biotechnology and biocatalysis to introduce new functionalities and enhance the production of enzymes. In order to be proved beneficial for industrial purposes, the enzymes need to be optimized by applying protein engineering. This article specifically reviews the recent advancements in the computational approaches for enzyme engineering and structural determination of the enzymes developed in recent years to improve the efficiency of enzymes, and the creation of novel functionalities to obtain
products with high added value for industrial applications.
(more…)

Continue Reading

LATEST ISSUE

ADVERT