Show simple item record

Files in this item


Item metadata

dc.contributor.authorFilgueira, Rosa
dc.contributor.authorAwaysheh, Feras M.
dc.contributor.authorCarter, Adam
dc.contributor.authorWhite, Darren J.
dc.contributor.authorRana, Omar
dc.identifier.citationFilgueira , R , Awaysheh , F M , Carter , A , White , D J & Rana , O 2022 , SparkFlow : towards high-performance data analytics for Spark-based genome analysis . in 20252 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid) . IEEE , pp. 1007-1016 , Workshop on Clusters, Clouds and Grids for Life Sciences (CCGrid Life 2022) , Taormina , Italy , 16/05/22 .
dc.identifier.otherPURE: 278376911
dc.identifier.otherPURE UUID: d80c456d-f3ae-4bb7-bc16-9085e042bda2
dc.identifier.otherScopus: 85135742296
dc.identifier.otherWOS: 000855065800111
dc.description.abstractThe recent advances in DNA sequencing technology triggered next-generation sequencing (NGS) research in full scale. Big Data (BD) is becoming the main driver in analyzing these large-scale bioinformatic data. However, this complicated process has become the system bottleneck, requiring an amalgamation of scalable approaches to deliver the needed performance and hide the deployment complexity. Utilizing cutting-edge scientific workflows can robustly address these challenges. This paper presents a Spark-based alignment workflow called SparkFlow for massive NGS analysis over singularity containers. SparkFlow is highly scalable, reproducible, and capable of parallelizing computation by utilizing data-level parallelism and load balancing techniques in HPC and Cloud environments. The proposed workflow capitalizes on benchmarking two state-of-art NGS workflows, i.e., BaseRecalibrator and ApplyBQSR. SparkFlow realizes the ability to accelerate large-scale cancer genomic analysis by scaling vertically (HyperThreading) and horizontally (provisions on-demand). Our result demonstrates a trade-off inevitably between the targeted applications and processor architecture. SparkFlow achieves a decisive improvement in NGS computation performance, throughput, and scalability while maintaining deployment complexity. The paper’s findings aim to pave the way for a wide range of revolutionary enhancements and future trends within the High-performance Data Analytics (HPDA) genome analysis realm.
dc.relation.ispartof20252 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)en
dc.rightsCopyright © 2022 IEEE. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at
dc.subjectBig dataen
dc.subjectScientific workflowen
dc.subjectGenome analysisen
dc.subjectApache Sparken
dc.subjectHigh-performance data analyticsen
dc.subjectQA75 Electronic computers. Computer scienceen
dc.subjectQH301 Biologyen
dc.subjectQH426 Geneticsen
dc.subjectSDG 3 - Good Health and Well-beingen
dc.titleSparkFlow : towards high-performance data analytics for Spark-based genome analysisen
dc.typeConference itemen
dc.contributor.institutionUniversity of St Andrews. School of Computer Scienceen

This item appears in the following Collection(s)

Show simple item record