Distributed Computing in a Pandemic

A Review of Technologies available for Tackling COVID-19


The current COVID-19 global pandemic caused by the SARS-CoV-2 betacoronavirus has resulted in over a million deaths and is having a grave socio-economic impact, hence there is an urgency to find solutions to key research challenges. Much of this COVID-19 research depends on distributed computing. In this article, I review distributed architectures -- various types of clusters, grids and clouds -- that can be leveraged to perform these tasks at scale, at high-throughput, with a high degree of parallelism, and which can also be used to work collaboratively. High-performance computing (HPC) clusters will be used to carry out much of this work. Several bigdata processing tasks used in reducing the spread of SARS-CoV-2 require high-throughput approaches, and a variety of tools, which Hadoop and Spark offer, even using commodity hardware. Extremely large-scale COVID-19 research has also utilised some of the world's fastest supercomputers, such as IBM's SUMMIT -- for ensemble docking high-throughput screening against SARS-CoV-2 targets for drug-repurposing, and high-throughput gene analysis -- and Sentinel, an XPE-Cray based system used to explore natural products. Grid computing has facilitated the formation of the world's first Exascale grid computer. This has accelerated COVID-19 research in molecular dynamics simulations of SARS-CoV-2 spike protein interactions through massively-parallel computation and was performed with over 1 million volunteer computing devices using the Folding@home platform. Grids and clouds both can also be used for international collaboration by enabling access to important datasets and providing services that allow researchers to focus on research rather than on time-consuming data-management tasks.
  • Referencias
  • Cómo citar
  • Del mismo autor
  • Métricas
Abraham, M. J., Murtola, T., Schulz, R., Páll, S., Smith, J. C., Hess, B., and Lindahl, E., 2015. GROMACS: High performance molecular simulations through multi-level parallelism from laptops to supercomputers. SoftwareX, 1:19–25.

Agbehadji, I. E., Awuzie, B. O., Ngowi, A. B., and Millham, R. C., 2020. Review of Big Data Analytics, Artificial Intelligence and Nature-Inspired Computing Models towards Accurate Detection of COVID-19 Pandemic Cases and Contact Tracing. International journal of environmental research and public health, 17(15):5330.

Alnasir, J. J., 2021. Fifteen quick tips for success with HPC, ie, responsibly BASHing that Linux cluster. PLOS Computational Biology, 17(8):e1009207.

Alnasir, J. J. and Shanahan, H. P., 2020. The application of hadoop in structural bioinformatics. Briefings in bioinformatics, 21(1):96–105.

Amaro, R. E., Baudry, J., Chodera, J., Demir, Ö., McCammon, J. A., Miao, Y., and Smith, J. C., 2018. Ensemble docking in drug discovery. Biophysical journal, 114(10):2271–2278.

Amazon, 2016. Amazon EMR (Elastic MapReduce). https://aws.amazon.com/emr/. [Online; accessed 14-April-2019].

Amazon AWS, 2020a. COVID researchers can apply for free cloud services. https://aws.amazon.com/government-education/nonprofits/disaster-response/diagnostic-dev-initiative/. [Online; accessed 01-April-2020].

Amazon AWS, 2020b. Tech Against COVID: Rescale and Microsoft Azure donate supercomputing resources to help researchers combat global pandemic. https://partner.microsoft.com/ru-ru/case-studies/rescale. [Online; accessed 01-April-2020].

Anderson, D. P., 2004. Boinc: A system for public-resource computing and storage. In Fifth IEEE/ACM international workshop on grid computing, pages 4–10. IEEE.

Au, A., Curcin, V., Ghanem, M., Giannadakis, N., Guo, Y., Jafri, M., Osmond, M., Oleynikov, A., Rowe, A., Syed, J. et al., 2004. Why grid-based data mining matters? fighting natural disasters on the grid: from SARS to land slides. In UK e-science all-hands meeting (AHM 2004), Nottingham, UK, pages 121–126.

Ayris, P., Berthou, J.-Y., Bruce, R., Lindstaedt, S., Monreale, A., Mons, B., Murayama, Y., Södergård, C., Tochtermann, K., and Wilkinson, R., 2016. Realising the european open science cloud.

Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., and Warfield, A., 2003. Xen and the art of virtualization. In ACM SIGOPS operating systems review, volume 37, pages 164–177. ACM.

Beberg, A. L., Ensign, D. L., Jayachandran, G., Khaliq, S., and Pande, V. S., 2009. Folding@ home: Lessons from eight years of volunteer distributed computing. In Parallel & Distributed Processing, 2009. IPDPS 2009. IEEE International Symposium on, pages 1–8. IEEE.

Bhavsar, M. D. and Pradhan, S. N., 2009. Scavenging idle CPU cycles for creation of inexpensive supercomputing power. International Journal of Computer Theory and Engineering, 1(5):602.

Borgman, C. L., 2015. Big Data, little data, no data: Scholarship in the networked world. Mit Press.

Bragazzi, N. L., Dai, H., Damiani, G., Behzadifar, M., Martini, M., and Wu, J., 2020. How Big Data and Artificial Intelligence Can Help Better Manage the COVID-19 Pandemic. International Journal of Environmental Research and Public Health, 17(9):3176.

Byler, K., Landman, J., and Baudry, J., 2020. High Performance Computing Prediction of Potential Natural Product Inhibitors of SARS-CoV-2 Key Targets.

Coulouris, G. F., Dollimore, J., and Kindberg, T., 2005. Distributed systems: concepts and design. pearson education.

COVID-19 HPC Consortium, 2020. COVID researchers can apply for free cloud services. https://www.xsede.org/covid19-hpc-consortium. [Online; accessed 01-April-2020].

Dai, L., Gao, X., Guo, Y., Xiao, J., and Zhang, Z., 2012. Bioinformatics clouds for Big Data manipulation. Biology direct, 7(1):43.

Dong, E., Du, H., and Gardner, L., 2020. An interactive web-based dashboard to track COVID-19 in real time. The Lancet infectious diseases, 20(5):533–534.

Dudley, J. T., Pouliot, Y., Chen, R., Morgan, A. A., and Butte, A. J., 2010. Translational bioinformatics in the cloud: an affordable alternative. Genome medicine, 2(8):51.

EPSRC, 2016. An analysis of the impacts and outputs of investment in national HPC. https://epsrc.ukri.org/newsevents/pubs/impactofnationalhpc/. [Online; accessed 07-September-2020].

Ferreira, L., Berstis, V., Armstrong, J., Kendzierski, M., Neukoetter, A., Takagi, M., Bing-Wo, R., Amir, A., Murakawa, R., Hernandez, O. et al., 2003. Introduction to grid computing with globus. IBM redbooks, 9.

Ferretti, L., Wymant, C., Kendall, M., Zhao, L., Nurtay, A., Abeler-Dörner, L., Parker, M., Bonsall, D., and Fraser, C., 2020. Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science, 368(6491).

Fish, B., Kun, J., Lelkes, A. D., Reyzin, L., and Turán, G., 2015. On the computational complexity of mapreduce. In International Symposium on Distributed Computing, pages 1–15. Springer.

Foster, I., Zhao, Y., Raicu, I., and Lu, S., 2008. Cloud computing and grid computing 360-degree compared. In Grid Computing Environments Workshop, 2008. GCE’08, pages 1–10. Ieee.

Gagliardi, F., 2004. The EGEE European grid infrastructure project. In International Conference on High Performance Computing for Computational Science, pages 194–203. Springer.

Garvin, M. R., Alvarez, C., Miller, J. I., Prates, E. T., Walker, A. M., Amos, B. K., Mast, A. E., Justice, A., Aronow, B., and Jacobson, D., 2020. A mechanistic model and therapeutic interventions for COVID-19 involving a RAS-mediated bradykinin storm. Elife, 9:e59177.

Genomics England, 2014. 100,000 Genomes project by numbers. https://www.genomicsengland.co.uk/the-100000-genomes-project-by-numbers/. [Online; accessed 24-November-2019].

Genomics England, 2018. Secretary of State for Health and Social Care announces ambition to sequence 5 million genomes within five years. https://www.genomicsengland.co.uk/matt-hancock-announces-5-million-genomes-within-five-years/. [Online; accessed 30- October-2020].

Genomics England, 2020. Genomics England Research Environment - HPC (Helix) Migration 2020. https://cnfl.extge.co.uk/display/GERE/HPC+%28Helix%29+Migration+2020#HPC(Helix)Migration2020-ChangestothePhysicalComputeNodes. [Online; accessed 22-September-2020].

George, L., 2011. HBase: The Definitive Guide: Random Access to Your Planet-Size Data. «O’Reilly Media, Inc.».

Gunarathne, T., Wu, T.-L., Qiu, J., and Fox, G., 2010. MapReduce in the Clouds for Science. In Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, pages 565–572. IEEE.

Hassan, Y., Ogg, S., and Ge, H., 2018. Expression of novel fusion antiviral proteins ricin a chain-pokeweed antiviral proteins (RTA-PAPs) in Escherichia coli and their inhibition of protein synthesis and of hepatitis B virus in vitro. BMC biotechnology, 18(1):47.

Hassan, Y., Ogg, S., and Ge, H., 2020. Novel anti-SARS-CoV-2 mechanisms of fusion broad range anti-infective protein ricin A chain mutant-pokeweed antiviral protein 1 (RTAM-PAP1) in silico.

Hilgenfeld, R., 2014. From SARS to MERS: crystallographic studies on coronaviral proteases enable antiviral drug design. The FEBS journal, 281(18):4085–4096.

Hill, M. D., Jouppi, N. P., and Sohi, G., 2000. Readings in computer architecture. Gulf Professional Publishing.

HPC wire, 2020. Genomics England Scales Up Genomic Sequencing with Quantum ActiveScale Object Storage. https://www.hpcwire.com/off-the-wire/genomics-england-scales-up-genomic-sequencing-with-quantum-activescale-object-storage/. [Online; accessed 22-September-2020].

HPC Wire (2020). The Good Hope Net Project and Russian Supercomputer Achieve New Milestone in COVID-19 Fight. https://www.hpcwire.com/off-the-wire/the-good-hope-net-project-and-russian-supercomputer-achieve-new-milestone-in-covid-19-fight/. [Online; accessed 08-August-2021].

Huang, C., Wang, Y., Li, X., Ren, L., Zhao, J., Hu, Y., Zhang, L., Fan, G., Xu, J., Gu, X. et al., 2020a. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The lancet, 395(10223):497–506.

Huang, X., Li, Z., Jiang, Y., Li, X., and Porter, D., 2020b. Twitter reveals human mobility dynamics during the COVID-19 pandemic. PloS one, 15(11):e0241957.

Hussain, H., Malik, S. U. R., Hameed, A., Khan, S. U., Bickler, G., Min-Allah, N., Qureshi, M. B., Zhang, L., Yongji, W., Ghani, N. et al., 2013. A survey on resource allocation in high performance distributed computing systems. Parallel Computing, 39(11):709–736.

IBM (2020). IBM World Community Grid - about page. https://www.worldcommunitygrid.org/about_us/viewAboutUs.do. [Online; accessed 16-September-2020].

James Gallagher, BBC, 2014. DNA project ’to make UK world genetic research leader’. http://www.bbc.co.uk/news/health-28488313. [Online; accessed 21-January-2019].

Joshua, J., Alao, D., Okolie, S., and Awodele, O., 2013. Software Ecosystem: Features, Benefits and Challenges.

Kalil, A. C., 2020. Treating COVID-19—off-label drug use, compassionate use, and randomized clinical trials during pandemics. Jama, 323(19):1897–1898.

Kerner, S.M., 2018. IBM Unveils Summit, the World’s Fastest Supercomputer (For Now). https://www.serverwatch.com/server-news/ibm-unveils-summit-the-worlds-faster-supercomputer-for-now.html. [Online; accessed 07-September-2020].

Kissler, S. M., Tedijanto, C., Goldstein, E., Grad, Y. H., and Lipsitch, M., 2020. Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period. Science, 368(6493):860–868.

Kong, R., Wang, F., Zhang, J., Wang, F., and Chang, S., 2019. CoDockPP: A Multistage Approach for Global and Site-Specific Protein–Protein Docking. Journal of chemical information and modeling, 59(8):3556–3564.

Krieger, E. and Vriend, G., 2002. Models@ Home: distributed computing in bioinformatics using a screensaver based approach. Bioinformatics, 18(2):315–318.

Kuba, K., Imai, Y., Rao, S., Gao, H., Guo, F., Guan, B., Huan, Y., Yang, P., Zhang, Y., Deng, W. et al., 2005. A crucial role of angiotensin converting enzyme 2 (ACE2) in SARS coronavirus–induced lung injury. Nature medicine, 11(8):875–879.

Lake, M. A., 2020. What we know so far: COVID-19 current clinical knowledge and research. Clinical Medicine, 20(2):124.

Laney, D., 2001. 3D Data Management: Controlling Data Volume, Velocity, and Variety. Technical report, META Group.

Li, J. W.-H. and Vederas, J. C., 2009. Drug discovery and natural products: end of an era or an endless frontier? Science, 325(5937):161–165.

Li, W., Moore, M. J., Vasilieva, N., Sui, J., Wong, S. K., Berne, M. A., Somasundaran, M., Sullivan, J. L., Luzuriaga, K., Greenough, T. C. et al., 2003. Angiotensin-converting enzyme 2 is a functional receptor for the SARS coronavirus. Nature, 426(6965):450–454.

Lifebit, 2020. Lifebit Provides Free Cloud Operating System, Data Hosting & Analysis Tools to COVID-19 Researchers. https://blog.lifebit.ai/2020/03/30/lifebit-provides-free-cloud-operating-system-data-hosting-analysis-tools-to-covid-19-researchers [Online; accessed 01-April-2020].

Lu, H., Stratton, C. W., and Tang, Y.-W., 2020. Outbreak of pneumonia of unknown etiology in Wuhan, China: The mystery and the miracle. Journal of medical virology, 92(4):401–402.

Lyubimov, D. and Palumbo, A., 2016. Apache Mahout: Beyond MapReduce. CreateSpace Independent Publishing Platform.

Marx, V., 2015. The DNA of a nation. Nature, 524(7566):503–505.

Mell, P., Grance, T. et al., 2011. The NIST definition of cloud computing.

Meng, X.-Y., Zhang, H.-X., Mezei, M., and Cui, M., 2011. Molecular docking: a powerful approach for structure-based drug discovery. Current computer-aided drug design, 7(2):146–157.

Messerschmitt, D. G., Szyperski, C. et al., 2005. Software ecosystem: understanding an indispensable technology and industry. MIT Press Books, 1.

Miller, S., 2020. COVID researchers can apply for free cloud services. https://gcn.com/articles/2020/03/24/cloud-vendors-covid-research.aspx. [Online; accessed 09-September-2020].

Morris, G. M. and Lim-Wilby, M., 2008. Molecular docking. Molecular modeling of proteins, pages 365–382.

Moses, H., Dorsey, E. R., Matheson, D. H., and Thier, S. O., 2005. Financial anatomy of biomedical research. Jama, 294(11):1333–1342.

Navale, V. and Bourne, P. E., 2018. Cloud computing applications for biomedical science: A perspective. PLoS computational biology, 14(6):e1006144.

Nicola, M., Alsafi, Z., Sohrabi, C., Kerwan, A., Al-Jabir, A., Iosifidis, C., Agha, M., and Agha, R., 2020. The socio-economic implications of the coronavirus pandemic (COVID-19): A review. International journal of surgery (London, England), 78:185.

Novick, P. A., Ortiz, O. F., Poelman, J., Abdulhay, A. Y., and Pande, V. S., 2013. SWEETLEAD: an in silico database of approved drugs, regulated chemicals, and herbal isolates for computer-aided drug discovery. PLoS One, 8(11):e79568.

Olston, C., Reed, B., Srivastava, U., Kumar, R., and Tomkins, A., 2008. Pig latin: a not-so-foreign language for data processing. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1099–1110. ACM.

Organization, W. H. et al., 2020. WHO Director-General’s opening remarks at the media briefing on COVID-19- 11 March 2020.

Ossyra, J., Sedova, A., Tharrington, A., Noé, F., Clementi, C., and Smith, J. C., 2019. Porting adaptive ensemble molecular dynamics workflows to the summit supercomputer. In International Conference on High Performance Computing, pages 397–417. Springer.

Pairo-Castineira, E., Clohisey, S., Klaric, L., Bretherick, A., Rawlik, K., Parkinson, N., Pasko, D., Walker, S., Richmond, A., Fourman, M. H. et al., 2020. Genetic mechanisms of critical illness in Covid-19. medRxiv.

Perez, G. I. P. and Abadi, A. T. B., 2020. Ongoing Challenges Faced in the Global Control of COVID-19 Pandemic. Archives of Medical Research.

Pordes, R., Petravick, D., Kramer, B., Olson, D., Livny, M., Roy, A., Avery, P., Blackburn, K., Wenaus, T., Würthwein, F. et al., 2007. The open science grid. In Journal of Physics: Conference Series, volume 78, page 012057. IOP Publishing.

Rawlins, M. D., 2004. Cutting the cost of drug development? Nature reviews Drug discovery, 3(4):360–364.

Roche, J. A. and Roche, R., 2020. A hypothesized role for dysregulated bradykinin signaling in COVID-19 respiratory complications. The FASEB Journal.

Rowe, A., Kalaitzopoulos, D., Osmond, M., Ghanem, M., and Guo, Y., 2003. The discovery net system for high throughput bioinformatics. Bioinformatics, 19(suppl_1):i225–i231.

Savin, G., Shabanov, B., Telegin, P., and Baranov, A., 2019. Joint supercomputer center of the Russian Academy of Sciences: Present and future. Lobachevskii Journal of Mathematics, 40(11):1853–1862.

Schatz, M. C., Langmead, B., and Salzberg, S. L., 2010. Cloud computing and the DNA data race. Nature biotechnology, 28(7):691.

Schwede, T., Kopp, J., Guex, N., and Peitsch, M. C., 2003. SWISS-MODEL: an automated protein homology- modeling server. Nucleic acids research, 31(13):3381–3385.

Sciaba, A., Campana, S., Litmaath, M., Donno, F., Moscicki, J., Magini, N., Renshall, H., and Andreeva, J., 2010. Computing at the Petabyte scale with the WLCG. Technical report.

Shanahan, H. P., Owen, A. M., and Harrison, A. P., 2014. Bioinformatics on the cloud computing platform Azure. PloS one, 9(7):e102642.

Shanahan, J. G. and Dai, L., 2015. Large scale distributed data science using apache spark. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2323–2324. ACM.

Shipman, G. M., Woodall, T. S., Graham, R. L., Maccabe, A. B., and Bridges, P. G., 2006. Infiniband scalability in Open MPI. In Parallel and Distributed Processing Symposium, 2006. IPDPS 2006. 20th International, pages 10–pp. IEEE.

Siva, N., 2015. UK gears up to decode 100 000 genomes from NHS patients. The Lancet, 385(9963):103–104.

Smith, J. E. and Nair, R., 2005. The architecture of virtual machines. Computer, 38(5):32–38.

Smith, M. and Smith, J. C., 2020. Repurposing therapeutics for COVID-19: supercomputer-based docking to the SARS-CoV-2 viral spike protein and viral spike protein-human ACE2 interface.

Smith, T., 2020. IA Supercomputer Analyzed Covid-19 — and an Interesting New Theory Has Emerged. https://elemental.medium.com/a-supercomputer-analyzed-covid-19-and-an-interesting-new-theory-has-emerged-31cb8eba9d63. [Online; accessed 09-September-2020].

Sorokina, M. and Steinbeck, C., 2020. COlleCtion of Open NatUral producTs. http://doi.org/10.5281/zenodo.3778405. [Online; accessed 11-September-2020].

Sun, K., Chen, J., and Viboud, C., 2020. Early epidemiological analysis of the coronavirus disease 2019 outbreak based on crowdsourced data: a population-level observational study. The Lancet Digital Health.

Sun, M., Liu, S., Wei, X., Wan, S., Huang, M., Song, T., Lu, Y., Weng, X., Lin, Z., Chen, H. et al., 2021. Aptamer Blocking Strategy Inhibits SARS-CoV-2 Virus Infection. Angewandte Chemie, 133(18):10354–10360.

Taylor, R. C., 2010. An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics. BMC bioinformatics, 11(Suppl 12):S1.

The Good Hope Net (2020). The Good Hope Net project uses Russian supercomputer to develop treatment against coronavirus infection. https://thegoodhope.net. [Online; accessed 05-August-2021].

Thusoo, A., Sarma, J. S., Jain, N., Shao, Z., Chakka, P., Anthony, S., Liu, H., Wyckoff, P., and Murthy, R., 2009. Hive: a warehousing solution over a map-reduce framework. Proceedings of the VLDB Endowment, 2(2):1626–1629.

Vazhkudai, S. S., De Supinski, B. R., Bland, A. S., Geist, A., Sexton, J., Kahle, J., Zimmer, C. J., Atchley, S., Oral, S., Maxwell, D. E. et al., 2018. The design, deployment, and evaluation of the CORAL pre-exascale systems. In SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 661–672. IEEE.

Vilar, S., Cozza, G., and Moro, S., 2008. Medicinal chemistry and the molecular operating environment (MOE): application of QSAR and molecular docking to drug discovery. Current topics in medicinal chemistry, 8(18):1555–1572.

Wassenaar, T. A., Van Dijk, M., Loureiro-Ferreira, N., Van Der Schot, G., De Vries, S. J., Schmitz, C., Van Der Zwan, J., Boelens, R., Giachetti, A., Ferella, L. et al., 2012. WeNMR: structural biology on the grid. Journal of Grid Computing, 10(4):743–767.

WHO, 2020. Statement on the second meeting of the International Health Regulations (2005) Emergency Committee regarding the outbreak of novel coronavirus (2019-nCoV).

Wong, Z. S., Zhou, J., and Zhang, Q., 2019. Artificial intelligence for infectious disease big data analytics. Infection, disease & health, 24(1):44–48.

World Health Organisation (2020). Off-label use of medicines for COVID-19. https://www.who.int/publications/i/item/off-label-use-of-medicines-for-covid-19-scientific-brief. [Online; accessed 11- September-2020].

Younge, A. J., Henschel, R., Brown, J. T., Von Laszewski, G., Qiu, J., and Fox, G. C., 2011. Analysis of virtualization technologies for high performance computing environments. In Cloud Computing (CLOUD), 2011 IEEE International Conference on, pages 9–16. IEEE.

Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M. J., Shenker, S., and Stoica, I., 2012. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, pages 2–2. USENIX Association.

Zhang, D., Hu, M., and Ji, Q., 2020. Financial markets under the global pandemic of COVID-19. Finance Research Letters, page 101528.

Zhang, X., Wong, S. E., and Lightstone, F. C., 2013. Message passing interface and multithreading hybrid for parallel molecular docking of large databases on petascale high performance computing machines. Journal of computational chemistry, 34(11):915–927.

Zhang, Y., 2020. Initial genome release of novel coronavirus.

Zimmerman, M. I., Porter, J. R., Ward, M. D., Singh, S., Vithani, N., Meller, A., Mallimadugula, U. L., Kuhn, C. E., Borowsky, J. H., Wiewiora, R. P., Hurley, M. F. D., Harbison, A. M., Fogarty, C. A., Coffland, J. E., Fadda, E., Voelz, V. A., Chodera, J. D., & Bowman, G. R. (2020). SARS-CoV-2 Simulations Go Exascale to Capture Spike Opening and Reveal Cryptic Pockets Across the Proteome. BioRxiv, 2020.06.27.175430. https://doi.org/10.1101/2020.06.27.175430.
Alnasir, J. (2022). Distributed Computing in a Pandemic: A Review of Technologies available for Tackling COVID-19. ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, 11(1), 19–43. https://doi.org/10.14201/adcaij.27337


Download data is not yet available.