{"response":{"award":[{"abstractText":"The University of California at San Diego will provide a ground-breaking new computing facility, Wildfire, that will be made available to the research community to both well established users of high end computing (HEC) and especially to new user communities that are less familiar with how HEC can advance their scientific and engineering goals. \r\nThe distinguishing features of Wildfire are:\r\n(i)\tDeliver 1.8-2.0 Petaflop/s of long sought capacity for the 98% of XSEDE jobs (50% of XSEDE core hours) that use fewer than 1,000 cores and also support larger jobs. The exact number will depend on the speed of the processor being delivered by Intel but cannot be less that 1.8 Petaflop/s.\r\n(ii)\tProvide 7 PB of Lustre-based Performance Storage at 200 GB/s bandwidth for both scratch and allocated storage as well as 6 PB of Durable Storage\r\n(iii)\tEnsure high throughput and responsiveness using allocation/scheduling using proven policies on earlier deployed systems such as Trestles and Gordon\r\n(iv)\tEstablish a rapid-access queue to provide new accounts within one day of the request\r\n(v)\tEnable community-supported custom software stacks via virtualization for communities that are unfamiliar with HPC environments. These virtual clusters will be able to perform at or near native InfinBand bandwidth/latency\r\n\r\nWildfire will provide novel approaches for resource allocation, scheduling, and user support, queues with quicker response for high-throughput computing, medium-term storage allocations, virtualized environments with customized software stack, dedicated allocations of physical/virtual machines, support for Science Gateways and bandwidth reservations on high-speed networks. Wildfire has been designed to efficiently serve the 98% of XSEDE jobs that need fewer than 1,000 cores, while also supporting larger jobs. The award leverages but also enhances the services available through the XSEDE project. \r\n\r\nThe Wildfire acquisition will work to increase the diversity of researchers able to effectively make use of advanced computational resources and establish a pipeline of potential users through virtualization, science gateways and educational activities focused on the undergraduate, graduate and post-graduate levels.","activeAwd":"false","agency":"NSF","awardAgencyCode":"4900","awardee":"UNIVERSITY OF CALIFORNIA, SAN DIEGO","awardeeAddress":"9500 GILMAN DR","awardeeCity":"LA JOLLA","awardeeCountryCode":"US","awardeeDistrict":"50","awardeeDistrictCode":"CA50","awardeeName":"University of California-San Diego","awardeePhone":"8585344896","awardeeStateCode":"CA","awardeeZipCode":"920930021","cfdaNumber":"47.070","coPDPI":["Amitava Majumdar majumdar@sdsc.edu","Robert Sinkovits rssinkovits@ucsd.edu","Shawn M Strande sstrande@sdsu.edu","Chaitanya K Baru (Former) baru@sdsc.edu","Philip M Papadopoulos (Former) ppapadopoulos@ucsd.edu","Richard L Moore (Former) rlm@sdsc.edu","Nancy R Wilkins-Diehr (Former) wilkinsn@sdsc.edu"],"date":"09/27/2013","dirAbbr":"CSE","divAbbr":"OAC","estimatedTotalAmt":"12000000","expDate":"07/31/2021","fundAgencyCode":"4900","fundProgramName":"XD-Extreme Digital, Innovative HPC","fundsObligated":["FY 2013 = $12,000,000.00","FY 2014 = $9,599,963.00","FY 2015 = $2,399,881.00","FY 2016 = $21,000.00","FY 2017 = $906,388.00","FY 2018 = $2,386,245.00"],"fundsObligatedAmt":"27313476","histAwd":"false","id":"1341698","initAmendmentDate":"09/27/2013","jrnl":[{"artPageNum":"78","artTitl":"Medial temporal lobe white matter pathway variability is associated with individual differences in episodic memory in cognitively normal older adults.","auth":"Alm, K., Faria, A., Moghekar, A., Pettigrew, C., Soldan, A., et al.","dgtlObjId":"DOI:10.1016/j.neurobiolaging.2019.11.011","jrnlTitl":"Neurobiology of Aging","jrnlVol":"87","jrnlYr":"2020"},{"artPageNum":"12","artTitl":"Shape analysis of hypertrophic and hypertensive heart disease using MRI-based 3D surface models of left ventricular geometry.","auth":"Ardekani, S., Jain, S., Sanzi, A., Corona-Villalobos, C., Abraham, T., et al.","dgtlObjId":"10.1016/j.media.2015.11.004","jrnlTitl":"Medical image analysis","jrnlVol":"29","jrnlYr":"2016"},{"artTitl":"Very High-Order Upwind Multi-Layer Compact (MLC) Schemes with Spectral-Like Resolution II: Two-Dimensional Case","auth":"Bai, Z., and X. Zhong","dgtlObjId":"doi:10.2514/6.2019-1398","jrnlTitl":"AIAA Scitech 2019 Forum","jrnlYr":"2019"},{"artTitl":"TreeCluster: Clustering biological sequences using phylogenetic trees","auth":"Balaban, M., N. Moshiri, U. Mai, X. Jia, and S. Mirarab","dgtlObjId":"doi:10.1371/journal.pone.0221068","jrnlTitl":"S. Bozdag, PLOS ONE","jrnlYr":"2019"},{"artTitl":"TreeCluster: Clustering biologicalsequences using phylogenetic trees","auth":"Balaban, M., N. Moshiri, U. Mai, X. Jia, and S. Mirarab","dgtlObjId":"doi:10.1371/journal.pone.0221068","jrnlTitl":"PLOS ONE","jrnlYr":"2019"},{"artPageNum":"1743","artTitl":"Primates, Lice and Bacteria: Speciationand Genome Evolution in the Symbionts of Hominid Lice","auth":"Boyd, B. M., J. M. Allen, N.-P. Nguyen, P. Vachaspati, Z. S. Quicksall, T. Warnow, L.Mugisha, K. P. Johnson, and D. L. Reed","dgtlObjId":"10.1093/molbev/msx117","jrnlTitl":"Molecular Biology and Evolution","jrnlVol":"34","jrnlYr":"2017"},{"artTitl":"Using Computational Methods to Teach Chemical Principles: Overview, Using Computational Methods To Teach Chemical Principles","auth":"Grushow, A., and M. S. Reeves","dgtlObjId":"10.1021/bk-2019-1312.ch001","jrnlTitl":"ACS Symposium Series","jrnlYr":"2019"},{"artTitl":"HPCmatlab: A Framework for Fast Prototyping of Parallel Applications in Matlab","auth":"Guo, X., M. Dave, and M. Sayeed","dgtlObjId":"10.1016/j.procs.2016.05.467","jrnlTitl":"Procedia Computer Science","jrnlYr":"2016"},{"artTitl":"Direct Numerical Simulation of Hypersonic Flow over a Blunt Cone with Axisymmetric Isolated Roughness","auth":"Haley, C. L., and X. Zhong","dgtlObjId":"10.2514/6.2017-4514","jrnlTitl":"47th AIAA Fluid Dynamics Conference","jrnlYr":"2017"},{"artTitl":"Mode F/S Wave Packet Interference And Acoustic-like Emissions in a Mach 8 Flow Over a Cone","auth":"Haley, C. L., and X. Zhong","dgtlObjId":"doi:10.2514/6.2020- 1579","jrnlTitl":"AIAA Scitech 2020 Forum","jrnlYr":"2020"},{"artTitl":"Numerical Study of Hypersonic Boundary Layer Receptivity over a Blunt Cone to Freestream Pulse Disturbances.","auth":"He, S., Zhong, X","dgtlObjId":"DOI:10.2514/6.2020-2996.","jrnlTitl":"AIAA AVIATION 2020 FORUM. http://dx.doi.org/10.2514/6.2020-2996","jrnlYr":"2020"},{"artTitl":"Readily Available Primary Aminoboranes as Powerful Reagents for Aldimine Synthesis","auth":"Junor, G. P., E. A. Romero, X. Chen, R. Jazzar, and G. Bertrand","dgtlObjId":"10.1002/anie.201814081","jrnlTitl":"Angewandte Chemie International Edition","jrnlYr":"2019"},{"artTitl":"Trapping and Structural Characterization of the XNO2·NO3? (X = Cl, Br, I) Exit Channel Complexes in the Water-Mediated X? + N2O5 Reactions with Cryogenic Vibrational Spectroscopy","auth":"Kelleher, P. J., F. S. Menges, J. W. DePalma, J. K. Denton, M. A. Johnson, G. H. Weddle, B. Hirshberg, and R. B. Gerber","dgtlObjId":"doi:10.1021/acs.jpclett.7b02120","jrnlTitl":"The Journal of Physical Chemistry Letters","jrnlYr":"2017"},{"artTitl":"TreeShrink: fast and accurate detection of outlier long branches in collections of phylogenetic trees","auth":"Mai, U., and S. Mirarab","dgtlObjId":"10.1186/s12864- 018-4620-2","jrnlTitl":"BMC Genomics","jrnlYr":"2018"},{"artTitl":"TreeShrink: fast and accurate detection of outlier long branches in collections of phylogenetic trees","auth":"Mai, U., and S. Mirarab","dgtlObjId":"doi:10.1186/s12864- 018-4620-2","jrnlTitl":"BMC Genomics","jrnlYr":"2018"},{"artPageNum":"2376","artTitl":"Interpreting the ionization sequence in AGN emission-line spectra","auth":"Richardson, C. T., J. T. Allen, J. A. Baldwin, P. C. Hewett, and G. J. Ferland","dgtlObjId":"10.1093/mnras/stt2056","jrnlTitl":"Monthly Notices of the Royal AstronomicalSociety","jrnlVol":"437","jrnlYr":"2013"},{"artPageNum":"2376","artTitl":"Interpretingthe ionization sequence in AGN emission-line spectra","auth":"Richardson, C. T., J. T. Allen, J. A. Baldwin, P. C. Hewett, and G. J. Ferland","dgtlObjId":"doi:10.1093/mnras/stt2056","jrnlTitl":"Monthly Notices of the RoyalAstronomical Society","jrnlVol":"437","jrnlYr":"2013"},{"artPageNum":"17","artTitl":"Anchoring quartet-based phylogenetic distances and applications to species tree reconstruction","auth":"Sayyari, E., and S. Mirarab","dgtlObjId":"10.1186/s12864-016-3098-z","jrnlTitl":"BMC Genomics","jrnlYr":"2016"},{"artTitl":"Anchoring quartet?based phylogenetic distances and applications to species tree reconstruction","auth":"Sayyari, E., S. Mirarab","dgtlObjId":"10.1186/s12864?016?3098? z","jrnlTitl":"BMC Genomics","jrnlYr":"2016"},{"artTitl":"Expanding the Computational Anatomy Gateway from clinical imaging to basic neuroscience research, Proceedingsof the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning)","auth":"Tward, D., A. Kolasny, F. Khan, J. Troncoso, and M. Miller","dgtlObjId":"doi:10.1145/3332186.3332217","jrnlTitl":"PEARC 2019","jrnlYr":"2019"},{"artTitl":"On the complexity of human neuroanatomy at the millimeter morphome scale: Developing codes and characterizing entropy indexed to spatial scale.","auth":"Tward, D., Miller, M","jrnlTitl":"Frontiers in neuroscience","jrnlYr":"2017"}],"latestAmendmentDate":"02/09/2021","managingPec":"761900","orgCodeDir":"05000000","orgCodeDiv":"05090000","orgLongName":"Directorate for Computer and Information Science and Engineering","orgLongName2":"Office of Advanced Cyberinfrastructure (OAC)","orgUrl":"https://www.nsf.gov/div/index.jsp?div=OAC","parentUeiNumber":"","pdPIName":"Michael L Norman","perfAddress":"9500 Gilman Drive","perfCity":"San Diego","perfCountryCode":"US","perfDistrict":"50","perfDistrictCode":"CA50","perfLocation":"University of California-San Diego","perfStateCode":"CA","perfZipCode":"920930934","pi":["Michael L Norman mlnorman@ucsd.edu"],"piEmail":"mlnorman@ucsd.edu","piFirstName":"Michael","piId":"000151265","piLastName":"Norman","piMiddeInitial":"L","poEmail":"edwalker@nsf.gov","poName":"Edward Walker","poPhone":"7032924863","primaryProgram":["01001314DB NSF RESEARCH & RELATED ACTIVIT","01001415DB NSF RESEARCH & RELATED ACTIVIT","01001516DB NSF RESEARCH & RELATED ACTIVIT","01001617DB NSF RESEARCH & RELATED ACTIVIT","01001718DB NSF RESEARCH & RELATED ACTIVIT","01001819DB NSF RESEARCH & RELATED ACTIVIT"],"progEleCode":"747600, 761900","program":"EQUIPMENT ACQUISITIONS, REU SUPP-Res Exp for Ugrd Supp","progRefCode":"7619, 9251","projectOutComesReport":"
\n

The San Diego Supercomputer Center at the University of California, San Diego deployed the Comet supercomputer as a natioanl resource in 2015. It was operated for allocated access by academic researchers, educators, and students through the NSF XSEDE project from May 2015 to July 2021. Following its decommissioning as an NSF-funded resource, Comet transitioned to a resource for the Center for Western Weather and Water Extremes (CW3E), a research and service project of the Scripps Institution of Oceanography. During its 75 months of operation as an XSEDE resource, Comet ran over 28 million jobs; provided over 2 billion core-hours and 13 million GPU-hours of compute time; served over 100,000 unique users, most of whom gained access via a science gateway rather than the command line; enabled publication of over 2,000 scientific papers; and supported research across virtually every domain of science and engineering.

\n

Comet has a peak speed of 2.8 Pflop/s delivered by 48,784 cores in 1,944 compute nodes, each with two, 12-core Intel Haswell processors and 128 GB DRAM. It also has 72 GPU nodes, half with 4 NVIDIA K80s and half with 4 NVIDIA P100s, plus 4 large-memory (1.5 TB) nodes. Like its Gordon predecessor, Comet features a large amount of flash memory via solid-state discs on every compute and GPU node. SDSC designed Comet in collaboration with vendor partners Dell, Intel, NVIDIA, Mellanox, and Aeon Computing.

\n

Comet was designed explicitly to serve the long tail of science, defined as the large number of researchers who require only modest numbers of cores or GPUs. Such users also benefited from Comet's optimized scheduling and allocations policies that lowered the barrier for accessing a complex high-performance computer. The design incorporated several significant technology and policy innovations:

\n

-A heterogenous architecture of CPUs, GPUs, large-memory nodes, along with a rich storage hierarchy, supported a broad range of science and engineering research.

\n

-One compute rack of 2,016 cores, connected by an FDR InfiniBand, non-blocking fat tree, supported a wide range of job sizes, from single-core to modest-scale, fully-coupled applications.

\n

-Virtual Cluster (VC) software, developed by SDSC in partnership with collaborators at Indiana University, provided a low-overhead virtualization layer that allowed customized software to run side-by-side with the standard cluster software stack.

\n

-Restricting the allocation limit of an individual PI to 10M core-hours allowed Comet to support more projects. A higher limit of 20M core-hours for science gateways provided access for many more users without the overhead of requesting their own allocation.

\n

Comet's Virtual Cluster interface was used by researchers from the Laser Interferometer Gravitational-Wave Observatory (LIGO) in support of the confirmation of the landmark detection of gravitational waves as hypothesized by Albert Einstein over 100 years ago. LIGO researchers consumed nearly 630,000 hours of computational time on Comet via the Open Science Grid (OSG) using a VC that supported the direct integration of OSG's high-throughput scheduler into Comet's batch scheduler. Comet also became one of the first NSF national resources to use Singularity, which allowed users to run containerized application software that would otherwise not be feasible with a standard cluster software stack.

\n

Comet set out to reach 10,000 unique users over its lifetime, a goal that was achieved within the first year of operations. Notable science gateways included CIPRES, I-TASSER, and the Neuroscience Gateway. Between these and the other gateways on Comet, over 100,000 unique users accessed Comet to study a wide range of physical, chemical, and biological systems.

\n

During its lifetime, Comet became a primary source of GPUs for the community. A research team led by UCSD's Rommie Amaro and Arvind Ramanathan, a computational biologist at Argonne National Laboratory, explored the movement of SARS-CoV-2's spike protein to understand how it behaves and gains access to the human cell. Using Comet's GPU resources as part of the scaling work, the team built a workflow based on artificial intelligence (AI) to more efficiently simulate the spike protein. The work led to a special Gordon Bell Award at the Annual Supercomputing Conference. In 2020, Comet joined the COVID-19 HPC Consortium, adding resources to help understand the spread of COVID-19 and help search for treatments and vaccines. 

\n

Outreach activities exposed thousands of researchers, educators, and students to the benefits of Comet's unique features and ease of use. SDSC staff hosted tutorials at scientific meetings, workshops at SDSC and on other university campuses, and annual summer institutes. Following travel constraints due to COVID-19, programs were conducted virtually. SDSC used that opportunity to improve remote training processes and tools, ultimately increasing participation rates above those seen before the pandemic. In the final years of service, there was a marked increase in the interest in machine learning and AI. Targeted outreach to meet this demand resulted in a body of training materials now being used with SDSC's Expanse system.


\n

\n\t\t\t\t \tLast Modified: 11/26/2021
\n\t\t\t\t\tModified by: Michael L Norman

\n
\n
\n
","publicAccessMandate":"0","publicationResearch":["Neurobiology of Aging~2020~87~Alm, K., Faria, A., Moghekar, A., Pettigrew, C., Soldan, A., et al.~DOI:10.1016/j.neurobiolaging.2019.11.011~78~Medial temporal lobe white matter pathway variability is associated with individual differences in episodic memory in cognitively normal older adults.~2020-12-18 17:15:37.61","Medical image analysis~2016~29~Ardekani, S., Jain, S., Sanzi, A., Corona-Villalobos, C., Abraham, T., et al.~10.1016/j.media.2015.11.004~12~Shape analysis of hypertrophic and hypertensive heart disease using MRI-based 3D surface models of left ventricular geometry.~2016-12-20 15:24:34.983","AIAA Scitech 2019 Forum~2019~Bai, Z., and X. Zhong~doi:10.2514/6.2019-1398~Very High-Order Upwind Multi-Layer Compact (MLC) Schemes with Spectral-Like Resolution II: Two-Dimensional Case~2019-11-27 15:11:45.57","S. Bozdag, PLOS ONE~2019~Balaban, M., N. Moshiri, U. Mai, X. Jia, and S. Mirarab~doi:10.1371/journal.pone.0221068~TreeCluster: Clustering biological sequences using phylogenetic trees~2020-09-17 16:47:07.66","PLOS ONE~2019~Balaban, M., N. Moshiri, U. Mai, X. Jia, and S. Mirarab~doi:10.1371/journal.pone.0221068~TreeCluster: Clustering biologicalsequences using phylogenetic trees~2020-02-24 17:21:29.653","Molecular Biology and Evolution~2017~34~Boyd, B. M., J. M. Allen, N.-P. Nguyen, P. Vachaspati, Z. S. Quicksall, T. Warnow, L.Mugisha, K. P. Johnson, and D. L. Reed~10.1093/molbev/msx117~1743~Primates, Lice and Bacteria: Speciationand Genome Evolution in the Symbionts of Hominid Lice~2018-08-07 14:43:32.423","ACS Symposium Series~2019~Grushow, A., and M. S. Reeves~10.1021/bk-2019-1312.ch001~Using Computational Methods to Teach Chemical Principles: Overview, Using Computational Methods To Teach Chemical Principles~2019-08-03 13:48:15.84","Procedia Computer Science~2016~Guo, X., M. Dave, and M. Sayeed~10.1016/j.procs.2016.05.467~HPCmatlab: A Framework for Fast Prototyping of Parallel Applications in Matlab~2017-08-15 16:25:29.313","47th AIAA Fluid Dynamics Conference~2017~Haley, C. L., and X. Zhong~10.2514/6.2017-4514~Direct Numerical Simulation of Hypersonic Flow over a Blunt Cone with Axisymmetric Isolated Roughness~2018-12-21 14:58:06.736","AIAA Scitech 2020 Forum~2020~Haley, C. L., and X. Zhong~doi:10.2514/6.2020- 1579~Mode F/S Wave Packet Interference And Acoustic-like Emissions in a Mach 8 Flow Over a Cone~2020-08-21 13:28:31.513","AIAA AVIATION 2020 FORUM. http://dx.doi.org/10.2514/6.2020-2996~2020~He, S., Zhong, X~DOI:10.2514/6.2020-2996.~Numerical Study of Hypersonic Boundary Layer Receptivity over a Blunt Cone to Freestream Pulse Disturbances.~2021-05-20 18:59:14.073","Angewandte Chemie International Edition~2019~Junor, G. P., E. A. Romero, X. Chen, R. Jazzar, and G. Bertrand~10.1002/anie.201814081~Readily Available Primary Aminoboranes as Powerful Reagents for Aldimine Synthesis~2019-05-17 15:24:56.423","The Journal of Physical Chemistry Letters~2017~Kelleher, P. J., F. S. Menges, J. W. DePalma, J. K. Denton, M. A. Johnson, G. H. Weddle, B. Hirshberg, and R. B. Gerber~doi:10.1021/acs.jpclett.7b02120~Trapping and Structural Characterization of the XNO2·NO3? (X = Cl, Br, I) Exit Channel Complexes in the Water-Mediated X? + N2O5 Reactions with Cryogenic Vibrational Spectroscopy~2018-09-26 23:48:03.17","BMC Genomics~2018~Mai, U., and S. Mirarab~10.1186/s12864- 018-4620-2~TreeShrink: fast and accurate detection of outlier long branches in collections of phylogenetic trees~2019-04-09 18:07:29.433","BMC Genomics~2018~Mai, U., and S. Mirarab~doi:10.1186/s12864- 018-4620-2~TreeShrink: fast and accurate detection of outlier long branches in collections of phylogenetic trees~2019-09-26 18:17:47.326","Monthly Notices of the Royal AstronomicalSociety~2013~437~Richardson, C. T., J. T. Allen, J. A. Baldwin, P. C. Hewett, and G. J. Ferland~10.1093/mnras/stt2056~2376~Interpreting the ionization sequence in AGN emission-line spectra~2016-02-22 07:35:49.613","Monthly Notices of the RoyalAstronomical Society~2013~437~Richardson, C. T., J. T. Allen, J. A. Baldwin, P. C. Hewett, and G. J. Ferland~doi:10.1093/mnras/stt2056~2376~Interpretingthe ionization sequence in AGN emission-line spectra~2016-09-28 14:08:40.92","BMC Genomics~2016~Sayyari, E., and S. Mirarab~10.1186/s12864-016-3098-z~17~Anchoring quartet-based phylogenetic distances and applications to species tree reconstruction~2017-03-04 19:48:03.846","BMC Genomics~2016~Sayyari, E., S. Mirarab~10.1186/s12864?016?3098? z~Anchoring quartet?based phylogenetic distances and applications to species tree reconstruction~2017-09-26 21:16:31.776","PEARC 2019~2019~Tward, D., A. Kolasny, F. Khan, J. Troncoso, and M. Miller~doi:10.1145/3332186.3332217~Expanding the Computational Anatomy Gateway from clinical imaging to basic neuroscience research, Proceedingsof the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning)~2020-05-14 16:16:13.256","Frontiers in neuroscience~2017~Tward, D., Miller, M~On the complexity of human neuroanatomy at the millimeter morphome scale: Developing codes and characterizing entropy indexed to spatial scale.~2018-06-18 12:57:38.453"],"startDate":"10/01/2013","title":"Gateways to Discovery: Cyberinfrastructure for the Long Tail of Science","transType":"Cooperative Agreement","ueiNumber":"UYTTZT6G9DT1"}],"metadata":{"offset":0,"rpp":25,"totalCount":1}}}