ARCHIVED: Project: FutureGrid: Experimental Test-bed Grid
Primary UITS contact: Gary Miksik
Last update: September 13, 2012
Description: FutureGrid is a distributed, high-performance test-bed that allows scientists to collaboratively create and test innovative approaches to parallel, grid, and cloud computing. The test-bed comprises distributed clusters of high-performance computing resources connected to a high-speed network and linked to TeraGrid, the NSF's national cyberinfrastructure for scientific research. The FutureGrid project is led by Indiana University and funded by a $10.1 million grant from the National Science Foundation.
Purpose: FutureGrid allows researchers to experiment at all levels, including customizing network protocols and experimenting with new middleware. By using virtual machines running on real hardware, FutureGrid enables scientists to have full control over their testing environments without interfering with other users.
Scientists can also pursue interactive research and international collaboration, creating unprecedented potential for scientific discovery and innovation as they explore new uses for cloud technologies in science and engineering. Additionally, FutureGrid's availability to university students familiarizes the next generation of workers with these paradigms and their applications.
Operational model: FutureGrid's operating model is different from both TeraGrid conventional clusters and commercial clouds, achieving flexibility by dynamically provisioning software onto "bare-metal" instead of loading images onto virtual machines. FutureGrid supports a growing Image Library that features platforms such as MPI, OpenMP, MapReduce (Hadoop, Dryad, Twister), gLite, Unicore, Globus, Xen, ScaleMP (distributed Shared Memory), Nimbus, Eucalyptus, OpenStack, OpenNebula, KVM, Windows, etc.
Organizations involved in FutureGrid:
- Indiana University (IU): Architecture, core software, user support, Knowledge Base, Eucalyptus support, OpenStack, OpenNebula, project/subaward management
- Purdue University (PU): High-throughput computing hardware
- San Diego Supercomputer Center (SDSC) at the University of California San Diego: Inca for monitoring resources
- University of Chicago/Argonne National Labs (UC/ANL): Nimbus
- University of Florida (UF): ViNe, virtual appliances; education and outreach
- University of Southern California (USC) Information Sciences: Pegasus for managing experiments
- University of Tennessee - Knoxville (UTK): Benchmarking; PAPI
- University of Texas/Texas Advanced Computing Center (TACC): Portal; experiment harness; HPC support
- University of Virginia (UV): UNICORE, Genesis, gLite
- Center for Information Services and GWT-TUD from Technischen Universitt Dresden (TUD): VAMPIR
Project team:
- Geoffrey Fox, PI, Indiana University
- Kate Keahey, co-PI, University of Chicago
- Jose Fortes, co-PI, University of Florida
- Andrew Grimshaw, co-PI, University of Virginia
- Warren Smith, co-PI, University of Texas/Texas Advanced Computing Center (TACC)
- Shava Smallen, University of California San Diego, San Diego Supercomputing Center (SDSC)
- Ewa Deelman, University of Southern California, Information Sciences Institute (ISI)
- Mats Rynge, University of Southern California, Information Sciences Institute (ISI)
- Renato Figueiredo, University of Florida
- Gregor von Laszewski, Indiana University, Software Lead
- Dave Hancock, Indiana University, UITS, Hardware and Networks
- Koji Tanaka, Indiana University, Systems Lead
- Tom Johnson, Indiana University, UITS, Network Engineer
- Jonathan Bolte, Indiana University, UITS, Knowledge Base
- Gary Miksik, Indiana University, Business/Project Manager
- Barbara O'Leary, Indiana University, Outreach Consultant and User Support
Project governance: FutureGrid is sponsored by the National Science Foundation under Grant No. 0910812 to Indiana University for "FutureGrid: An Experimental, High-Performance Grid Test-bed".
Compute hardware:
| Name | System type | # Nodes | # CPUs | Cores | Teraflops | Total RAM | Secondary Storage (TB) | Site |
|---|---|---|---|---|---|---|---|---|
| india | IBM iDataPlex | 128 | 256 | 1024 | 307211 | 3072 | 335 | IU |
| hotel | IBM iData Plex | 84 | 168 | 672 | 7 | 2016 | 120 | UC |
| sierra | IBM iDataPlex | 84 | 168 | 672 | 7 | 2688 | 96 | SDSC |
| foxtrot | IBM iDataPlex | 32 | 64 | 256 | 3 | 768 | 0 | UF |
| alamo | Dell PowerEdge | 96 | 192 | 768 | 8 | 1152 | 30 | TACC |
| xray | Cray XT5m | 1 | 168 | 672 | 6 | 1344 | 335 | IU |
| bravo | HP Proliant | 16 | 32 | 128 | 1.7 | 3072 | 192 | IU |
| delta | SuperMicro | 16 | 32 | 192 | TBD | 3072 | 144 | IU |
| Total | 457 | 1080 | 4384 | 43.7 | 17184 | 1252 |
Services offered:
| Services | India | Sierra | Hotel | Foxtrot | Alamo | Xray | Bravo | |
|---|---|---|---|---|---|---|---|---|
| myHadoop |  |
|
|
|||||
| Nimbus | |
|
|
|
||||
| Eucalyptus | |
|
||||||
| ViNe |  |
|
|
|
|
ViNe can be installed on the other resources via Nimbus |
||
| Genesis II | |
|
|
|
||||
| Unicore | |
|
|
|||||
| MPI | |
|
|
|
|
|
|
|
| OpenMP | |
|||||||
| ScaleMP | |
|||||||
| Ganglia | |
|
||||||
| Pegasus | |
|
|
|
|
Pegasus available via Nimbus and Eucalyptus images | ||
| Inca | |
|
|
|
|
|
||
| Portal |  |
|
|
|
|
|
Access to the resource is requested through the portal |
|
| PAPI | |
|||||||
| Vampir | ||||||||
| SAGA | |
|
|
FutureGrid portal: https://portal.futuregrid.org
All FutureGrid interactions are directly supported in the FutureGrid Portal.
In order for you to gain access to the FutureGrid resources, you need to follow three simple steps:
- Create a FutureGrid Portal account. Every FutureGrid user must
first have a Portal account.
- Apply for either a new FutureGrid project or join an existing
project. To apply for a new project, fill out the project creation
form. To join an existing project, ask the project lead or project
manager for that project to add you to the project using that same
form.
- Once you have been approved to work on a project, you will be able
to access the resources and services that your project has requested
and been authorized to use. Good places to start learning about how to
use FutureGrid are the tutorials, specifically:
- The Nimbus tutorial if you wish to use an IaaS-style cloud platform for creating and running virtual machines
- The Eucalyptus tutorial if you wish to explore IaaS-style usage with Eucalyptus
- The HPC tutorial if you wish to use FutureGrid clusters as a High Performance Computing cluster
Project highlights
Education:
- System Programming and Cloud Computing, Fresno State. Teaches system programming and cloud computing in different computing environments
- REU: Cloud Computing, Arkansas. Offers hands-on experience with FutureGrid tools and technologies
- Workshop: A Cloud View on Computing, Indiana School of Informatics and Computing (SoIC). Boot camp on MapReduce for faculty and graduate students from underserved ADMI institutions
- Topics on Systems: Distributed Systems, Indiana SoIC. Covers core computer science distributed system curricula (for 60 students)
Interoperability:
- SAGA, Louisiana State. Explores use of FutureGrid components for extensive portability and interoperability testing of Simple API for Grid Applications, and scale-up and scale-out experiments
- XSEDE/OGF Unicore and Genesis Grid endpoints tests for new US and European grids
Bio applications:
- Metagenomics Clustering, North Texas. Analyzes metagenomic data from samples collected from patients
- Genome Assembly, Indiana SoIC. De novo assembly of genomes and metagenomes from next-generation sequencing data
Non-bio applications:
- Physics: Higgs boson, Virginia, Matrix Element calculations representing production and decay mechanisms for Higgs and background processes
- Business Intelligence on MapReduce, Cal State - L.A. Market basket and customer analysis designed to execute MapReduce on Hadoop platform
Computer science:
- Data Transfer Throughput, Buffalo. End-to-end optimization of data transfer throughput over wide-area, high-speed networks
- Elastic Computing, Colorado. Tools and technologies to create elastic computing environments using IaaS clouds that adjust to changes in demand automatically and transparently
- The VIEW Project, Wayne State. Investigates Nimbus and Eucalyptus as cloud platforms for elastic workflow scheduling and resource provisioning
