Indiana University
  •  
  •  
  •  

ARCHIVED: Project: FutureGrid: Experimental Test-bed Grid

Primary UITS contact: Gary Miksik

Last update: September 13, 2012

Description: FutureGrid is a distributed, high-performance test-bed that allows scientists to collaboratively create and test innovative approaches to parallel, grid, and cloud computing. The test-bed comprises distributed clusters of high-performance computing resources connected to a high-speed network and linked to TeraGrid, the NSF's national cyberinfrastructure for scientific research. The FutureGrid project is led by Indiana University and funded by a $10.1 million grant from the National Science Foundation.

Purpose: FutureGrid allows researchers to experiment at all levels, including customizing network protocols and experimenting with new middleware. By using virtual machines running on real hardware, FutureGrid enables scientists to have full control over their testing environments without interfering with other users.

Scientists can also pursue interactive research and international collaboration, creating unprecedented potential for scientific discovery and innovation as they explore new uses for cloud technologies in science and engineering. Additionally, FutureGrid's availability to university students familiarizes the next generation of workers with these paradigms and their applications.

Operational model: FutureGrid's operating model is different from both TeraGrid conventional clusters and commercial clouds, achieving flexibility by dynamically provisioning software onto "bare-metal" instead of loading images onto virtual machines. FutureGrid supports a growing Image Library that features platforms such as MPI, OpenMP, MapReduce (Hadoop, Dryad, Twister), gLite, Unicore, Globus, Xen, ScaleMP (distributed Shared Memory), Nimbus, Eucalyptus, OpenStack, OpenNebula, KVM, Windows, etc.

Organizations involved in FutureGrid:

  • Indiana University (IU): Architecture, core software, user support, Knowledge Base, Eucalyptus support, OpenStack, OpenNebula, project/subaward management
  • Purdue University (PU): High-throughput computing hardware
  • San Diego Supercomputer Center (SDSC) at the University of California San Diego: Inca for monitoring resources
  • University of Chicago/Argonne National Labs (UC/ANL): Nimbus
  • University of Florida (UF): ViNe, virtual appliances; education and outreach
  • University of Southern California (USC) Information Sciences: Pegasus for managing experiments
  • University of Tennessee - Knoxville (UTK): Benchmarking; PAPI
  • University of Texas/Texas Advanced Computing Center (TACC): Portal; experiment harness; HPC support
  • University of Virginia (UV): UNICORE, Genesis, gLite
  • Center for Information Services and GWT-TUD from Technischen Universitt Dresden (TUD): VAMPIR

Project team:

  • Geoffrey Fox, PI, Indiana University
  • Kate Keahey, co-PI, University of Chicago
  • Jose Fortes, co-PI, University of Florida
  • Andrew Grimshaw, co-PI, University of Virginia
  • Warren Smith, co-PI, University of Texas/Texas Advanced Computing Center (TACC)
  • Shava Smallen, University of California San Diego, San Diego Supercomputing Center (SDSC)
  • Ewa Deelman, University of Southern California, Information Sciences Institute (ISI)
  • Mats Rynge, University of Southern California, Information Sciences Institute (ISI)
  • Renato Figueiredo, University of Florida
  • Gregor von Laszewski, Indiana University, Software Lead
  • Dave Hancock, Indiana University, UITS, Hardware and Networks
  • Koji Tanaka, Indiana University, Systems Lead
  • Tom Johnson, Indiana University, UITS, Network Engineer
  • Jonathan Bolte, Indiana University, UITS, Knowledge Base
  • Gary Miksik, Indiana University, Business/Project Manager
  • Barbara O'Leary, Indiana University, Outreach Consultant and User Support

Project governance: FutureGrid is sponsored by the National Science Foundation under Grant No. 0910812 to Indiana University for "FutureGrid: An Experimental, High-Performance Grid Test-bed".

Compute hardware:

Name System type # Nodes # CPUs Cores TeraFLOPS Total RAM Secondary Storage (TB) Site
india IBM iDataPlex 128 256 1024 307211 3072 335 IU
hotel IBM iData Plex 84 168 672 7 2016 120 UC
sierra IBM iDataPlex 84 168 672 7 2688 96 SDSC
foxtrot IBM iDataPlex 32 64 256 3 768 0 UF
alamo Dell PowerEdge 96 192 768 8 1152 30 TACC
xray Cray XT5m 1 168 672 6 1344 335 IU
bravo HP Proliant 16 32 128 1.7 3072 192 IU
delta SuperMicro 16 32 192 TBD 3072 144 IU
Total 457 1080 4384 43.7 17184 1252

Services offered:

Services India Sierra Hotel Foxtrot Alamo Xray Bravo
myHadoop
Nimbus
Eucalyptus
ViNe
ViNe can be installed on the other resources via Nimbus
Genesis II
Unicore
MPI
OpenMP
ScaleMP
Ganglia
Pegasus
Pegasus available via Nimbus and Eucalyptus images
Inca
Portal
Access to the resource is requested through the portal
PAPI
Vampir
SAGA

FutureGrid portal: https://portal.futuregrid.org

All FutureGrid interactions are directly supported in the FutureGrid Portal.

In order for you to gain access to the FutureGrid resources, you need to follow three simple steps:

  1. Create a FutureGrid Portal account. Every FutureGrid user must first have a Portal account.

  2. Apply for either a new FutureGrid project or join an existing project. To apply for a new project, fill out the project creation form. To join an existing project, ask the project lead or project manager for that project to add you to the project using that same form.

  3. Once you have been approved to work on a project, you will be able to access the resources and services that your project has requested and been authorized to use. Good places to start learning about how to use FutureGrid are the tutorials, specifically:

    • The Nimbus tutorial if you wish to use an IaaS-style cloud platform for creating and running virtual machines
    • The Eucalyptus tutorial if you wish to explore IaaS-style usage with Eucalyptus
    • The HPC tutorial if you wish to use FutureGrid clusters as a High Performance Computing cluster

Project highlights

Education:

  • System Programming and Cloud Computing, Fresno State. Teaches system programming and cloud computing in different computing environments
  • REU: Cloud Computing, Arkansas. Offers hands-on experience with FutureGrid tools and technologies
  • Workshop: A Cloud View on Computing, Indiana School of Informatics and Computing (SoIC). Boot camp on MapReduce for faculty and graduate students from underserved ADMI institutions
  • Topics on Systems: Distributed Systems, Indiana SoIC. Covers core computer science distributed system curricula (for 60 students)

Interoperability:

  • SAGA, Louisiana State. Explores use of FutureGrid components for extensive portability and interoperability testing of Simple API for Grid Applications, and scale-up and scale-out experiments
  • XSEDE/OGF Unicore and Genesis Grid endpoints tests for new US and European grids

Bio applications:

  • Metagenomics Clustering, North Texas. Analyzes metagenomic data from samples collected from patients
  • Genome Assembly, Indiana SoIC. De novo assembly of genomes and metagenomes from next-generation sequencing data

Non-bio applications:

  • Physics: Higgs boson, Virginia, Matrix Element calculations representing production and decay mechanisms for Higgs and background processes
  • Business Intelligence on MapReduce, Cal State - L.A. Market basket and customer analysis designed to execute MapReduce on Hadoop platform

Computer science:

  • Data Transfer Throughput, Buffalo. End-to-end optimization of data transfer throughput over wide-area, high-speed networks
  • Elastic Computing, Colorado. Tools and technologies to create elastic computing environments using IaaS clouds that adjust to changes in demand automatically and transparently
  • The VIEW Project, Wayne State. Investigates Nimbus and Eucalyptus as cloud platforms for elastic workflow scheduling and resource provisioning