Indiana University
  •  
  •  
  •  

Disk space on IU's research systems

On this page:

General information

Home directories on the research systems

Following is the naming convention for home directory paths on the UITS research systems:

/N/u/username/BigRed Big Red
/N/u/username/Quarry Quarry
/N/u/username/RDC Research Database Complex (RDC)

When you log into any of these systems, you can access your home directory from any of the other systems by changing to the parent directory (/N/u/username), and then moving down into the desired system home directory, for example:

[jdoe@bh2 Quarry]$ pwd /N/u/jdoe/Quarry [jdoe@bh2 Quarry]$ cd .. [jdoe@bh2 jdoe]$ ls Quarry RDC [jdoe@bh2 jdoe]$ cd RDC [jdoe@bh2 RDC]$

User home directories reside on the IBM N5500 NAS storage device with disk quotas of 10 GB per user. Note that this quota will be shared by your Big Red, Quarry, and RDC accounts, if you have accounts on those systems. If you need additional disk space, email the High Performance Systems group. The Scholarly Data Archive (SDA) is also available if you need more storage.

Back to top

Checking your quota usage

To determine your current disk quota usage, enter quota. You will see output similar to the following:

username@Quarry: ~> quota Disk quotas for user username (uid 123456): Filesystem blocks quota limit grace files quota limit grace iupui-nas2:/vol/hd03 3468288 10137600 10485760 21216 4294967295 4294967295

"Filesystem" is the name of the file system where your home directory resides, and "blocks" is the number of 1 KB blocks you are currently using. In this example, over 3.4 GB have been used out of a quota of 10 GB. The limit signifies the absolute hard limit of disk space you can use during the default grace period (seven days). If you exceed your quota and do not reduce your disk space usage to within your quota, after the grace period you won't be able to use more than the quota. If you reach your quota while a job is running, the job will continue to run, but no output files will be appended to or created.

Back to top

Scratch space

Scratch disk space is available locally on each node in /tmp and on an additional large scratch file system shared by all nodes within the system.

Scratch file systems are not backed up to tape; data purged from the scratch file systems cannot be recovered. No disk quotas are enforced on these file systems. You are responsible for making your own copies of the data in case of a disk failure or the need to free space.

  • Local scratch space: /tmp is a relatively small local file system; many programs use /tmp by default for temporary work space. You can redirect files from /tmp to other scratch disk space by setting the environment variable TMPDIR. Files in /tmp are automatically deleted once they are 24 hours old.

    If your batch jobs encounter an error when trying to write to or read from the scratch file systems, a subdirectory with your username probably does not exist. Before reading from or writing to scratch file systems, check that your scratch subdirectory still exists, and create the subdirectory if necessary.

  • Common scratch space: Common scratch disk space refers to disk space that is available to all users, and accessible to multiple systems. It is intended for short-term storage (e.g., working space for running programs, and temporary storage for program output), not online archival space. Files in common scratch space are purged automatically based on age to free up space, and are not backed up. For details, see the information below for specific systems.

Back to top

Big Red

  • Home directory: Your Big Red home directory disk space is allocated on the IBM N5500 NAS storage device. You have a 10 GB disk quota, which is shared with Quarry and the RDC (if you have accounts on those systems).

  • Local scratch: Scratch disk space is available locally on each node in /scratch (67 GB). Files in /scratch are automatically deleted once they are 14 days old.

  • Shared scratch: The path to your scratch space is /N/dc/scratch/username (replace username with your username). Shared scratch space is hosted on the Data Capacitor. The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated, and its total capacity fluctuates based on project space requirements. The Data Capacitor is mounted on Big Red as /N/dc/..., and behaves like any other disk device on Big Red. If you have an account on Big Red, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see Data Capacitor. Files in shared scratch space more than 60 days old are periodically purged, following user notification.

Back to top

Quarry

  • Home directory: Your Quarry home directory disk space is allocated on the IBM N5500 NAS storage device. You have a 10 GB disk quota, which is shared with Big Red and the RDC (if you have accounts on those systems).

  • Local scratch: Scratch disk space is available locally on each node in /scratch (19 GB). Files in /scratch are automatically deleted once they are 14 days old.

  • Shared scratch: The path to your scratch space is /N/dc/scratch/username (replace username with your username). Shared scratch space is hosted on the Data Capacitor. The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated, and its total capacity fluctuates based on project space requirements. The Data Capacitor is mounted on Quarry as /N/dc/..., and behaves like any other disk device on Quarry. If you have an account on Quarry, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see Data Capacitor. Files in shared scratch space more than 60 days old are periodically purged, following user notification.

Back to top

Research Database Complex

  • Home directory: Your RDC home directory disk space is allocated on the IBM N5500 NAS storage device. You have a 10 GB disk quota, which is shared with Big Red and Quarry (if you have accounts on those systems).

  • Local scratch: Scratch disk space is available locally on each node in /tmp (1 GB) and an additional large scratch file system named /scr (10 GB). Files in /tmp are automatically deleted once they are 24 hours old. Files in /scr are deleted when they are 30 days old. No disk quotas are enforced on /tmp or /scr.

  • Shared scratch: The path to your scratch space is /N/dc/scratch/username (replace username with your username). Shared scratch space is hosted on the Data Capacitor. The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated, and its total capacity fluctuates based on project space requirements. For details, see Data Capacitor. Files older than 60 days are periodically purged, following user notification.

Back to top

Mail spool disk space usage

UITS does not provide a production mail service on any of the research systems. However, TORQUE (also called PBS) and LoadLeveler communicate via email. Mail forwarding should be configured during account creation, but if you are receiving mail on any of the research systems, you may need to set up a .forward file for your account. For help, see How do I forward my mail from a Unix account?

If you do leave email on the research systems, it may be purged. Twice a month /var/spool/mail is checked for mail files larger than 100 KB. Files larger than 100 KB are moved to /scr/mail. You may have up to two old mail files in /scr/mail. After a mail file is 30 days old, it will be purged from /scr/mail.

Back to top

SDA on Big Red and Quarry

About SDA

The Scholarly Data Archive (SDA) provides extensive capacity (15 PB) for storing and accessing research data. It uses the High Performance Storage System (HPSS), a hierarchical storage management software package. The system is located at both Indiana University Bloomington and IUPUI, providing automatic off-site copies of data for disaster recovery. The SDA is HIPAA-aligned, but users should encrypt electronic protected health information (ePHI) data before storing it on the SDA.

If you need more than 10 GB of permanent storage, consider applying for an SDA account (see below). For more, see the SDA home page.

Back to top

Accounts

See Applying for your SDA or RFS account

Quotas on the SDA start at 1 TB. IU researchers can request additional storage if necessary.

Back to top

Access

See At IU, how do I access the SDA?

To use HSI from Big Red to access SDA, add the +hpss keyword to your ~/.soft file, and run the resoft command. For more, see On Big Red and Quarry at IU, how can I use SoftEnv to customize my software environment?

Back to top

Security

SDA security is based on Kerberos. All SDA services are fully Kerberized.

Note: Neither FTP or pftp_client encrypt data during transmission. If you are storing confidential data (e.g., medical records or similar material), encrypt the data before transferring it to the SDA. That way, not even SDA administrators will be able to read the data.

Back to top