Storage tiers

There are three tiers of NFS storage, with different performance characteristics: scratch, production, and archival. Some research groups have their own scratch and production file servers, but TIG operates servers in these tiers for shared use by all CSAIL members.

All three storage tiers are split between 32-341 (the main machine room in Stata, where high-speed access is available from OpenStack and group-owned compute clusters in Stata) and OC40-250 (the Massachusetts Green High-Performance Computing Center in Holyoke). Due to the non-pipelining nature of the NFS protocol, when performance is a concern users should make sure that they are using servers in the same building as the data they are accessing. The Holyoke data center is subject to an annual 24-hour maintenance shutdown during which all servers and data located there will be inaccessible; the date of the shutdown is usually in May and will be announced to the community a few months in advance.

Scratch

Scratch storage does what it says on the tin. It is intended for high-performance storage of temporary files and intermediate computations as well as similar data that can be easily reconstructed or otherwise does not require backups.

Production

Production storage is the main workhorse of CSAIL NFS. Production servers are optimized for relatively high performance with parallel reads and writes from multiple client systems.

Archival

Archival storage is intended for data that is no longer being actively updated and does not require high performance parallel random reads. Presently archival storage is split between Stata and OC40, but TIG’s long-term plan is to shift all archival storage to OC40 in the future.