---
title: "Parallel File Storage (PFS) for GPU Training — High-Throughput Shared FS"
description: "E2E Parallel File Storage: a shared high-throughput filesystem for multi-node GPU training in India. Mounts into TIR instances at a path you choose, feeds checkpoints and shared datasets, and pairs with SFS and S3-compatible object storage. INR billing."
url: "https://www.e2enetworks.com/parallel-file-storage"
canonical: "https://www.e2enetworks.com/parallel-file-storage"
provider: "E2E Networks Limited"
type: "Service"
keywords: ["parallel file system gpu training", "high throughput file storage ai india", "shared file storage for gpu clusters", "pfs for machine learning checkpoints"]
priceCurrency: "INR"
taxNote: "Prices exclude GST"
region: "India (Delhi NCR, Chennai)"
generated: "2026-09-11"
---

# Parallel File Storage (PFS) for GPU Training — High-Throughput Shared FS

> E2E Parallel File Storage: a shared high-throughput filesystem for multi-node GPU training in India. Mounts into TIR instances at a path you choose, feeds checkpoints and shared datasets, and pairs with SFS and S3-compatible object storage. INR billing.

Canonical page: https://www.e2enetworks.com/parallel-file-storage

One filesystem, every GPU in the run. Parallel File Storage mounts into your TIR training instances as a plain directory — shared datasets in, checkpoints out, visible to every node at once. No per-node copies, no S3 client in your training loop, no idle H100s waiting on I/O.

## At a glance

- **Shared** — Many instances, one namespace
- **/mnt/pfs** — Mounts at a path you choose
- **Read-only mode** — Protect curated datasets
- **API-first** — EPFS in the MyAccount API

## Platform capabilities

- **File semantics, zero rewrites** — Your training code sees a directory, not a bucket. PyTorch DataLoaders, HuggingFace datasets and checkpoint writers work as-is — the mount path is just part of the instance spec.
- **Shared across nodes** — Every instance that mounts the filesystem sees the same namespace — rank 0 writes a checkpoint, every rank can resume from it. No rsync between nodes, no divergent local copies.
- **Read-only dataset mounts** — The documented pfs_read_only flag mounts the filesystem read-only per instance — share one curated dataset across dozens of jobs without anyone clobbering it.
- **Provision by API** — EPFS is a first-class resource in the MyAccount REST API — create, list, update and delete — so your training infra can be codified alongside the cluster itself.
- **Next to the GPUs** — PFS lives in the same Indian data centers (Delhi, Chennai) as your H100/H200 capacity and TIR instances — one network hop, one jurisdiction, one bill.
- **Sized to your workload** — PFS throughput scales with provisioned capacity and node count. Benchmark with your real I/O pattern, or ask an engineer to size it for your cluster.

## How it works

1. **Provision.** — Create a PFS resource from the console or the EPFS API in your region.
2. **Mount at launch.** — When creating a TIR instance, set the PFS mount path (and read-only if it's a dataset volume). PFS mounts apply to container-based instances; full-OS VMs use block volumes instead.
3. **Train.** — Point your dataloader at the mount, write checkpoints back to it. Every node in the run shares the same view.
4. **Persist what matters.** — Sync final weights and artifacts to S3-compatible Object Storage for versioned, lockable long-term storage — where inference endpoints pick them up.

## Mounting PFS on a TIR instance

The relevant fields of the TIR create-instance call (POST /notebooks/):

```json
// Fields documented in the TIR API reference
{
  "name": "llama-finetune-worker",
  "image_type": "pre-built",
  "sku_item_price_id": 4,

  "pfs_path": "/mnt/pfs",       // your chosen mount path
  "pfs_read_only": false,       // true for curated datasets

  "sfs_path": "/mnt/sfs",       // optional SFS mount
  "dataset_id_list": [42]       // optional EOS-backed datasets
}
```

Then in your training code: data_dir = "/mnt/pfs/fineweb". That's the whole integration.

## Parallel File Storage vs Scalable File System vs Object Storage

| Dimension | Parallel File Storage (PFS) | Scalable File System (SFS) | Object Storage (EOS) |
| --- | --- | --- | --- |
| Interface | Filesystem mount at a chosen path (e.g. /mnt/pfs) | Filesystem mount (e.g. /mnt/sfs); Kubernetes PVs | S3-compatible API (keys + endpoint) |
| Access model | Shared across TIR container instances; per-instance read-only option | Shared across nodes/pods; VPC-scoped access (allowed CIDRs) | Per-bucket permissions, tokens, pre-signed URLs |
| Where it's surfaced | TIR instances & AI workloads; EPFS API | MyAccount SFS API, TIR, Kubernetes SFS volumes | Console, API, any S3 tool, Datashim in Kubernetes |
| Data protection | Read-only mounts; pair with EOS for versioned copies | Managed backup activate/deactivate via API | Versioning, lifecycle, replication, object lock |
| Best for | Hot shared data during multi-node GPU training — checkpoints, active datasets | Fault-tolerant shared volumes for services and clusters | Source-of-truth datasets, model artifacts, backups, logs |
| Not the tool for | Long-term archival; single-writer databases (use Block Storage) | Object workflows and WORM retention | Random-access file I/O inside a training loop |

## Common use cases

- **Checkpoints that every rank can see** — Write once from rank 0, resume from any node after a preemption or failure. Sync milestone checkpoints out to object storage with versioning and lock for keeps.
- **Shared datasets, mounted read-only** — Curate once, mount read-only into every TIR training job and notebook. No N-times-replicated local copies eating your node disks.
- **Multi-node cluster workflows** — Scheduler-driven clusters — including Slurm-style setups on GPU nodes and TIR training clusters — assume a shared filesystem for job scripts, environments and outputs. PFS plays that role on E2E, with CPU compute nodes handling the head-node and preprocessing duty.

## Frequently asked questions

### Do I have to change my training code to use PFS?

No — that's the point. PFS appears as a directory at the mount path you set when launching the instance (e.g. /mnt/pfs). Dataloaders, checkpoint callbacks and logging write to it like local disk. Standards-based file semantics mean no vendor SDK in your training loop, and nothing to unpick if you leave.

### Which workloads can mount PFS?

Per the TIR API reference, PFS mounts (pfs_path, pfs_read_only) are supported on TIR container-based instances — notebooks and training workloads. They're not supported on full-OS VMs, which use block volumes; PFS also appears in TIR's storage surface alongside SFS and the container registry.

### PFS or SFS — which one do I want?

PFS is the shared tier surfaced in TIR for AI workloads: hot datasets and checkpoints during training. SFS is E2E's shared filesystem with documented Kubernetes persistent-volume support, VPC CIDR-scoped access and managed backups — a better fit for long-lived service volumes. If your question is "where do my GPUs read training data from", it's PFS.

### Why not just stream from object storage?

For sequential one-pass reads, streaming from EOS is fine and cheap. But random-access dataset sampling, frequent checkpoint writes and shared state across ranks want file semantics and low-latency re-reads — that's the gap PFS fills. The common pattern: EOS as source of truth, PFS as the hot layer for the duration of the run.

### What performance numbers can you commit to?

PFS performance scales with provisioned capacity and cluster size, so we commit numbers per deployment rather than a one-size-fits-all figure. Run your real access pattern, or talk to an engineer — we'll size PFS for your node count and dataset and stand behind it.

## Get started

Provision PFS next to your training cluster in minutes. Pricing: https://www.e2enetworks.com/pricing · Talk to sales: https://www.e2enetworks.com/contact-sales

- [Start Free](https://myaccount.e2enetworks.com/accounts/signup)
- [Talk to an Engineer](https://www.e2enetworks.com/contact-sales)

---

This Markdown is generated from the same data that renders https://www.e2enetworks.com/parallel-file-storage. Provider: E2E Networks Limited (NSE: E2E), India. Site index for AI agents: https://www.e2enetworks.com/llms.txt