Skip to content
@go-filesystems

go-filesystems

go-filesystems

go-filesystems

Pure-Go filesystem drivers — read & write disk images, no cgo, no root.

docs drivers Go license


What is this?

go-filesystems is a set of independent, dependency-free Go modules that read and write on-disk filesystem images — one module per filesystem, all implementing a small shared contract. No cgo, no root.

The contract lives in go-filesystems/interface:

// package filesystem
type Filesystem interface {
	Close() error
	ReadFile(path string) ([]byte, error)
	ListDir(path string) ([]DirEntry, error)
	Stat(path string) (Stat, error)
	WriteFile(path string, data []byte, perm os.FileMode) error
	ReadLink(path string) (string, error)
	MkDir(path string, perm os.FileMode) error
	DeleteFile(path string) error
	DeleteDir(path string) error
	Rename(oldPath, newPath string) error
}

Filesystem drivers (15)

Module Filesystem On-disk format
apfs Apple File System. Real APFS on-disk (kext-mountable), GPT-aware
btrfs Copy-on-write Linux fs; read-only subvolumes/snapshots. Single-device, CRC32c (btrfs-progs ≥ 5.x)
exfat Extended FAT for large media. exFAT
ext4 Linux ext4 — extents, 64-bit, journaling, metadata_csum. ext4 — extents, 64-bit, flex_bg, dir htree, metadata_csum (CRC32c)
fat32 FAT with 32-bit allocation. FAT32
ffs NetBSD/OpenBSD Berkeley Fast File System. FFSv1/FFSv2 (= UFS1/UFS2), re-export of ufs
hfsplus Apple HFS+ / HFSX (Mac OS Extended). Big-endian catalog + extents-overflow B-trees
iso9660 ISO 9660 / ECMA-119 optical-disc filesystem. ISO 9660 / ECMA-119 + Rock Ridge (names/perms/symlinks) + Joliet (UCS-2 names)
ntfs Windows NT filesystem. Minimal in-image blob model — NOT the real NTFS on-disk format; separate read-only real-NTFS reader
squashfs Compressed read-only archive filesystem. SquashFS 4.0 read-only archive; gzip/xz/zstd/lzo/lz4 blocks + fragments
uefi EFI System Partition (FAT-based). OVMF/EDK2 NvVar variable store; time-based authenticated writes
ufs Unix File System (BSD). UFS2 (FreeBSD 14.x) read+write; UFS1 read; NetBSD/OpenBSD FFSv1/FFSv2 (via ffs alias module)
xfs High-performance journaling filesystem. XFS v5 (CRC32c, ftype), reflink + quotas
zfs Copy-on-write pooled storage filesystem. Single pool, single-vdev writer; multi-vdev/RAID-Z read
oci Read-only OCI/Docker image filesystem. Merged tar-layer overlay (whiteouts, opaque dirs, hardlinks) — image-as-filesystem, not an on-disk format

Also in the org: detect, a blkid-style type prober and driver-opener registry that dispatches to the drivers above (not an on-disk format, so it has no row here).

More drivers are added over time. This table is the on-disk formats; the sections below carry the rest of the org, which it used to claim to cover and did not — nine repositories were named on no page at all.

Archives, read as a filesystem (7)

Not on-disk formats, so they have no row above — but each one answers the same filesystem.Filesystem contract, so anything that walks a disk image walks these too.

Module What it reads
cab Microsoft Cabinet (.cab)
rar RAR, including multi-volume sets
rpm RPM packages — the lead, the signature and header regions, and the payload
sevenzip 7z, through bodgit/sevenzip — and it writes
warc WARC, ISO 28500 — reader and writer
xar XAR — the macOS .pkg installer: a gzip'd XML table of contents and a heap
unarchive any of the above, sniffed from the bytes rather than the extension

A parser the modules share, and no filesystem (1)

Module What it parses
cpio cpio records — newc, crc, odc and the old binary variant, both byte orders

It has no row in the section above on purpose. Two modules needed this parser and needed different filesystems over it: unarchive indexes into an io.ReaderAt it keeps open, rpm has the whole payload decompressed in memory. Sharing the filesystem would have meant one of them taking the other's shape; sharing the parser costs neither anything — and the parser is where the defects live.

Serving one to something else (5)

The other direction: take any driver above and put it on a port, so a machine that cannot link Go can still read the image.

Module Protocol
nfs NFSv3 — mountable on macOS, Linux and Windows
smb SMB2
sftp SFTP v3 — one TCP port, SSH keys, no client to install
webdav WebDAV over HTTP — browsable and mountable
s3 an S3-compatible object API

The same filesystem, in another process (1)

Not serving to a foreign client, but splitting one filesystem across two processes so the one that touches the files need not be the one that talks to the network. Serve answers requests against any driver above; New is that driver at the other end of the connection, with the same optional interfaces and the same errors — an errno comes back as itself, so errors.Is(err, fs.ErrNotExist) answers as it did on the server.

go-fileshare/fileshare uses it to serve a person's files as that person: a worker under their uid holds the directory, behind a front that parses SMB, WebDAV or SFTP and holds no privilege. Neither side trusts the other's bytes — every frame and every field in it is bounded before anything is allocated for it.

Module What it does
remote a Filesystem served over any io.ReadWriteCloser — a socketpair to a child, a pipe, a TCP connection

A host directory, as a filesystem (1)

Every driver above decodes a format out of an image. This one does not: it hands each call to the host kernel, relative to one directory, so a server from the section above can serve a plain directory with no image in the way.

Module What it does
osfs a host directory as a Filesystem, confined with os.Root

Making a read-only image writable (1)

Module What it does
overlay changes land in a spool, cheaply, and one rewrite happens when you say so

Moving bytes between two of them (1)

Copying a range out of one driver and into another normally means pulling it through this process. When both ends are files of the host, it does not have to: the kernel can be asked to do the copy and the bytes never arrive here.

Module What it does
inflight bounded, optionally ordered concurrent request dispatch shared by the NFS, SMB and SFTP servers: per-connection and server-wide caps taken before a request is read, in-order or any-order replies (v0.2.0 batches ready replies into one write), byte-range ordering, barriers
ssh a fork of golang.org/x/crypto/ssh that flushes once per channel Write instead of once per packet: 96 → 32 writes per MiB; same API, to be reported upstream; used by sftp v0.11.0
hostcopy one call that becomes copy_file_range(2) when both files are the host's, and ReadAt/WriteAt when they are not

Trying one without installing anything (1)

Module What it does
playground the drivers compiled to WebAssembly: open a disk image in the browser and walk it. The image is read in the page and nothing is uploaded — try it

Support matrix

Read = open & inspect images · Write = mutate in place · Format = create a fresh image. Per-driver details on the docs site.

Module Read Write Format Label Symlinks Quota Snapshots Encryption Platform
apfs ✅ ✅ ✅ — ✅ — ✅ ✅ macOS / iOS
btrfs ✅ ✅ ✅ ✅ ✅ ✕ ✅ read-only — Linux
exfat ✅ ✅ ✅ ✅ ✕ — — — Windows / removable
ext4 ✅ ✅ ✅ ✅ ✅ ✕ — ✕ Linux
fat32 ✅ ✅ ✅ ✅ ✕ — — — cross-platform
ffs ✅ ✅ ✅ — ✅ ✕ ✕ — BSD
hfsplus ✅ ✅ ✅ ✅ ✅ — — — macOS (legacy)
iso9660 ✅ ✕ ✕ — ✅ — — — cross-platform (optical)
ntfs ✅ ✅ ✅ ✅ ✕ — — — Windows
squashfs ✅ ✕ ✅ — ✅ — — — Linux
uefi ✅ ✅ ✅ — — — — — firmware (UEFI)
ufs ✅ ✅ ✅ — ✅ ✕ ✕ — BSD
xfs ✅ ✅ ✅ ✅ ✅ ✅ — ✕ Linux
zfs ✅ ✅ ✅ — ✅ ✕ ✅ ✅ read-only illumos / BSD / Linux
oci ✅ ✕ — — ✅ — — — any (image, not disk)

Legend: ✅ supported by the driver · ✕ not yet implemented (the on-disk format has the feature) · — not applicable (the format has no such feature). Read = open & inspect · Write = mutate in place · Format = create a fresh image · Label = volume label (Labeller). Quota / Snapshots / Encryption reflect what the driver implements today.

Links


Branding in go-filesystems/brand.

Popular repositories Loading

  1. zfs zfs Public

    Go 2

  2. apfs apfs Public

    Go 1

  3. ufs ufs Public

    Go 1

  4. ffs ffs Public

    Go 1

  5. nfs nfs Public

    Pure-Go NFSv3 server that exports any go-filesystems Filesystem — mountable on macOS, Linux and Windows with no kext, no FSKit entitlement, no cgo.

    Go 1

  6. interface interface Public

    Go

Repositories

Showing 10 of 41 repositories

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…