Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Backends

A backend is a named storage, described once in [[backends]]; mounts refer to it by name.

[[backends]]
name = "disque"        # the name that mounts refer to
type = "local"         # local, sftp, s3 or webhdfs
root = "/srv/sftp"     # the type-specific keys

[[roles]]
name = "utilisateurs"

[[roles.mounts]]
backend = "disque"
home_dir = "/{username}"
acl = [{ path = "/", rights = ["read", "write", "list"], recursive = true }]

[[backends]] are written in config.toml or in roles_file, where they are reloaded at runtime with the roles.

The types

Typetype =The storageWhat home_dir points toWrites
Local"local"a directory on the servera subdirectory of rootyes
SFTP proxy"sftp"an upstream SFTP server, under a service accounta directory on the upstreamyes
S3 and compatibles"s3"an AWS S3 bucket, MinIO, Garage, Scaleway…a key prefix, after prefixyes
WebHDFS (Knox)"webhdfs"HDFS through Apache Knoxan HDFS directoryread-only

The published binaries and images carry all four types (The binary’s features).

The keys of every backend

All keys: Reference [[backends]].

A subtable ([backends.auth], [backends.credentials]) attaches to the [[backends]] before it: write it right after it. GET /admin/config and the Configuration tab show each backend, secrets replaced by ***.

Keys common to all types

What they do on each type is on the type’s page; their default and their effect, in the same Reference.

hidden_stores resolves from the mount, then the backend, then [server.hidden_stores]; stale_partials from the backend, then [uploads.stale_partials]; key by key (Uploads).

Reservation across instances

The first upload to a destination holds it until it ends (why). Across instances, this is a lock <lock_prefix><name> placed next to the destination. lock_prefix:

  • is 8 characters to 64 bytes long, with no / and no control character;
  • also names the server’s throwaway names (PPnew.<token>, PPprobe., PPstale. for a prefix P);
  • reserves for the server any name that starts with it, regardless of case, like .craftfilegate-upload...: choose one that no user file carries.

Availability

Each instance visits each backend in the background ([server.backend_probe]): up, down after failures_before_down failed visits, unknown before the first one; a WARN at each change (Troubleshooting), the state in GET /admin/health, the Instances view and craftfilegate_backend_up (Metrics). On S3, a visit is a billed ListObjectsV2, every 15 s per instance by default.