Backends
A backend is a named storage, described once in [[backends]]; mounts
refer to it by name.
[[backends]]
name = "disque" # the name that mounts refer to
type = "local" # local, sftp, s3 or webhdfs
root = "/srv/sftp" # the type-specific keys
[[roles]]
name = "utilisateurs"
[[roles.mounts]]
backend = "disque"
home_dir = "/{username}"
acl = [{ path = "/", rights = ["read", "write", "list"], recursive = true }]
[[backends]] are written in config.toml or in roles_file, where they
are reloaded at runtime with the roles.
The types
| Type | type = | The storage | What home_dir points to | Writes |
|---|---|---|---|---|
| Local | "local" | a directory on the server | a subdirectory of root | yes |
| SFTP proxy | "sftp" | an upstream SFTP server, under a service account | a directory on the upstream | yes |
| S3 and compatibles | "s3" | an AWS S3 bucket, MinIO, Garage, Scaleway… | a key prefix, after prefix | yes |
| WebHDFS (Knox) | "webhdfs" | HDFS through Apache Knox | an HDFS directory | read-only |
The published binaries and images carry all four types (The binary’s features).
The keys of every backend
All keys: Reference [[backends]].
A subtable ([backends.auth], [backends.credentials]) attaches to the
[[backends]] before it: write it right after it. GET /admin/config
and the Configuration tab show each backend, secrets replaced by ***.
Keys common to all types
What they do on each type is on the type’s page; their default and their effect, in the same Reference.
hidden_stores resolves from the mount, then the backend, then [server.hidden_stores];
stale_partials from the backend, then [uploads.stale_partials]; key by key
(Uploads).
Reservation across instances
The first upload to a destination holds it until it ends
(why). Across instances, this is a
lock <lock_prefix><name> placed next to the destination. lock_prefix:
- is 8 characters to 64 bytes long, with no
/and no control character; - also names the server’s throwaway names (
PPnew.<token>,PPprobe.,PPstale.for a prefixP); - reserves for the server any name that starts with it, regardless of case,
like
.craftfilegate-upload...: choose one that no user file carries.
Availability
Each instance visits each backend in the background ([server.backend_probe]): up, down after failures_before_down failed visits, unknown before the first one; a WARN at each change (Troubleshooting), the state in GET /admin/health, the Instances view and craftfilegate_backend_up (Metrics). On S3, a visit is a billed ListObjectsV2, every 15 s per instance by default.