SFTP proxy

The sftp backend stores files on an upstream SFTP server, reached over SSH with a service account that users do not know.

[[backends]]
name = "acme-amont"
type = "sftp"
host = "sftp.acme.example"
host_key_fingerprint = "SHA256:2hZbXq1b5Xb3vN2mQ6w7l3Fv0tXk4yJ8aUeYp9rS0cE"   # checked out of band

[backends.auth]
type = "password"
username = "relais"
password = "changez-moi"

[[roles]]
name = "acme"

[[roles.mounts]]
backend = "acme-amont"
home_dir = "/depot"
acl = [{ path = "/", rights = ["read", "write", "list"], recursive = true }]

A client of the acme role uploads /f.txt: the file is /depot/f.txt on the upstream.

The keys

All keys: Reference [[backends]] type = “sftp”; common keys: Backends. Key authentication:

[backends.auth]
type = "private_key"
username = "relais"
private_key_pem = """
-----BEGIN OPENSSH PRIVATE KEY-----
...
-----END OPENSSH PRIVATE KEY-----
"""

Prefer an ed25519 or ECDSA key. An RSA key signs with rsa-sha2-512 or rsa-sha2-256 depending on server-sig-algs, ssh-rsa (SHA-1) as a last resort.

The upstream host key

The proxy checks the upstream host key before authenticating: otherwise, a man in the middle would receive the service account credentials. The fingerprint:

ssh-keyscan -p 22 sftp.acme.example | ssh-keygen -lf -

• Pin the ED25519 fingerprint, failing that ECDSA, failing that RSA: this is the order in which the proxy negotiates.
• Verify it out of band: on the upstream, ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub.
• Lowercase sha256: and trailing = padding are accepted.
• A key presented as a certificate is judged on the key it certifies.
• Rotation: one fingerprint per backend; change it at switchover, the roles reload applies it to the next connections.

Uploads

With hidden_stores.enabled = true, an upload writes to an in-transfer file on the upstream, next to the destination, then publishes it:

At step 3, the destination is missing for one round trip. Without hidden_stores, the upload writes to the destination, and resume and append are served (Uploads).

The service account can create, rename and delete in the destination directory. An overwrite replaces the inode (mode, owner, ACL, extended attributes lost) and takes twice the space until publication.

Common keys on an SFTP proxy

Performance and limits

• Read: one READ request in flight per handle; a read that follows the previous one is served from the block already read. A download in order reads the file about once on the upstream.
• Write: packets as large as the upstream accepts, eight in flight.
• One SSH connection per SFTP session and per REST request: the handshake weighs on short REST requests.
• The proxy speaks SFTP v3 (statuses 0 to 8), like OpenSSH.
• Upstream links are not told apart: stat follows a link, a listing marks no entry as a link.

What you will see