Table of Contents

Transfer files with SFTP

SFTP is SSH's file protocol. client.OpenSftpAsync() opens an SFTP session over an existing connection; the SftpClient it returns uploads and downloads files, streams and whole directories, and browses and changes the server's file system.

When to use

  • Copying files to or from a server: releases, backups, reports, logs.
  • Working on remote files in place: appending to a log, reading the end of a large file, renaming, setting permissions.
  • Prefer SFTP to SCP whenever the server offers it (OpenSSH always does): it can list, resume and change files, and OpenSSH's own scp command uses SFTP by default since 9.0.

Open a session

await using var sftp = await client.OpenSftpAsync(cancellationToken); // one SFTP session; the connection stays shared
Console.WriteLine($"SFTP v{sftp.ProtocolVersion}, home {await sftp.GetHomeDirectoryAsync(cancellationToken)}");

An SftpClient is one SFTP session (one SSH channel). Several can share an SshClient, and one can be used by several tasks at once. Disposing it ends the session; the connection stays up.

Upload and download

// Relative remote paths start at the login directory; '/' separates, whatever this machine uses.
long sent = await sftp.UploadFileAsync("report.pdf", "incoming/report.pdf", cancellationToken: cancellationToken);
long received = await sftp.DownloadFileAsync("outgoing/data.csv", "data.csv", cancellationToken: cancellationToken);

// Streams work too: anything readable up, anything writable down.
using var buffer = new MemoryStream();
await sftp.DownloadAsync("/etc/os-release", buffer, cancellationToken: cancellationToken);
Console.WriteLine(Encoding.UTF8.GetString(buffer.ToArray()));

How a transfer runs

SFTP is a request-and-reply protocol, and on a link with latency, waiting for each reply before sending the next request would be slow. The transfer methods keep many requests in flight at once (64 by default, as OpenSSH's sftp does), each as large as the server allows (limits@openssh.com, or 32 KiB):

sequenceDiagram
    participant C as SftpClient.UploadFileAsync
    participant S as SFTP server
    C->>S: OPEN incoming/report.pdf (write, create, truncate)
    S-->>C: HANDLE
    par up to MaxRequestsInFlight requests outstanding
        C->>S: WRITE offset 0
        C->>S: WRITE offset 32768
        C->>S: WRITE offset 65536 ...
    and replies, in any order
        S-->>C: STATUS ok
        S-->>C: STATUS ok
    end
    Note over C,S: each reply frees a slot for the next WRITE
    C->>S: CLOSE handle
    S-->>C: STATUS ok
    opt Preserve = true
        C->>S: SETSTAT (times, permissions)
    end

Downloads work the same way with READ. A transfer is still bounded by the channel's flow-control window (2 MiB), which is where segmented downloads help (below).

Progress, overwrite and preserved attributes

var options = new SftpTransferOptions
{
    Preserve = true,                 // keep times and permission bits, as sftp -p
    Overwrite = false,               // fail rather than replace an existing file
    Progress = new Progress<SftpTransferProgress>(p =>
        Console.Write($"\r{p.Path}: {p.BytesTransferred:N0} of {p.TotalBytes?.ToString("N0") ?? "?"} bytes")),
};
await sftp.UploadFileAsync("site.tar.gz", "/srv/releases/site.tar.gz", options, cancellationToken);

Resume an interrupted transfer

With Resume = true, a transfer sends only the part the destination lacks, as reget and reput do; a failed download keeps what arrived. Retry in a loop, with a new connection when the old one broke:

// Try, and on failure try again: each attempt sends only what the destination still lacks.
var resume = new SftpTransferOptions { Resume = true };
for (var attempt = 1; ; attempt++)
{
    try
    {
        await sftp.DownloadFileAsync("backups/db.tar.zst", "db.tar.zst", resume, cancellationToken);
        break;
    }
    catch (SshException ex) when (attempt < 5 && ex.Reason is not SshFailureReason.SftpFailed)
    {
        // The session or connection broke. A real job reconnects and opens a new SftpClient here (see the file sync scenario).
        await Task.Delay(TimeSpan.FromSeconds(attempt * 2), cancellationToken);
    }
}
Warning

Resuming trusts that the bytes already at the destination are the start of the same file; nothing compares them. Resume only into a destination that nothing else writes.

Download large files faster

One SFTP session moves at most about one window (2 MiB) per round trip: some 20 MiB/s at 100 ms of latency, however many requests are in flight. Segments splits a download into ranges and fetches each over its own session on the same connection:

// A large file over a high-latency link: four ranges in parallel, each over its own SFTP session.
await sftp.DownloadFileAsync("images/vm.qcow2", "vm.qcow2", new SftpTransferOptions { Segments = 4 }, cancellationToken);

It helps on high-latency links and where one server process is the limit; it does not help where bandwidth is the limit. The server may allow fewer sessions than asked (sshd's MaxSessions). Segments apply to DownloadFileAsync and DownloadDirectoryAsync only, and not together with Resume.

Copy whole directories

var options = new SftpTransferOptions { Preserve = true };
var up = await sftp.UploadDirectoryAsync("publish", "/srv/www/app", options, cancellationToken);
var down = await sftp.DownloadDirectoryAsync("/var/log/app", "logs", options, cancellationToken);
Console.WriteLine($"up: {up.Files} files, {up.Bytes:N0} bytes; down: {down.Files} files, {down.Skipped} skipped");

Symbolic links and special files are skipped in both directions. A download treats the server as untrusted: a name that cannot be a safe file name here is skipped and counted in Skipped, and a path that would land outside the target stops the download.

Browse and change the remote file system

await foreach (var entry in sftp.ListDirectoryAsync("/var/log", cancellationToken))
{
    var a = entry.Attributes;
    var kind = a.IsDirectory ? "dir " : a.IsSymbolicLink ? "link" : "file";
    Console.WriteLine($"{kind} {a.Size,12:N0} {a.LastWriteTime:yyyy-MM-dd HH:mm} {entry.Name}");
}

if (!await sftp.ExistsAsync("archive", cancellationToken))
{
    await sftp.CreateDirectoryAsync("archive", UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute, cancellationToken);
}

await sftp.RenameAsync("app.log.1", "archive/app.log.1", overwrite: true, cancellationToken);
var disk = await sftp.GetFileSystemInfoAsync("/", cancellationToken);
Console.WriteLine($"{disk.AvailableBytes:N0} bytes free");

Replace a file atomically

Readers must never see a half-written configuration file. Upload under a temporary name and rename over the target; with overwrite: true the rename is atomic (posix-rename@openssh.com):

// Readers never see a half-written file: upload under a temporary name, then rename over the target.
const string target = "/srv/app/config.json";
var temporary = $"{target}.{Guid.NewGuid():N}.tmp";
await sftp.UploadFileAsync("config.json", temporary, cancellationToken: cancellationToken);
await sftp.RenameAsync(temporary, target, overwrite: true, cancellationToken); // posix-rename@openssh.com

Work with remote files as streams

OpenAsync, OpenReadAsync and OpenWriteAsync return a seekable Stream over a remote file, with FileMode and FileAccess meaning what they do for FileStream:

// Append a line to a remote file, in place.
await using (var log = await sftp.OpenAsync("deploy.log", FileMode.Append, FileAccess.Write, cancellationToken))
{
    await log.WriteAsync(Encoding.UTF8.GetBytes($"{DateTimeOffset.UtcNow:O} deployed\n"), cancellationToken);
    await log.FlushAsync(cancellationToken);  // fsync@openssh.com where the server has it
}

// Read the last 4 KiB of a large file without downloading the rest.
await using var file = await sftp.OpenReadAsync("/var/log/app/app.log", cancellationToken);
file.Seek(Math.Max(0, file.Length - 4096), SeekOrigin.Begin);
var tail = new byte[4096];
var read = await file.ReadAtLeastAsync(tail, tail.Length, throwOnEndOfStream: false, cancellationToken);
Console.WriteLine(Encoding.UTF8.GetString(tail, 0, read));

Handle errors

A request the server refuses throws SftpException, with the SFTP Status and the Path:

try
{
    await sftp.DownloadFileAsync("missing.txt", "missing.txt", cancellationToken: cancellationToken);
}
catch (SftpException ex) when (ex.Status == SftpStatusCode.NoSuchFile)
{
    Console.WriteLine($"Not on the server: {ex.Path}");
}
catch (SftpException ex) when (ex.Status == SftpStatusCode.PermissionDenied)
{
    Console.WriteLine($"No access to {ex.Path}");
}

Notes and pitfalls

  • Paths. Remote paths go to the server as they are: relative paths start at the login directory, and / is the separator whatever this machine uses. GetFullPathAsync and ExpandPathAsync resolve them.
  • Asynchronous streams. SFTP file streams support only the asynchronous Read/Write methods; the synchronous ones throw NotSupportedException. For whole files, DownloadAsync is much faster than reading a stream.
  • Times. SFTP version 3 carries modification times in whole seconds.
  • Extensions. An operation that needs an extension the server lacks (an overwriting rename, a hard link, a server-side copy) throws NotSupportedException; sftp.Extensions lists what the server has.
  • A broken session or connection throws SshException (ChannelClosed or ConnectionClosed); open a new SftpClient.

See also

Edit this page