Using Rclone with OtterStorage

Rclone is a command-line tool for managing files in object storage. Since OtterStorage is S3 API-compatible, you can configure a remote in a few minutes and use all the usual rclone commands to copy, sync, verify, and mount your buckets. This guide covers configuring the remote and the most common workflows.

Requirements

• Rclone installed (version 1.55 or later recommended). Download it from rclone.org/downloads.
• A bucket created in OtterStorage with its credential pair: access key and secret key. In OtterStorage keys are generated per bucket; see the documentation to get them.
• The connection details: endpoint https://es-mad-1.s3.otterstorage.io and region eu-mad.

Configure the remote with rclone config

The interactive rclone config wizard is the simplest way to create the remote. Run it and follow the steps:

rclone config

Answer the wizard's questions with these values:

• n to create a new remote, and give it a name, for example otter.
• Storage type (Storage): select s3 (Amazon S3 Compliant Storage Providers).
• Provider (provider): choose Other (any other S3-compatible provider).
• Credentials mode (env_auth): false, to enter the keys manually.
• access_key_id: your bucket's access key.
• secret_access_key: your bucket's secret key.
• Region (region): eu-mad.
• Endpoint (endpoint): https://es-mad-1.s3.otterstorage.io.
• You can leave the rest of the options at their default value. Confirm and save.

When the wizard asks about advanced options, you usually don't need to change anything. If it offers to configure location_constraint or acl, leave them blank.

Equivalent block in rclone.conf

If you prefer to edit the configuration file directly, add this block to your rclone.conf. You can find the file path with rclone config file.

[otter]
type = s3
provider = Other
env_auth = false
access_key_id = TU_ACCESS_KEY
secret_access_key = TU_SECRET_KEY
region = eu-mad
endpoint = https://es-mad-1.s3.otterstorage.io

Replace TU_ACCESS_KEY and TU_SECRET_KEY with your bucket's credentials. From here on, the remote is referenced as otter: and buckets as otter:nombre-del-bucket.

Check the connection

Before moving data, verify that the remote responds by listing your buckets and their contents:

# List the buckets available on the remote
rclone lsd otter:

# List the objects inside a bucket
rclone ls otter:mi-bucket

The rclone ls command shows the size and path of each object. Use rclone lsl if you also want the modification date, or rclone tree otter:mi-bucket to see a hierarchical view.

Copy files with rclone copy

rclone copy copies files from source to destination without deleting anything at the destination. It's the safest option for uploading or downloading data.

# Upload a local folder to a bucket
rclone copy ./datos otter:mi-bucket/datos --progress

# Download a bucket to a local folder
rclone copy otter:mi-bucket/datos ./datos --progress

copy only transfers new or modified files, comparing them by size and date (or by checksum if you specify it). Files that already exist and match are skipped.

Synchronize with rclone sync

rclone sync makes the destination identical to the source: it copies what's missing and deletes at the destination what no longer exists at the source. Use it carefully and always test first with --dry-run.

# Simulation: shows what it would do without touching anything
rclone sync ./web otter:mi-bucket/web --dry-run

# Real synchronization
rclone sync ./web otter:mi-bucket/web --progress

At OtterStorage we don't charge for requests or deletes, so synchronization operations that delete or rewrite many objects generate no extra cost for those operations. Even so, remember that sync is destructive at the destination: always confirm the source → destination direction.

Verify integrity with rclone check

rclone check compares source and destination and reports differences without transferring data. It's ideal for validating a migration or a backup.

# Compare checksums between local and the bucket
rclone check ./datos otter:mi-bucket/datos

# If you want to log the differences to a file
rclone check ./datos otter:mi-bucket/datos --differ differences.txt

By default check compares by checksum when both sides support it. Since the checks only issue listing and metadata requests —which OtterStorage doesn't bill— you can verify your data as often as you need.

Mount a bucket with rclone mount

rclone mount exposes a bucket as a local file system, useful for accessing the objects as if they were regular files.

# Linux / macOS (requires FUSE)
rclone mount otter:mi-bucket /mnt/otter --vfs-cache-mode writes

# Windows (requires WinFsp)
rclone mount otter:mi-bucket X: --vfs-cache-mode writes

The --vfs-cache-mode writes flag enables a write cache that improves compatibility with applications that rewrite or seek within files. To unmount, stop the rclone process (Ctrl+C) or use fusermount -u /mnt/otter on Linux. Mounting is intended for convenience and occasional access; for bulk transfers, copy and sync are faster and more reliable.

Useful performance flags

These flags let you tune the speed and visibility of operations:

• --transfers N: number of files transferred in parallel (default 4). Raising it speeds up batches with many small files, for example --transfers 16.
• --checkers N: number of existence/checksum checks in parallel (default 8). Increasing it speeds up the comparison phase in large directories, for example --checkers 32.
• --fast-list: uses fewer recursive listing requests and consumes more memory, which greatly speeds up operations on buckets with many objects.
• --progress (or -P): shows a real-time progress bar with speed, ETA, and files transferred.

rclone sync ./datos otter:mi-bucket/datos \
  --transfers 16 \
  --checkers 32 \
  --fast-list \
  --progress

Since we don't bill requests, you can use --fast-list and increase --checkers without worrying about the cost of API calls; the only practical limit is your bandwidth and the available memory.

Migration case

To migrate data from another S3 provider to OtterStorage, configure both remotes in rclone (for example, origen: for the old provider and otter: for OtterStorage) and transfer directly from one to the other, without going through local disk.

# 1. Initial server-to-server copy
rclone copy origen:bucket-antiguo otter:bucket-nuevo \
  --transfers 16 --checkers 32 --fast-list --progress

# 2. Verify that everything arrived correctly
rclone check origen:bucket-antiguo otter:bucket-nuevo --fast-list

# 3. Final sync to capture last-minute changes
rclone sync origen:bucket-antiguo otter:bucket-nuevo \
  --transfers 16 --fast-list --progress

Recommendations for the migration:

• Start with copy for the initial transfer (non-destructive) and leave sync for the final cutover, once writes to the source have stopped.
• Run rclone check when you finish to confirm integrity by checksum before decommissioning the source.
• Since OtterStorage doesn't charge for requests or deletes, you can repeat check and retry transfers as many times as needed at no per-operation cost.

With this you now have a complete rclone workflow on OtterStorage. For more details on buckets, credentials, and the S3 API, see the documentation.