Data has "gravity" (Data Gravity). The more data you store in AWS S3, the harder (and more expensive) it becomes to get it back out. This isn't due to technical difficulty – S3 is now an open industry standard – but rather the fear of complexity and egress fees.

At OutaCloud, we regularly migrate storage-intensive workloads from AWS to sovereign alternatives. In this guide, we open our playbook: We show which tools we use, how we choose the target architecture (Hetzner vs. Ceph), and how the technical migration process works step by step.

1) Architecture Decision: Where Should the Data Go?
- A) Hetzner Object Storage (Cold/Warm Data)
- B) Ceph RGW on Proxmox (Hot Data / High Performance)
2) The Tool of Choice: rclone is the Standard
3) Step-by-Step Guide: The Migration
4) Data Integrity & Validation
5) Common Pitfalls (Lessons Learned)
Conclusion: Let Complexity Be Managed
Don't Want to Do This Yourself?

1) Architecture Decision: Where Should the Data Go?

Before we copy data, we need to define the target. In the Proxmox/bare-metal environment, two targets have established themselves as standards:

A) Hetzner Object Storage (Cold/Warm Data)

Ideal for backups, archives, user uploads, or static web assets delivered via CDN.

Technology: MinIO-based backend, managed by Hetzner.
Advantage: Unbeatable price, traffic within Hetzner network is free, no maintenance.
Disadvantage: Latency (network roundtrip), shared infrastructure.

B) Ceph RGW on Proxmox (Hot Data / High Performance)

Ideal for applications that generate high I/O loads or need to process data in real-time.

Technology: We activate the Rados Gateway (RGW) on your Proxmox Ceph cluster.
Advantage: Data resides on the same NVMe SSDs as your VMs. 0ms external latency, maximum throughput, full data sovereignty.

Why Ceph RGW instead of MinIO? MinIO has changed its licensing policy and shifted focus. Ceph RGW is deeply integrated into Proxmox, Open Source, and the gold standard for enterprise storage.

2) The Tool of Choice: rclone is the Standard

Forget self-written Python scripts or the AWS CLI (aws s3 sync) for large migrations. The tool of choice is rclone.

Why rclone?

Multithreading: Can transfer thousands of objects in parallel.
Checksums: Guarantees data integrity (MD5/SHA1 verification).
Throttling: With --bwlimit we prevent the migration from crippling your production.
Provider-Agnostic: Speaks AWS S3, Ceph, and Hetzner perfectly.

3) Step-by-Step Guide: The Migration

Note: This guide assumes migration using rclone on a Linux jump host or directly on a Proxmox node.

Step 1: Configure Remotes

We define source (AWS) and destination (Hetzner/Ceph) in ~/.config/rclone/rclone.conf.

# rclone config (Interactive mode or direct editing)

[aws-source]
type = s3
provider = AWS
access_key_id = AKIA...
secret_access_key = ...
region = eu-central-1

[hetzner-dest]
type = s3
provider = Other
env_auth = false
access_key_id = ...
secret_access_key = ...
endpoint = https://fsn1.your-objectstorage.com

Step 2: The "Dry Run" (Safety Test)

Before we move terabytes, we test connectivity and permissions.

rclone sync aws-source:my-production-bucket hetzner-dest:new-bucket \
  --dry-run \
  --verbose

Check: Are files listed correctly? Do the paths match?

Step 3: The Initial Sync (Bulk Copy)

Now we move the bulk of the data. Since this can take days, we run this process in the background (e.g., in tmux or screen).

rclone sync aws-source:my-production-bucket hetzner-dest:new-bucket \
  --progress \
  --transfers 64 \
  --checkers 64 \
  --bwlimit 50M \
  --update \
  --fast-list

Flag Explanation:

--transfers 64: Copies 64 files simultaneously (important with many small files).
--bwlimit 50M: Limits bandwidth to 50 MB/s to avoid disrupting running operations.
--update: Skips files that already exist newer at the destination.
--fast-list: Uses more RAM to save API requests (and thus AWS costs).

Step 4: Delta Syncs (Catch-up)

While the initial sync was running, your users uploaded new files to AWS. We now run the command again (without bandwidth limit).

rclone sync aws-source:my-bucket hetzner-dest:new-bucket \
  --transfers 128 \
  --checkers 128

Rclone compares mod-times and checksums and transfers only the difference.

Step 5: The Cutover (The Switch)

This is the critical moment for your application.

Maintenance Mode: Switch the app to maintenance mode or "read-only".
Final Sync: One last rclone sync, which should now only take seconds.
Config Switch: Change your app's environment variables.
- Old: S3_ENDPOINT=s3.eu-central-1.amazonaws.com
- New: S3_ENDPOINT=https://fsn1.your-objectstorage.com (or internal Ceph IP)
Deployment: Restart the app.
Verification: Perform a test upload.

4) Data Integrity & Validation

Trust is good, hashes are better. After migration, you should verify randomly or completely.

The Full Check (Takes long, costs API calls):

rclone check aws-source:my-bucket hetzner-dest:new-bucket --one-way

This compares MD5 hashes of all objects without downloading data.

Warning: With AWS Multipart uploads, the ETag (MD5) sometimes doesn't match standard MD5. Rclone usually detects this, but you should be aware of it.

5) Common Pitfalls (Lessons Learned)

From our experience with various migrations:

The "Exit Tax" (Egress)

AWS charges ~$0.09 per GB for outgoing traffic. For 50 TB of data, the transfer alone costs $4,500 USD. That's painful, but a one-time fee ("ransom"). After that, your traffic costs at Hetzner/OutaCloud are near zero.

ACLs & Public Buckets

AWS has complex Access Control Lists (ACLs). Hetzner and Ceph use simpler models or S3 Bucket Policies. Check if your files need to be "public-read" and set the policy in the new bucket before you go live.

Presigned URLs

Does your app use temporary upload links? Make sure your S3 library (AWS SDK) correctly signs the new endpoint (usually set s3ForcePathStyle: true).

Conclusion: Let Complexity Be Managed

An S3 exit isn't rocket science, but requires precision in planning to avoid data loss and downtime. The combination of rclone and targets like Hetzner or Ceph RGW is powerful and permanently frees you from unpredictable storage costs.

If you want to leave AWS but don't want to sacrifice data integrity, a structured migration approach is the safest path.

Don't Want to Do This Yourself?

At OutaCloud, we handle the complete migration for you. We analyze your buckets, calculate egress costs upfront, and perform the sync as a managed service.

Start here:

Exit AWS – Overview of our AWS migration services
AWS to Hetzner – Our migration service for switching to Hetzner
Calculate Your Savings – See how much you could save

Or contact us directly for a free migration analysis.

AWS S3 Exit Strategy: The Technical Guide to Migrating to Hetzner & Ceph

Table of Contents