blog

Ultimate Command Line To Share Files Across The Internet   BLOG Tool


Using a single command, we can easily upload any files from our computers or servers to a web server. Then our friends or colleagues can download them directly from web browsers. Isn't that cool?
Worry about the security? We can encrypt files before transfer. Also we can claim the maxinum downloads of those files, or expire download link in several days.
Here we go! transfer.sh: easy and fast file sharing from the command-line. Zero Initial Setup and Super Easy to use.

Ultimate Command Line To Share Files Across The Internet


Problems For Current File Transfer

Working as a DevOps, I occasionally need to transfer files:
  1. Temporarily copy critical backup set from one server to another.
  2. Collect some log files and send back to dev team.
Usually I use scp to do the copy. Apparently we only scp via key file, instead of password. To scp files from one server to another, I need to store my ssh private key in the server. This certainly brings in some security concerns. Yes, I can protect ssh key by passphrase. Or even better, I can delete it immediately after I have finished the download. hmm... just a bit tricky, and too many manual steps.
When I'm asked to copy and share some log files, I usually scp them to my laptop. Then send over via Slack or Skype. Let's say, the file is ~500 MB. Literally speaking, we need to download 500 MB data, then send 500 MB data. Manually! This would take a lot of time for a doggy network. Isn't it? Even worse, if the transfer interrupts, our colleagues will definitely ping us to send again.

Use transfer.sh To Share Files Easily

People may solve part of the problems by scp/rsync, Dropbox[1] or ftp[2]. Or using AWS SDK or tools, upload to AWS S3 buckets.
However, if we expect no initial setup and super easy for end users, none of them can compete with this one. transfer.sh, an open source project in Github. It enables us to easily and quickly share files from the command-line.
Alternatives: mongoose[3], devd[4].
With transfer.sh, we can upload up to 10GB. And the files are stored for 14 days by default.

Ultimate Command Line To Share Files Across The Internet

All we need is a simple and famous tool, called curl. No more ssh key. No more blind wait for the doggy internet.
root@denny:~# curl --upload-file /etc/hosts https://transfer.sh/hosts
https://transfer.sh/BQTjC/hosts
For Window users, we can use Powershell.
# Upload using Powershell 
PS H:\> invoke-webrequest -method put -infile .\file.txt https://transfer.sh/file.txt
See more handy usages in transfer.sh.

What About Security?

Firstly, don't upload any very sensitive files.
Usually we can make the transfer more secured like this:
  • Limit download count. Like snapchat, we only allow one time sharing. I'm happy to get involved in this feature. See my discussion with Remco Verhoef in twitter.

Ultimate Command Line To Share Files Across The Internet

  • Expire in server days.
$ curl -H "Max-Downloads: 1" -H "Max-Days: 5" --upload-file ./hello.txt https://transfer.sh/hello.txt 
https://transfer.sh/66nb8/hello.txt 
# Download the file 
$ curl https://transfer.sh/66nb8/hello.txt -o hello.txt
  • Encrypt our files before transfer
# Encrypt files with password using gpg 
$ cat /tmp/hello.txt|gpg -ac -o-|curl -X PUT --upload-file "-" https://transfer.sh/test.txt 

# Download and decrypt 
$ curl https://transfer.sh/1lDau/test.txt|gpg -o- > /tmp/hello.txt

How fast the download/upload speeds are?

From my experience, it's fast enough. Frankly speaking, I don't worry about the speed.
It's issued by a command line. So machines can easily take over. For example, the intelligent retries, split big files into pieces, then upload in a parallel way. No human intervene is required in the whole process!
After that, we get a http download link for the recipients. This works perfectly in an asynchronous way.

Vendor Lock-In, Or Service Is Closed?

transfer.sh is an open source project in Github. We can even have a docker image to host the service in our server!
Everything is under control now. Right?
docker build -t transfersh .
docker run --publish 8080:8080 --rm transfersh --provider local --basedir /tmp/

---------------------------------------------------------------------

# --8<-------------------------- separator ------------------------>8--

# --8<-------------------------- separator ------------------------>8--

Footnotes:

Author: dennyzhang.com (denny.zhang001@gmail.com)

Created: 2016-11-17 Thu 20:23

Validate