Using rclone to backup your files encrypted to various cloud solutions

Use rclone to backup your files encrypted to a 3rd Party Cloud Provider

In this article, I want to create a tutorial on how you can use rclone to backup your files encrypted to various cloud solutions. In this case, I will use Google Drive to store my files encrypted. I've installed rclone with Ansible.

I want to structure the article into the following sections:

1. Setup rclone on your node
2. Create a rclone Google Drive backend
3. Create a second backend to copy your files encrypted
4. Quick roundup
5. Sources

Enough introduction - let's get started!

Setup rclone on your node

I am a friend of automated work. As soon as there are tasks that I have to do more than once, I already think about automating this specific activity. For me, it was quite likely that I would like to roll out my backup solution to some more hosts, so I looked at what was already available in the Ansible Galaxy in terms of existing Ansible roles or collections for rclone. It took me less than 5 minutes to find a suitable role that was perfect for my purpose. So the next step was to finish configuring the Ansible role.

Add the role to your workspace

In my case I use git submodules for 3rd party code that I want to add to my private repository:

git submodule add https://github.com/stefangweichinger/ansible-rclone.git git submodule update --init 

We should now have the role in our repository and can start writing a simple playbook to execute the role.

An example playbook for this might look like this:

---

- hosts: all
  gather_facts: true
  roles:
    - ansible-rclone

Now that we have done that we can already roll out and install rclone on our target server. To do that we can execute Ansible using our command prompt:

Note: in this example, my playbook is named 'rclone.yml'

ansible-playbook playbooks/rclone.yml

Now rclone should be installed on your node without any further configuration. The fact that we haven't configured rclone at this point is not a big deal because we'll worry about that later.

Create a rclone Google Drive backend

Next, we'll worry about how to get rclone to communicate with our Google Drive. In advance I recommend the official documentation for this because we will not go over all those steps in detail.

The Ansible role offers the option to store your rclone config in the Ansible variables and then store it later on the hosts in the associated files. But since I didn't know what to expect on my first try, we first create the backend for rclone manually and then add it to our Ansible code.

So let’s configure this remote location:

rclone config

Choose “New Remote” and give it a name. In my case, it's called "drive_test".

Choose "Option 15" for Google Drive. Skip “client_id” & “client_secret” for now.

Note: In case you want to know how to configure a client_id for Google Drive, I recommend this documentation snippet.

I just want to demonstrate that it works with this guide. So let’s choose the option "1 - Full Access" for now. If you want to learn more about the other options have a look right here.

In the further configuration we have to specify the root folder of the remote location, i.e. the folder we want to use as the destination folder of our backup in our Google Drive. So just navigate to your Drive in your browser and add or click on the wished folder:

Just cut & paste the folder ID into your terminal.

Don’t enter a “service_account”, we’ll use the interactive login screen.

Don’t enter Advanced Configuration as we do not need these options for our scenario.

Now, since we’ll be using an interactive login screen, rclone will probably give us a link to paste in our browser so that we can get the access token back.

In this guide, we work on a remote machine so let's say "No" here.

Paste this URL in your browser and follow the usual Google Drive authorization flow until you get this:

Paste the verification code into your terminal.

Say "No" again if you don't want a Shared Drive.

After this step, you will see a summary of your configuration and you can confirm it with "yes" if everything is correct.

Testing

Now we can see if everything has been configured correctly and if the unencrypted backup is already working.

mkdir rclone && cd rclone
echo "Hello World" > from_server.txt

Let's copy our test file to our Google Drive.

rclone copy . drive_test:

This command says that we want to copy the current folder contents to the remote root folder of our rclone backend.

Let's see if the whole thing worked.

Very nice!

Create a second backend to copy your files encrypted

To store our data encrypted in the drive we have to create a second rclone backend.

The location of the remote should be the name of the Google Drive remote we've created earlier and the path should be the name of the folder you want to store the encrypted files. It should look like: drive_test:/rclone.

After that, you can choose whether you want to encrypt the filenames and the directory names or not. Next, you should choose a really strong password for your encryption. You can also add a passphrase for your password.

Once complete, type "q" to exit the configurator.

Now it's time to test our configuration again.

rclone copy . drive_test_encrypted:

It worked! You can now download the file but you can't view it until you have decrypted it with your rclone password.

As a final step, we can now store our configuration in Ansible:

rclone_configs:
  - name: drive_test
    properties:
      type: drive
      client_id: "{{ google_drive_client_id }}"
      client_secret: "{{ google_drive_client_secret }}"
      token: ' {"access_token":"","token_type":"","refresh_token":"","expiry":""}'
      root_folder_id: "{{ root_folder_id }}"
  - name: drive_test_encrypted
    properties:
      type: crypt
      remote: drive_test:/rclone
      directory_name_encryption: false
      password: "{{ google_drive_encrypt_password }}"

Quick roundup

This was a pretty simple configuration of how to backup files encrypted from your server to Google Drive. I use this construct to create another backup of my cloud, for example. To regularly back up your data, you can simply set up a cron job that regularly copies the data.

Note: Please check out the official documentation first. Because there is a difference between rclone copy and rclone sync.

Sources

https://github.com/stefangweichinger/ansible-rclone - for the Ansible playbook
https://rclone.org/docs/ - official rclone documentation
https://rclone.org/drive/#making-your-own-client-id - create a "client_id" and a "client_secret" in Google Drive