NextCloudPi VM on VirtualBox: Easy setup and clone

This article addresses you, who avoid words like Docker or Vagrant (but please consider getting over it soon…), and you do not want to know more than you need about the configuration of NextCloudPi and a VM and you also can’t afford to buy a board computer. There is a ready-to-go solution for you too, friend.

Below you can find an automation script which does all the nasty work for you, configuring a NextCloudPi VM, just by prompting you to answer some simple questions – do not worry there are also details provided so that you don’t get messed up.

Moreover, it provides the option to easy clone an already created VM so that you can have multiple machines (What for? Check my previous article Make Your NextCloudPi highly available to find out 😉 ). This way you can have a faster setup of your NCP VM, by skipping the installation of NCP and – sometimes – even avoiding the box import (large) overhead in terms of time.

The only requirement is to install VirtualBox and Vagrant.

Let’s break it down and get into this script’s logic and see what options it offers.

Let’s face the truth. VirtualBox is one of the most popular software virtualization packages – especially among beginners level. Even though desktop virtualization lacks the benefits of the others (operating-system level virtualization like docker is way more portable and faster, and hypervisors type 1 – bare metal – are highly secure), VirtualBox has become prominent for some good reasons: it’s super easy and convenient to use, free and also cross-platform compatible.

So, even if I am a containers person – Docker now and forever – I’ve gone deep down in the configuration of a VirtualBox machine, and I’m offering you all the available settings for NCP on VBox.

Of course, I wouldn’t have gone this far if VirtualBox was not supported by Vagrant as a provider.

Vagrant is an excellent tool to work with virtual environments, and most often this means virtual machines. It offers a generic command-line client to configure and manage any virtual environment it supports. The “recipe” of how to do so resides in a template file, called Vagrantfile. With Vagrantfile one can define a whole environment in code which makes it easily reusable and easy to modify and share.

As you may guess, I used a Vagrantfile to setup NextCloudPi. The official Github repository of NCP offers a Vagrantfile, which I used as a base and enriched it with more specific instructions, addressing VirtualBox specifically.

Combining VirtualBox and Vagrant, I created an automation script which prompts user to give some input about the resources of the VM and the preferred networking, providing details about each one of them. Then the script either creates a new VM or clones an existing VM, according to user’s input. This makes a beginners life easier as all he has to do is have some interaction with the script and answering some questions.

New VM

Let’s execute the script and examine each step through an example.

The first thing the script needs to know is if user want to create a new VM or clone an existing VM.

It then asks to provide a name for the VM. If you hit enter you get the default name ‘NextCloudPi’. Don’t worry if the name is taken. VBoxManage will be used to check this for you, and you will be notified if it does. So good, so far.

Ok, this may be the most difficult for a beginner to answer, so let’s talk about it.

VirtualBox VMs have a variety of networking ways to choose from. Two of them, are the ones you see above. They are the most convenient networking options for setting up NextCloudPi. These two will get you an IP address to refer to your NCP directly – it’s better to avoid port forwarding option for an easy setup…

  • Public network, or Bridged networking as VirtualBox calls this option, offers an IP to the VM which is reachable by any machine in the same LAN. This means, that anyone who is using another machine, physical or VM, and has access to your NCP, can use it. The Vagrantfile command responsible for this type of network is the following (bridge will be given the value of host’s interface):
config.vm.network "public_network", bridge: "wlp4s0", ip: "192.168.1.2"
  • Private network, or Host-Only networking, is offering an IP as well, but this IP is coming from a private network and thus, it’s only reachable by the host or other VMs on the host. Anybody else on the LAN who uses remote physical machine cannot use your NCP. Also, this option does not waste an IP address of your LAN. The automation script will use the default range of DHCP on VBox (vboxnet0). If you want to modify this, check the pick_ip function. As for the responsible command in Vagrantfile:
config.vm.network "private_network", ip: "172.28.128.3"

So you should choose depending on the accessibility of your NextCloudPi. If you’re accessing it only through your host, then a private network will work for you. If you want to be able to access it from various hosts, then go for a public network. If you still don’t know, just go for public and make sure you don’t hand your NCP password around.

Time to make the IP address of the VM specific. If you have a favourite number or anything, give it a try and the automation script will tell you if the IP you specified is valid (check function valid_ip) and not reserved (check function reserved_ip). About the latter, it will try to ping the IP you specified and check if it’s responded, however this trick won’t work if it’s assigned to a firewalled device.

If you let the script pick an IP (option ‘any’), it will use host’s IP and host’s interface to get to the range of your LAN and will pick the first IP available.

The last thing that you can configure on your VM is its resources. That depends just on your host and the resources you can provide to your NCP VM. Just specify memory and cpu, plain and simple. If you hit enter, the default values will be provided as input. The commands used for this configuration in Vagrantfile are the following:

  config.vm.provider "virtualbox" do |v|
    # Resources
    v.memory = 2048
    v.cpus = 1
    # VM name
    v.name = "MyNextCloudPi"
  end

That’s it! You got your NCP VM created. You can start it either through the VirtualBox GUI (Headless start will do the work, if you just want to use the web panel) or by using the command specified ( VBoxManage startvm MyNextCloudPi –type headless ) – if VBoxManage package is installed, if it’s not the script won’t even tell you about the command.

In the output message you can see the IP that was eventually assigned to your VM.

Clone VM

Now that you’ve created your first NCP VM, let’s see how you can easily clone it.

We will skip the questions we explained before, and focus on the extra questions that you will be asked to answer for this option.

The first thing that the script will ask you about the cloning is to choose between two options:

  • Absolute path to parent’s VM Vagrantfile: If you used the automation script for the creation of the parent VM, then there is a Vagrantfile under the VM’s directory you created. In this case, the script will parse the Vagrantfile to get some info about the parent VM (IP and name), so that the cloned VM will not use the same and cause any conflict.
    You may have not used the automation script, but created a VM with your own Vagrantfile. That should work, too. But if the required details are not found inside the Vagrantfile, then the script will need to start the parent VM (in case that it’s not already running), either on its own using VBoxManage or with your manual help
  • Parent VM’s VirtualBox Name: If you remember the name of the VM that you want to clone, then choose this option. This option will not need to parse any Vagrantfile, it will just start the parent VM so that its assigned IP is excluded from the available set of IPs. VBoxManage will be used once again to check that the parent VM exists and if not, you will be notified.

Tip: VBoxManage is the command-line interface to VirtualBox and the commands that the automation script uses are the following:
VBoxManage list vms : Lists all existing VirtualBox VMs
VBoxManage startvm ${VBox_name} –type headless : Starts a VirtualBox VM with headless mode start

1. Vagrantfile’s path

If you’ve chosen this option, then you should obviously be prompted to specify this path:

The automation script will check the path for any existing packaged boxes and if there is any, you will be asked if you want to use it:

If you do choose to use it (Use existing box), any metadata of boxes with the name of this path will be removed. Let’s explain this part.

Vagrant has a command to package a VM into a box: vagrant package
This command will create file ‘package.box‘.
If a Vagrantfile uses this box to create a VM, then it will be automatically imported into Vagrant’s metadata, so that it can be reused with the name of the path (box: ‘path/to/box/package.box’).
But if the package.box is not used in a Vagrantfile or never created a VM with it, then it may not be added to the metadata.
So, a package.box that exists in a directory may be of a more recent version of the VM than the same name existing in Vagrant’s metadata (Imagine the scenario of somebody using vagrant package but never using this package.box to create a new VM).

If you’re not sure which version of the parent NCP VM it refers, then stick to the Create new box option.

This option will use the vagrant package command we mentioned before and will create a package.box file, which will be imported to the new Vagrantfile.

What happens if there is no existing package.box of the parent VM, but there is one box name related to parent’s path inside Vagrant’s metadata? Automation script will search for it and will ask you a similar question:

The options in this case are equal to the ones we mentioned when a package.box exists.

2. Parent’s VBox Name

This option is quite simple. You just specify parent VM’s name, VBoxManage will get it running, so as to exclude its IP, and the script will just use vagrant package –base parents_name to package it.

Metadata is not considered in this option, because specific named metadata will turn out to be very complicated. A name can be easily used twice for different boxes. Paths can be too, but it’s not so likely, as a name is. Thus, we just remove any metadata of parent’s name and we package the VM again.

No matter which option you choose, either Vagrantfile path or VirtualBox name, use existing package.box or existing metadata or create a package from scratch, you will end up with a new Vagrantfile, which the automation script will use to vagrant up and you will get your cloned VM. Just make sure you clone the version of the VM you want and not cloning an older package you forgot to delete.

Vagrantfile explained

You may be wondering what is this secret recipe that the produced Vagrantfile contains.

The new VM option has a Vagrantfile, quite similar to the one you can find at the official repository of nextcloudpi. It uses a debian box (you can specify the release), defines box name and hostname and installs NextCloudPi on it. The automation script will just add the VirtualBox specific commands we mentioned above, about the resources, networking and name of VM.

As for the cloned VM‘s Vagrantfile, it will change the box name from debian to the path of the package.box created, define hostname and VirtualBox configuration and the shell script will not have to do anything about NCP, as it is already installed (clone remember?).

There is one thing though that we have not mentioned yet, which is of utmost importance for cloning a VM.

Vagrant uses ssh to access a VM. In order to do this, it needs an ssh key. So which is it at first boot since we haven’t provided anything?

There is a publicly shared ssh keypair provided by Vagrant, named vagrant insecure key (obviously it’s insecure since everybody can have this). This key is used by Vagrant for the first boot and then it generates a new keypair.

However, since we are actually creating a new base box, we want each NextCloudPi VM to be able to get cloned anytime. Thus, in every Vagrantfile we download the public vagrant insecure key and add it to the authorized_keys file of the VM. Also, Vagrant needs some specific permissions on ssh files (/home/vagrant/.ssh needs 0700 mode and /home/vagrant/.ssh/authorized_keys needs 0600 mode) so we execute the appropriate chmod commands. This constitutes a VM clonable.

Since, the NCP VMs created by the script have the public vagrant insecure key in authorized_keys, we need to show Vagrant which private key to use on first boot by adding the following commands on every Vagrantfile:

  # First boot use vagrant insecure key
  config.ssh.private_key_path = '~/.vagrant.d/insecure_private_key'
  config.ssh.insert_key = false

This way, the know issue of not responding ssh after vagrant package is avoided.

Automation script

Enough talking. Go check it yourself and give me feedback if you find this useful.

The automation script can be found here: https://github.com/eellak/gsoc2019-NextCloudPi/blob/gsoc2019-vagrant/Vagrant/create_vm.sh