Restoring an accidentally deleted .vagrant directory

Submitted by nigel on Saturday 12th February 2022

I woke up one morning to discover to my horror all my virtual machines had disappeared off my Macbook! This was disconcerting to say the least! Opening VirtualBox UI revealed that it had 'lost' all my VMs - the list of available VMs in the left hand pane was totally empty.

My VMs are kept on a USB memory stick which may have been a contributory factor, but the drive had not been ejected as far as I could tell. 

It was early morning, I hadn't had a coffee when I started to troubleshoot, and I stupidly decided to see if I could vagrant up my most precious project - my football statistics website. Disaster! Vagrant immediately tried to download a fresh copy of Ubuntu from the cloud so I hit ctrl-c. I'm unsure what happened next - perhaps I deleted the .vagrant directory to try and fix things; perhaps Vagrant did. It was probably the former. 

Regardless of blame, I was forced to find a way of regenerating the .vagrant directory so that my football VM would load correctly. These are the steps I took. 

Import VMs back into Virtual Box UI

The first thing to do is import the VMs back into the Virtual Box UI. This is quite easy - click on the Tools part of the UI which will expose the large plus sign, which will open a file system dialog box, then navigate to the directory where the VMs are stored. There will be a directory per VM - open up the directory you want according to its name, then select the .vbox file. That will import the VM. You can see I've already done this for five of my VMs. I will import more of the 30 or so left if I find I need them again!

Re-associate VM with Vagrant
So importing the VM into VirtualBox won't automatically associate the VM with Vagrant. Any attempt to vagrant up will result in Vagrant fetching a new instance from the Cloud. The trick is to manually create the starting point of a .vagrant directory with an id file which will contain the VM's unique identity. The identifier can be obtained by using the VBoxManage list vms command. This can be explained better below:
$ VBoxManage list vms | grep Football
"Football VM" {b2f47455-eee0-457c-93a4-4d0f840598e0}
$ mkdir -p .vagrant/machines/default/virtualbox
$ touch .vagrant/machines/default/virtualbox/id
$ echo "b2f47455-eee0-457c-93a4-4d0f840598e0" > .vagrant/machines/default/virtualbox/id
Before I typed those commands I had already moved to the project's directory. I then got the unique identifier of the VM, recreated the directory structure and the id file, the copied the id into the file.
SSH Authentication
Doing a vagrant up will get you so far - the box will indeed start up (like if you invoked it in the Virtual Box UI) but the authentication will fail. However it's worth doing this and hitting ctrl-c when the authentication fails because Vagrant will recreate the remainder of the .vagrant directory for you and the VM will be running - essential for the next instruction.
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: [vagrant-hostsupdater] Checking for host entries
==> default: [vagrant-hostsupdater] Checking for host entries
==> default: Checking if box 'geerlingguy/ubuntu1804' version '1.1.9' is up to date...
==> default: [vagrant-hostsupdater] Checking for host entries
==> default: [vagrant-hostsupdater] Checking for host entries
==> default: [vagrant-hostsupdater] Checking for host entries
==> default: Clearing any previously set forwarded ports...
==> default: Fixed port collision for 22 => 2222. Now on port 2200.
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 22 (guest) => 2200 (host) (adapter 1)
==> default: [vagrant-hostsupdater] Checking for host entries
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address:
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Authentication failure. Retrying...
    default: Warning: Authentication failure. Retrying...
^C==> default: Waiting for cleanup before exiting...
The SSH configuration can be determined by:
$ vagrant ssh-config
Host default
  User vagrant
  Port 2200
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /Users/nigel/.vagrant.d/insecure_private_key
  IdentitiesOnly yes
  LogLevel FATAL
The SSH key we need to use must be known to the local ssh-agent. Find out which keys are available with the command below. You will almost certainly have at least one entry - the one you use for git.
$ ssh-add -L
Usually the default id_rsa will be available. Pick one. Whichever is chosen, copy the private part of the key into the IdentityFile mentioned above in the output of the vagrant ssh-config command, overwriting the content already in the file. Now do:
$ vagrant halt
$ vagrant up
Wrap Up

Your system shoud now be back! My Vagrantfile runs an Ansible playbook containing my VM config - as can be seen in the screenshot above. Once that completed I could ssh into the VM as normal. All good!!