Volve dataset - Command line access

Access to the Volve data set can be found here:

https://data.equinor.com/dataset/Volve

One option for downloading the data is to use the AzCopy program from the command-line. This a convenient way to go because it checkpoints the download, allowing it to be restarted if it stops. Some of the Volve data is large (>1 Tb) and the transmission likely to be interrupted.

*Clicking on the AZCopy box under "Geophysical Interpretations" will populate your clipboard with a string like this:

AzCopy /Source:"complicated url" /Dest:"Volve_Geophysical_Interpretations.zip"

To use the AzCopy command, get the AzCopy app here:

https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10

I use linux, so I will demonstrate the linux version of AzCopy, which gets installed in /usr/bin/azcopy.

This linux command will start the download:

azcopy --source "complicated url" --destination "Volve_Geophysical_Interpretations.zip"

But if the transmission is interrupted, you will have to manually restart it. What we need is a way to automatically restart the download if it is interrupted. From the linux command line (using bash), the "until" command creates a loop to do this:

until azcopy --source "complicated url" --destination "Volve_Geophysical_Interpretations.zip" --resume; do echo "Restarting"; sleep 1; done

If the transmission is interrupted, it will print "Restarting" and continue.

The last thing we need is to copy the above command into a text file, let's call it download_volve.sh, this command can be executed on a remote machine using:

nohup ./download_volve.sh >& download_status &

The nohup command means you can log off and let the download complete on its own.