GCP Storage

Google Cloud Platform (GCP) offers various storage options. The main storage options are : Google Cloud Storage, Google Cloud SQL, Google Cloud Spanner, Cloud Datastore, and Google Cloud Bigtable.

Google Cloud Storage offers highly available object storage. Object storage is not the same as file storage or block storage. Object storage is like you are telling to storage, “Here, keep this arbitrary sequence of bytes,,” and the storage lets you address it with a unique key, which is in the form of URL.

• Encryption: Google Cloud Storage always encrypts your data on the server side, before it is written to disk (no additional charge). Data in-transit is encrypted by default using HTTPS/TLS. In Cloud Storage files are organized into buckets. When you create a bucket: you give it a globally-unique name; specify a geographic location where the bucket and its contents are stored; and choose a default storage class.
• Versioning: Cloud Storage objects are immutable. You can turn on object versioning on your buckets if you want. If you do so, Cloud Storage keeps a?history of modifications. If you don’t turn on object versioning, new always overwrites old. Cloud Storage also offers lifecycle management policies so we can configure to delete older versions.
• Storage Class: Cloud Storage offers storage classes as: Regional, Multi-regional, Nearline and Coldline. Multi-regional and Regional are high-performance object storage, whereas Nearline and Coldline are backup and archival storage. All of the storage classes are accessed using the Cloud Storage API, and they all offer millisecond access times.
• Use case of storage classes: Multi-Regional Storage class is appropriate for storing frequently accessed data like: website content, interactive workloads. Regional storage class is appropriate to store data close to end users or computation. Nearline storage class is a better choice in scenarios where you plan to read or modify your data on average once a month or less. Coldline Storage class is useful for data archiving, online backup, and disaster recovery. Coldline Storage is the best choice for data that you plan to access at most once a year.
• Pricing: All storage classes incur a cost per gigabyte of data stored per month, with multi-regional having the highest storage price and coldline the lowest storage price. Egress and data transfer charges may also apply. In addition to those charges, Nearline storage also incurs an access fee per gigabyte of data read, and Coldline storage incurs a higher fee per gigabyte of data read.
• Transfer data into cloud Storage: There are several ways to bring data into Cloud Storage.?gsutil : command line utility from cloud SDK, online storage transfer service: it lets you schedule and manage batch transfers to Cloud Storage from another cloud provider, from a different Cloud Storage region, or from an HTTP(S) endpoint; Offline Transfer Appliance is a rackable, high-capacity storage server that you lease from Google Cloud. You simply connect it to your network, load it with data, and then ship it to an upload facility where the data is uploaded to Cloud Storage. The service enables you to securely transfer up to a petabyte of data on a single appliance.
• Integration with other GCP services: Google cloud storage is tightly integrated with many of the GCP products and services. For eg. you can import and export tables from and to BigQuery, as well as Cloud SQL, you can store App Engine logs, Cloud Datastore backups. Cloud Storage can also store instance startup scripts, Compute Engine images, and objects used by Compute Engine applications.

gsutil tool?: gsutil is a Python application that lets you access Cloud Storage from the command line.

• Create a bucket: gsutil mb -c standard -l us-east1 gs://my-storage-bucket/ gsutil uses the prefix?gs://?to indicate a resource in Cloud Storage.
• Upload object into bucket: gsutil cp my-local-file.png gs://my-storage-bucket
• Download object from a bucket: gsutil cp gs://my-storage-bucket/my-local-file.png /home/shekhar/downloads/my-local-file.png
• List contents of a bucket: gsutil ls gs://my-storage-bucket
• Make storage bucket publicly accessible: gsutil iam ch allUsers:objectViewer gs://my-storage-bucket
• Remove public access on bucket: gsutil iam ch -d allUsers:objectViewer gs://my-storage-bucket
• Give specific user access to bucket: gsutil iam ch user:myfriend@gmail.com:objectCreator,objectViewer gs://my-storage-bucket
• Delete an object from bucket: gsutil rm gs://my-storage-bucket/my-local-file.png
• Recursive delete bucket and objects in it : gsutil rm -r gs://my-storageawesome-bucket