www 101

All you need to know about the internet

Have a Question?

If you have any question you can ask below or enter what you are looking for!

Google Cloud Storage: Managing Buckets

Google Cloud Storage (GCS) is a very simple and powerful object storage offering from Google as a part of its Google Cloud Platform (GCP). It provides a highly durable, scalable, consistent and available storage solution to developers and is the same technology that Google uses to power its own object storage. 

It is billable with a pay for what you use model, and GCP comes with a 60-day trial period, so it is free to try out to see if it fits the needs of your organization. GCS has different service levels (also called storage classes) which can be chosen as needed (detailed discussion on these is out of the scope of this tutorial). GCS can be used for a variety of purposes, such as serving static/dynamic website content, storing user-specific application files, disaster recovery, or making large data objects downloadable to users.

Those who have worked on GCP will know that everything in GCP revolves around projects. Each project can have many buckets around which the architecture of Google Cloud Storage is structured. Buckets are the basic containers on GCS that contain the data stored. These are used as basic blocks to organize your data and look like folders on an operating system, but they cannot be nested. 

Each bucket can contain any number of objects, which can be folders and/or files. A bucket is assigned a storage class and geographic location when being created. These settings can be specified while creating the bucket but cannot be changed later.

Buckets have specific naming conventions which need to be followed strictly, otherwise GCP will not allow you to create a bucket. Bucket names are globally unique, so they need to chosen in a way that prevents conflicts. However, a name used by a deleted bucket can be reused. 

Also, the name cannot be changed once it’s been assigned to a bucket. The only solution if you want to change it is to create a new bucket with the desired name, move the contents from the previous bucket to the new one, and then delete the previous bucket.

In this tutorial, I will cover how to manage buckets from the Google Cloud Console. This is followed by a Python script where I will demonstrate performing the same operations programmatically.

Using Google Cloud Console

First, let’s see how to manage buckets using the web user interface provided by GCP known as Google Cloud Console

Open Storage Browser in a web browser of your choice. If you are a first-time user, you will be prompted to create a project first. Also, an option will be shown to sign up for a free trial. Go ahead with the free trial signup, otherwise you will not be allowed to create a new bucket by yourself. By default, GCP only provides one free bucket per App Engine instance. 

Once done with all these formal processes, navigating to this page should open up the page shown below.

Google Cloud Storage Browser

To create a new bucket, click on the Create Bucket button highlighted above. Create a bucket by filling in a desired name as shown below. The name should follow the bucket naming conventions.

Creating a new bucket

After you’ve created a bucket, the GCS browser will list it. Buckets can be deleted by selecting them from the list and clicking on the delete button.

Delete a bucket

Clicking on the refresh button will populate the UI with any changes to the list of buckets without refreshing the whole page.

Managing Buckets Programmatically

First, let’s create a Google Compute Engine instance as that will allow quick demonstration of the targeted concepts rather than dealing with extra authentication steps on local machines. To create a GCE instance, open the link and click on the Create Instance button as shown below.

Create a new Compute Engine instance

A form will come up asking for relevant details, which can be filled in at your convenience. Once the GCE instance is created, open up the SSH client as shown below, which by default opens in a new browser window.

SSH into Compute Engine instance

The SSH client screen will look something like as shown below. All the further operations in this tutorial will be done directly on the SSH client itself.

SSH Client for newly created Compute Engine instance

Writing a Python Script

Below are the commands you need to run to set up the newly created server for a Python development environment.

Below is the dependency that needs to be installed for writing this script.

On production systems, it is not advisable to install libraries using “sudo”. Please follow Python virtualenv best practices for this.

gcs_bucket.py

The above Python script demonstrates the major operations that can be performed on a bucket. These include:

  • creation of a new bucket in a project
  • listing of all buckets in a project
  • getting details of a specific bucket
  • deleting a specific bucket

Let’s see what these operations look like when the script is run.

Conclusion

In this tutorial, you saw how to manage buckets on Google Cloud Storage. This was also accompanied by a small introduction to creating a Google Compute Engine instance and using it via an SSH client.

In the next tutorial, I will cover how to manage objects, i.e. folders and files inside a bucket.