Quick tutorial on GCP Bucket

 

Google Cloud Platform (GCP) provides 'Google Cloud Storage' service, it allows you to store and retrieve any amount of data at any time. You can use 'Google Cloud Storage' service to store and retrieve any amount of data at any time.

 

Key Concepts

a.   Buckets: Containers to store objects.

b.   Objects: The individual pieces of data that you store in buckets.

 

Why GCP Bucket?

Use GCP Buckets when you need a scalable, secure, and cost-effective storage solution that integrates well with other cloud services and supports a wide range of use cases. Whether you're hosting a static website, archiving data, sharing files, or building a data lake for analytics, GCP Buckets provide the flexibility and performance you need.

 

Imagine you're setting up an online store to sell various products. You want to make sure your website runs smoothly, your data is safe, and your customers have a great experience. Let's see how Google Cloud Storage (GCP Buckets) can help you achieve that.

 

Storing Product images

Your e-commerce site needs to display many product images. You want these images to load quickly for your customers. Store all your product images in a GCP Bucket. This ensures they are quickly accessible to users. You can easily add, update, or delete images as your product catalog changes.

 

Distributing Product Videos and Marketing Materials

Sometimes, you want to show off your products in action with videos or share marketing materials like PDF catalogs. GCP Buckets allow you to store and distribute these files effortlessly. You can even set them up to load fast no matter where your customers are in the world.

 

Integrating with Other Tools and Services

GCP Buckets play well with other Google Cloud services like BigQuery for analyzing customer trends or AI tools for improving your site's search features. This integration allows you to continuously enhance your e-commerce experience based on real data insights.

 

Step-by-Step Instructions to create a GCP bucket

Step 1: Set Up Your GCP Environment

Create a Google Cloud Account: If you don't have a GCP account, create one here (https://cloud.google.com/?hl=en).

 

Step 2: Create a Project: Go to the Google Cloud Console (https://console.cloud.google.com/) and create a new project.

 

Step 3: Cloud Storage -> Buckets.

 

Create a folder in GCP Bucket

Step 1: Get the instance of GoogleCredentials from the credential file. You can download this file from GCP console.

GoogleCredentials googleCredentials = GoogleCredentials.fromStream(new FileInputStream(credentialsPath));

Step 2: Get the instance of Storage.

Storage storage = StorageOptions.newBuilder().setCredentials(googleCredentials).build().getService();

Step 3: Use storage.create method to create a folder.

final String folder = folderName + "/";
final BlobInfo blobInfo = BlobInfo.newBuilder(bucketName, folder).build();
Blob blob =  storage.create(blobInfo);

Find the below working application.

 

StringUtil.java

package com.sample.app.util;

public class StringUtil {

	public static boolean isNullOrBlank(String str) {
		if (str == null) {
			return true;
		}

		return str.isBlank();
	}

	public static void checkNotBlank(String str, String label) {
		if (isNullOrBlank(str)) {
			throw new RuntimeException(label + " is null or empty");
		}
	}

}

BucketUtil.java

package com.sample.app.util;

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.time.OffsetDateTime;

import com.google.auth.oauth2.GoogleCredentials;
import com.google.cloud.storage.Blob;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

public class BucketUtil {

	private String bucketName;

	private GoogleCredentials googleCredentials;
	private Storage storage;

	public BucketUtil(String bucketName, String credentialsPath) throws FileNotFoundException, IOException {
		this.bucketName = bucketName;

		// Load the credentials from the service account key file
		googleCredentials = GoogleCredentials.fromStream(new FileInputStream(credentialsPath));

		// Initialize the GCS client with the credentials
		storage = StorageOptions.newBuilder().setCredentials(googleCredentials).build().getService();

	}

	public Blob createFolder(final String folderName) {
		StringUtil.checkNotBlank(folderName, "folderName");
		final String folder = folderName + "/";

		final BlobInfo blobInfo = BlobInfo.newBuilder(bucketName, folder).build();
		return storage.create(blobInfo);

	}

	public static void printBlobDetails(Blob blob) {
		// Get the blob name
		String name = blob.getName();
		// Check if the blob is a file or a folder
		boolean isDirectory = name.endsWith("/");
		String type = isDirectory ? "Folder" : "File";

		// Get other details
		long size = blob.getSize();
		String contentType = blob.getContentType();
		OffsetDateTime createTime = blob.getCreateTimeOffsetDateTime();
		OffsetDateTime updateTime = blob.getUpdateTimeOffsetDateTime();
		String storageClass = blob.getStorageClass().name();

		BlobId blobId = blob.getBlobId();

		// Print details
		System.out.println("Name: " + name);
		System.out.println("blobId: " + blobId);
		System.out.println("Type: " + type);
		System.out.println("Size: " + size + " bytes");
		System.out.println("Content Type: " + contentType);
		System.out.println("Created: " + createTime);
		System.out.println("Updated: " + updateTime);
		System.out.println("Storage Class: " + storageClass);
	}
}

FolderCreationDemo.java

package com.sample.app;

import com.google.cloud.storage.Blob;
import com.sample.app.util.BucketUtil;

public class FolderCreationDemo {

	public static void main(String[] args) throws Exception {

		// Replace with your bucket name
		String bucketName = "";

		// Replace with the path to your service account key file
		String credentialsPath = "";

		BucketUtil bucketUtil = new BucketUtil(bucketName, credentialsPath);
		String folderName = "images/img1";
		Blob blob = bucketUtil.createFolder(folderName);
		
		BucketUtil.printBlobDetails(blob);

	}
}

Output

Name: images/img1/
blobId: BlobId{bucket=test, name=images/img1/, generation=1719315900358049}
Type: Folder
Size: 0 bytes
Content Type: application/octet-stream
Created: 2024-06-25T11:45:00.389Z
Updated: 2024-06-25T11:45:00.389Z
Storage Class: STANDARD

 

Check bucket access

public  void checkBucketAccess() {
  try {
    // Attempt to get the bucket's metadata
    Bucket bucket = storage.get(bucketName);
    if (bucket != null) {
      System.out.println("Access granted to bucket: " + bucketName);
    } else {
      System.out.println("Bucket does not exist: " + bucketName);
    }
  } catch (StorageException e) {
    System.err.println("Access denied to bucket: " + bucketName);
    System.err.println("Error: " + e.getMessage());
  }
}

Move the files from one folder to other

public void moveFolder(String oldFolderPath, String newFolderPath) {
	// List all blobs in the old folder
	Iterable<Blob> blobs = storage.list(bucketName, Storage.BlobListOption.prefix(oldFolderPath)).iterateAll();

	// List to hold blobs to delete after copying
	List<BlobId> blobsToDelete = new ArrayList<>();

	for (Blob blob : blobs) {
		// Compute the new blob name
		String newBlobName = newFolderPath + blob.getName().substring(oldFolderPath.length());

		// Copy the blob to the new location
		BlobId sourceBlobId = blob.getBlobId();
		BlobId targetBlobId = BlobId.of(bucketName, newBlobName);
		BlobInfo targetBlobInfo = BlobInfo.newBuilder(targetBlobId).build();

		storage.copy(Storage.CopyRequest.of(sourceBlobId, targetBlobInfo));

		// Add the old blob to the delete list
		blobsToDelete.add(sourceBlobId);
	}

	// Delete the old blobs
	for (BlobId blobId : blobsToDelete) {
		storage.delete(blobId);
	}

}

Upload a file to GCP bucket

public Blob uploadFile(String fileName, InputStream inputStream) throws IOException {
	// Create a BlobInfo object
	BlobInfo blobInfo = BlobInfo.newBuilder(bucketName, fileName).build();

	// Upload the file to GCS using the InputStream
	return storage.createFrom(blobInfo, inputStream);
}

Download a file from Bucket

public void downloadFile(String filePath, String destinationPath) throws IOException {
	// Get the Blob from GCS
	Blob blob = storage.get(BlobId.of(bucketName, filePath));

	if (blob == null) {
		throw new IOException("No such object exists in the bucket");
	}

	// Create the destination file
	File file = new File(destinationPath);
	file.getParentFile().mkdirs();
	file.createNewFile();

	// Write the content to the destination file
	try (ReadChannel reader = blob.reader();
			InputStream inputStream = Channels.newInputStream(reader);
			OutputStream outputStream = new FileOutputStream(file)) {
		byte[] buffer = new byte[1024];
		int bytesRead;
		while ((bytesRead = inputStream.read(buffer)) != -1) {
			outputStream.write(buffer, 0, bytesRead);
		}
	}
}

Print all the file names in a folder

public List<String> listFiles(String folderPath) {
    // List the blobs in the specified folder
    Page<Blob> blobs = storage.list(bucketName, BlobListOption.prefix(folderPath), BlobListOption.currentDirectory());
    List<String> fileNames = new ArrayList<>();
    
    for (Blob blob : blobs.iterateAll()) {
      fileNames.add(FileUtil.getFileName(blob.getName()));
    }
    
    return fileNames;
}