Creating a Dataset in BigQuery Using the bq Command-Line Tool

Google BigQuery provides the bq command-line tool as a convenient way to interact with datasets and tables directly from your terminal. This guide walks you through different ways to create a dataset using bq mk, including setting metadata like default table expiration, partition expiration, and descriptions. 

Basic Syntax

To create a dataset using the bq command-line tool, you can use any of the following equivalent commands:

bq mk --dataset my_dataset
bq mk -d my_dataset
bq mk my_dataset

 

If you omit --dataset or -d, BigQuery assumes the positional argument is a dataset name.

 

Creating a Dataset with Additional Metadata

You can also define additional properties when creating the dataset, such as:

 

·      Default table expiration (--default_table_expiration): Automatically deletes tables after the specified number of seconds.

 

·      Default partition expiration (--default_partition_expiration): Sets the expiration time (in seconds) for partitions in a partitioned tables.

 

·      Description (--description): Adds a human-readable description to the dataset.

bq mk \
  --default_table_expiration 4000 \
  --default_partition_expiration 3000 \
  --description "my org historical data" \
  project_id:my_dataset

If you're working within your default project, you can omit the project ID:

bq mk \
  --default_table_expiration 4000 \
  --default_partition_expiration 3000 \
  --description "my org historical data" \
  my_dataset

 

Following command create my_org dataset in default project.

bq mk \
  --default_table_expiration 4000 \
  --default_partition_expiration 3000 \
  --description "my org historical data" \
  my_org

$bq mk \
>   --default_table_expiration 4000 \
>   --default_partition_expiration 3000 \
>   --description "my org historical data" \
>   my_org
Dataset 'demo-project:my_org' successfully created.

  

Previous                                                    Next                                                    Home