Description:
When you use Dataproc, cluster and job data is stored on Persistent Disks (PDs) associated with the Compute Engine VMs in your cluster and in a Cloud Storage staging bucket. This PD and bucket data is encrypted using a Google-generated data encryption key (DEK) and key encryption key (KEK). The CMEK feature allows you to create, use, and revoke the key encryption key (KEK). Google still controls the data encryption key (DEK).
Rationale:
"Cloud services offer the ability to protect data related to those services using encryption keys managed by the customer within Cloud KMS. These encryption keys are called customer-managed encryption keys (CMEK). When you protect data in Google Cloud services with CMEK, the CMEK key is within your control.
Using Customer Managed Keys involves additional overhead in maintenance by administrators.
From Google Cloud Console
From Google Cloud CLI
Before creating cluster ensure that the selected KMS Key have Cloud KMS CryptoKey Encrypter/Decrypter role assign to Dataproc Cluster service account ("serviceAccount:[email protected]").
Run clusters create command to create new cluster with customer-managed key:
gcloud dataproc clusters create --region=us-central1 --gce-pd-kms-key=
The above command will create a new cluster in the selected region.
Once the cluster is created migrate all your workloads from the older cluster to the new cluster and Run clusters delete command to delete cluster:
gcloud dataproc clusters delete --region=us-central1
Repeat step no. 1 to create a new Dataproc cluster.
Change the project by running the below command and repeat the remediation procedure for other projects:
gcloud config set project "