Terraform- GCP Data Proc Component Gateway Enable Issue

Sriram Raghavan

I’m trying to create data proc cluster in GCP using terraform resource google_dataproc_cluster. I would like to create Component gateway along with that. Upon seeing the documentation, it has been stated as to use the below snippet for creation:

cluster_config {
 endpoint_config {
 enable_http_port_access = "true"
 }
}

Upon running the terraform plan, i see the error as " Error: Unsupported block type". And also tried using the override_properties and in the GCP data proc, i could see that the property is enabled, but still the Gateway Component is disabled. Wanted to understand, is there an issue upon calling the one given in the Terraform documentation and also is there an alternate for me to use it what?

software_config {
image_version = "${var.image_version}"
override_properties = {
"dataproc:dataproc.allow.zero.workers" = "true"
"dataproc:dataproc.enable_component_gateway" = "true"
}
}

The below is the error while running the terraform apply.

Error: Unsupported block type
 
 on main.tf line 35, in resource "google_dataproc_cluster" "dataproc_cluster":
35:           endpoint_config {
 Blocks of type "endpoint_config" are not expected here.

RESOURCE BLOCK:

resource "google_dataproc_cluster" "dataproc_cluster" {
  name     = "${var.cluster_name}"
  region   = "${var.region}"
  graceful_decommission_timeout = "120s"
  labels =  "${var.labels}"
  

  cluster_config {
 
        staging_bucket        = "${var.staging_bucket}"
        /*endpoint_config {
               enable_http_port_access = "true"
        }*/
       
        software_config {
            image_version       = "${var.image_version}"
            override_properties = {
                "dataproc:dataproc.allow.zero.workers" = "true"
                "dataproc:dataproc.enable_component_gateway" = "true"  /* Has Been Added as part of Component Gateway Enabled which is already enabled in the endpoint_config*/
            }
        }
         gce_cluster_config {
         // network          = "${var.network}"
          subnetwork       = "${var.subnetwork}"
          zone                = "${var.zone}"
         //internal_ip_only    = true
          tags                = "${var.network_tags}"
          service_account_scopes = [
              "cloud-platform"
      ]
        }

        master_config {
            num_instances     = "${var.master_num_instances}"
            machine_type      = "${var.master_machine_type}"
            disk_config {
                boot_disk_type    = "${var.master_boot_disk_type}"
                boot_disk_size_gb = "${var.master_boot_disk_size_gb}"
                num_local_ssds    = "${var.master_num_local_ssds}"
            }
        }

       
  }
  
  depends_on = [google_storage_bucket.dataproc_cluster_storage_bucket]

  timeouts {
    create = "30m"
    delete = "30m"
  }
}
Sriram Raghavan

Below is the snippet that worked for me to enable component gateway in GCP

provider "google-beta" {
  project     = "project_id"
}

resource "google_dataproc_cluster" "dataproc_cluster" {
  name     = "clustername"
  provider = google-beta 
  region   = us-east1
  graceful_decommission_timeout = "120s"
  

  cluster_config {
        endpoint_config {
               enable_http_port_access = "true"
        }
}

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Component gateway enable issues with dataproc cluster thru terraform

From Dev

Terraform - How to enable API Gateway execution logging?

From Dev

Terraform AWS API Gateway aws_api_gateway_model issue

From Dev

Terraform: Retrieving each subnet gateway address from a list. GCP

From Dev

Terraform - GCP adding IP section on google_compute_vpn_gateway

From Dev

How to enable GCP service agent account via Terraform?

From Dev

Can I automatically enable APIs when using GCP cloud with terraform?

From Dev

How to enable bigtable api (read) on a GCP VM instance using terraform

From Dev

Terraform GCP Data catalog Policy Tag

From Dev

Enable caching for url query parameters in aws api gateway with terraform

From Dev

Enable CloudWatch logs for AWS API Gateway using Terraform

From Dev

GCP: API-Gateway Cors Issue and adding custom authentication

From Dev

Terraform application gateway Data for certificate is invalid

From Dev

Terraform data structures: object and list of object issue

From Dev

Issue Creating IR in Azure Data factory with Terraform

From Dev

How to Enable Block project-wide SSH keys in GCP using terraform

From Dev

Unable to reference an AWS API Gateway Usage Plan as a data source in Terraform

From Dev

How to enable API Gateway endpoint Authorization flag to use AWS Cognito user pool - using Terraform

From Dev

How can I enable CloudWatch logging on an API Gateway v2 API via Terraform?

From Dev

How to Schedule Data Proc pyspark jobs on GCP using Data Fusion/Cloud Composer

From Dev

Terraform nat gateway AWS

From Dev

I have an issue with a vue component not rendering data

From Dev

API data rendering Issue in Child component in React

From Dev

Enable cors on api gateway

From Dev

Service Bus Component of Azure on Premise data gateway - Pricing

From Dev

Cold start in GCP API Gateway?

From Dev

GCP Api Gateway with Firebase Auth

From Dev

FIle upload in GCP API Gateway

From Dev

Frequent errors on GCP API Gateway

Related Related

  1. 1

    Component gateway enable issues with dataproc cluster thru terraform

  2. 2

    Terraform - How to enable API Gateway execution logging?

  3. 3

    Terraform AWS API Gateway aws_api_gateway_model issue

  4. 4

    Terraform: Retrieving each subnet gateway address from a list. GCP

  5. 5

    Terraform - GCP adding IP section on google_compute_vpn_gateway

  6. 6

    How to enable GCP service agent account via Terraform?

  7. 7

    Can I automatically enable APIs when using GCP cloud with terraform?

  8. 8

    How to enable bigtable api (read) on a GCP VM instance using terraform

  9. 9

    Terraform GCP Data catalog Policy Tag

  10. 10

    Enable caching for url query parameters in aws api gateway with terraform

  11. 11

    Enable CloudWatch logs for AWS API Gateway using Terraform

  12. 12

    GCP: API-Gateway Cors Issue and adding custom authentication

  13. 13

    Terraform application gateway Data for certificate is invalid

  14. 14

    Terraform data structures: object and list of object issue

  15. 15

    Issue Creating IR in Azure Data factory with Terraform

  16. 16

    How to Enable Block project-wide SSH keys in GCP using terraform

  17. 17

    Unable to reference an AWS API Gateway Usage Plan as a data source in Terraform

  18. 18

    How to enable API Gateway endpoint Authorization flag to use AWS Cognito user pool - using Terraform

  19. 19

    How can I enable CloudWatch logging on an API Gateway v2 API via Terraform?

  20. 20

    How to Schedule Data Proc pyspark jobs on GCP using Data Fusion/Cloud Composer

  21. 21

    Terraform nat gateway AWS

  22. 22

    I have an issue with a vue component not rendering data

  23. 23

    API data rendering Issue in Child component in React

  24. 24

    Enable cors on api gateway

  25. 25

    Service Bus Component of Azure on Premise data gateway - Pricing

  26. 26

    Cold start in GCP API Gateway?

  27. 27

    GCP Api Gateway with Firebase Auth

  28. 28

    FIle upload in GCP API Gateway

  29. 29

    Frequent errors on GCP API Gateway

HotTag

Archive