No files when generating with --resoruces="*" even tho says imported

[tdiazvelez]

Hi guys, i need to terraform a whole existing infra on google. I have encountered many issues and I was able to fix them reading old issues, but with this I have no clue.

While I import, for example, using:

terraformer import google --resources=gcs,forwardingRules,httpHealthChecks --connect=true --regions=europe-west4 --projects=MY-PROJECT

I get the files correct, 10 for gcs, 1 forwarding rule and 0 healthcheck. And there's a generated/ folder

But when im trying to import everything:

terraformer import google --resources="*" --connect=true --regions=europe-west4 --projects=MY-PROJECT

it seems like everything is working fine (some exceptions of services i dont use, etc) saying "importing done", even listing the ammount of resources for each service found. BUT I have no generated/ folder, why??

[tdiazvelez]

This is the last piece of what i get when i run it with --resources="*"

2023/06/09 12:57:04 Number of resources for service globalForwardingRules: 7
2023/06/09 12:57:04 Number of resources for service pubsub: 0
2023/06/09 12:57:04 Number of resources for service backendBuckets: 0
2023/06/09 12:57:04 Number of resources for service externalVpnGateways: 0
2023/06/09 12:57:04 Number of resources for service monitoring: 0
2023/06/09 12:57:04 Number of resources for service regionInstanceGroupManagers: 5
2023/06/09 12:57:04 Number of resources for service targetPools: 0
2023/06/09 12:57:04 Number of resources for service targetHttpsProxies: 4
2023/06/09 12:57:04 Number of resources for service gke: 0
2023/06/09 12:57:04 Number of resources for service regionDisks: 0
2023/06/09 12:57:04 Number of resources for service regionSslCertificates: 0
2023/06/09 12:57:04 Number of resources for service regionTargetHttpsProxies: 0
2023/06/09 12:57:04 Number of resources for service routes: 47
2023/06/09 12:57:04 Refreshing state... google_compute_image.tfer--huffingtonpost-cms-pre-batch-202210270913
2023/06/09 12:57:04 Refreshing state... google_compute_image.tfer--huffingtonpost-cms-pre-bastion-202210182251
2023/06/09 12:57:04 Refreshing state... google_sql_database.tfer--huffpost-pre-mysql-01e5ea30-information_schema
2023/06/09 12:57:04 Refreshing state... google_compute_image.tfer--huffingtonpost-cms-pre-bo-202211020907
2023/06/09 12:57:04 Refreshing state... google_compute_route.tfer--default-route-1549a13566f4f57f
2023/06/09 12:57:04 Refreshing state... google_project_iam_member.tfer--projects-002F-huffingtonpost-cms-pre-002F-roles-002F-huffpost-002E-pre-002E-varnish-002E-5964a1943190serviceAccount-003A-huffpost-pre-varnish-0040-huffingtonpost-cms-pre-002E-iam-002E-gserviceaccount-002E-com
2023/06/09 12:57:04 Refreshing state... google_compute_url_map.tfer--huffpost-pre-static-alb
2023/06/09 12:57:04 Refreshing state... google_compute_route.tfer--default-route-ae7326bb0756ca14
2023/06/09 12:57:04 Refreshing state... google_compute_global_address.tfer--pre-varnish-alb-address
2023/06/09 12:57:04 Refreshing state... google_compute_subnetwork.tfer--ami-subnet
2023/06/09 12:57:04 Refreshing state... google_compute_image.tfer--huffingtonpost-cms-pre-bo-202209131442
2023/06/09 12:57:04 Refreshing state... google_compute_global_forwarding_rule.tfer--huffpost-pre-static-alb-gfr
2023/06/09 12:57:04 Refreshing state... google_compute_image.tfer--huffingtonpost-cms-pre-static-202211150929
2023/06/09 12:57:04 Refreshing state... google_compute_global_address.tfer--redis-address-f28b2268
2023/06/09 12:57:04 Refreshing state... google_compute_managed_ssl_certificate.tfer--temp-xalok-bo
2023/06/09 12:57:04 Refreshing state... google_compute_region_instance_group.tfer--huffpost-pre-php-instance-group-manager
panic: unknown resource type google_compute_region_instance_group

goroutine 1939 [running]:
github.com/hashicorp/terraform/plugin.(*GRPCProvider).getResourceSchema(0xa4a7223?, {0xa4b5fd0, 0x24})
github.com/hashicorp/[email protected]/plugin/grpc_provider.go:92 +0x10b
github.com/hashicorp/terraform/plugin.(*GRPCProvider).ReadResource(0xc00093b7a0, {{0xa4b5fd0, 0x24}, {{{0xb446618, 0xc001c15aa0}}, {0x8f175a0, 0xc002dca930}}, {0x116927b8, 0x0, 0x0}})
github.com/hashicorp/[email protected]/plugin/grpc_provider.go:321 +0x74
github.com/GoogleCloudPlatform/terraformer/terraformutils/providerwrapper.(*ProviderWrapper).Refresh(0xc0000b7730, 0xc001c3a9b0, 0xc001c3a960)
github.com/GoogleCloudPlatform/terraformer/terraformutils/providerwrapper/provider.go:171 +0x3a9
github.com/GoogleCloudPlatform/terraformer/terraformutils.(*Resource).Refresh(0xc0023fdef0, 0x52?)
github.com/GoogleCloudPlatform/terraformer/terraformutils/resource.go:130 +0x4e
github.com/GoogleCloudPlatform/terraformer/terraformutils.RefreshResourceWorker(0x0?, 0x0?, 0x0?)
github.com/GoogleCloudPlatform/terraformer/terraformutils/utils.go:153 +0xad
created by github.com/GoogleCloudPlatform/terraformer/terraformutils.RefreshResources
github.com/GoogleCloudPlatform/terraformer/terraformutils/utils.go:80 +0xe7

[DanielRieske]

What are the versions of your Terraformer and the terraform-google-provider that you're using?

[tdiazvelez]

@DanielRieske

Hi Daniel, this are my versions. BTW i updated my first comment with the output, maybe is useful :)

Terraform: v1.4.6
Terraformer: v0.8.22
gcp provider: v4.0.0 because using a newer one (i think latest its 4.6.something) explodes everywhere and i found HERE a solution using 4.0.0

[DanielRieske]

@tdiazvelez Had to dig a little bit but this is because google_compute_region_instance_group is not a resource but a data source.
As far as I know Terraformer only creates terraform_remote_state data sources based on the resources it has but doesn't create any other.

You can solve this by excluding it by

terraformer import google --resources="*" --excludes="regionInstanceGroups" --regions=europe-west4 --projects=MY-PROJECT

@sergeylanzman want me to create a PR to remove the google_compute_region_instance_group resource?

Also, about the issues with GCP provider, the issue started in 4.60.0 due to the plugin framework added back in, so using version 4.59.0 will work.

[tdiazvelez]

thank you @DanielRieske , now its working :)

Hope you have a great weekend!