Enable gekhubfeaturemembership fixture test

[ziyue-101]

Checklist

• I did not find a related open enhancement request.
• I understand that enhancement requests filed in the GitHub repository are by default low priority.
• If this request is time-sensitive, I have submitted a corresponding issue with GCP support.

Describe the feature or resource

The current test for TestCreateNoChangeUpdateDelete/gkehub/basic-gkehubfeaturemembership is not working.
We need to enable the test for the newly added Scifi controller so we have more confidence.

https://screenshot.googleplex.com/7sqp73hCpwwpQDw

Additional information

No response

Importance

No response

[ziyue-101]

 testreconciler.go:105: reconcile returned unexpected error: Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Project Service projects/gkehubfm-ji2x6w3yjssjc5ry4rqa/services/: Request `List Project Services gkehubfm-ji2x6w3yjssjc5ry4rqa` returned error: Failed to list enabled services for project gkehubfm-ji2x6w3yjssjc5ry4rqa: googleapi: Error 403: Project 'gkehubfm-ji2x6w3yjssjc5ry4rqa' not found or permission denied.

The test is confused with finding the dependent project.
gkehubfm-ji2x6w3yjssjc5ry4rqa is the metadata.name, but the project is actually acquired by resourceID.

KCC team pre-created project to fix the test, but didn't fix the corresponding references. The project references needs to be fixed with the actual Project ID instead of the KRM Project name.

[ziyue-101]

  testreconciler.go:105: reconcile for ContainerCluster:ciptm34plbljxc2urowa/containercluster-ciptm34plbljxc2urowa took 3.06459604s, result was ({false 0s}, Update call failed: error applying desired state: summary: googleapi: Error 400: Currently, the only supported value for workload pool is "cnrm-test-3rcopwuaaspgmndm-01.svc.id.goog".

the workloadpool for the GKE cluster needs to be updated as well

[ziyue-101]

The test always fails with waiting for the resource.Get returning 404 after deleting the resource

  dynamic_controller_integration_test.go:532: expected GCP client to return NotFound for 'gkehubfeaturemembership-5i5caz4t5cs7e56hly4q', instead got: expected error, instead got 'nil'

Even after extend the wait to be 1 hour, the Get always returns the resource and no errors. see https://prow-gob.gcpnode.com/view/gs/cnrm-prow/pr-logs/pull/cnrm-review.googlesource.com_cnrm/70021/cnrm-test/1790874740928286720#

Trying to get Hub related resources in the test project

$ gcloud container hub features list --project=cnrm-test-mgnkg6ps1b1hii3l-01
NAME                STATE
authorizer          ACTIVE
fleetobservability  ACTIVE
metering            ACTIVE
policycontroller    ACTIVE
$ gcloud container hub features describe authorizer --project=cnrm-test-mgnkg6ps1b1hii3l-01
createTime: '2024-05-16T02:04:07.916816687Z'
name: projects/cnrm-test-mgnkg6ps1b1hii3l-01/locations/global/features/authorizer
resourceState:
  state: ACTIVE
spec: {}
updateTime: '2024-05-16T02:04:08.767685875Z'
$ gcloud container hub features describe fleetobservability --project=cnrm-test-mgnkg6ps1b1hii3l-01
createTime: '2024-05-16T02:04:13.316451779Z'
name: projects/cnrm-test-mgnkg6ps1b1hii3l-01/locations/global/features/fleetobservability
resourceState:
  state: ACTIVE
spec:
  fleetobservability: {}
state:
  fleetobservability:
    monitoring:
      state:
        code: OK
  state:
    updateTime: '2024-05-16T02:05:51.706410970Z'
updateTime: '2024-05-16T02:05:52.553686346Z'
$ gcloud container hub features describe metering --project=cnrm-test-mgnkg6ps1b1hii3l-01
createTime: '2024-05-16T02:04:18.624221949Z'
name: projects/cnrm-test-mgnkg6ps1b1hii3l-01/locations/global/features/metering
resourceState:
  state: ACTIVE
spec: {}
updateTime: '2024-05-16T02:04:20.315573903Z'
$ gcloud container hub features describe policycontroller --project=cnrm-test-mgnkg6ps1b1hii3l-01
createTime: '2024-05-16T02:05:56.161742948Z'
name: projects/cnrm-test-mgnkg6ps1b1hii3l-01/locations/global/features/policycontroller
resourceState:
  state: ACTIVE
spec: {}
updateTime: '2024-05-16T02:06:23.406387287Z'
$ gcloud container hub memberships list --project=cnrm-test-mgnkg6ps1b1hii3l-01
Listed 0 items.

[ziyue-101]

Narrowed down the issue with the PoCo(service) API leak, Internal ticket is b/339458480#comment8

To Sum up, we need to

• Fix the projects references to used the acquired project ID instead of metadata.name
• Remove problematic poco fields in the test.
• Re-enable the test by removing regex filters.