Figure out a way to manage the discrepancy on windows nodes from the linux node

[dchen1107]

https://github.com/kubernetes/kubernetes/pull/60275/files/2d942dab68b64e684dd2b75232cf9daabc6e0a95#diff-10055ae93a8699af13ceba0482fc43c3

I am expecting we have more windows specific config like code coming to Kubelet and other node components. We should figure out a way to handle this, so that the code can be easily managed? Instead of letting windows-special handling code like this scattered everywhere in our codebase.

cc/ @feiskyer @michmike @kubernetes/sig-windows-bugs

[feiskyer]

@dchen1107 Thanks. This is indeed a problem of windows containers. Usually, we handle this problem in three ways

• (1) By checking runtime.GOOS == "windows"
• (2) By splitting codes into platform specific files, e.g. file_linux.go, file_windows.go, file_unsupported.go
• (3) By splitting codes into different packages.

I think (2) is our preferred way, because (1) would make codes too complex and scattered, while (3) would make codes duplicate.

Windows features have been also broken frequently as no upstream Windows tests are running.
#51540 is aiming to add windows node e2e tests, but the testing environment is still not ready yet. I'd like to clean up related codes after that.

[feiskyer]

/assign

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[feiskyer]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[yujuhong]

@feiskyer is there anything left to do for this bug?

[feiskyer]

@yujuhong this is actually cleanup instead of bug. I think there're still many places needing to improve.

/kind cleanup

[pacoxu]

(2) By splitting codes into platform specific files, e.g. file_linux.go, file_windows.go, file_unsupported.go

(2) is current preferred way. For cmd/kubelet and pkg\kubelet, there are 40 matches across 17 files for runtime.GOOS.

/triage accepted
/help
/good-first-issue

[k8s-ci-robot]

@pacoxu:
This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

In response to this:

(2) By splitting codes into platform specific files, e.g. file_linux.go, file_windows.go, file_unsupported.go

(2) is current preferred way. For cmd/kubelet and pkg\kubelet, there are 40 matches across 17 files for runtime.GOOS.

/triage accepted
/help
/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[pmgk07]

I'd like to help out on this issue. Needed pointers on if anything more than the below is required:

• I find a file with runtime.GOOS
• Move that piece of code in "file" to "file_linux" / "file_windows" / "file_others"
• Tag the platform specific things with // +build linux , // +build windows

[adisky]

I'd like to help out on this issue. Needed pointers on if anything more than the below is required:

• I find a file with runtime.GOOS
• Move that piece of code in "file" to "file_linux" / "file_windows" / "file_others"
• Tag the platform specific things with // +build linux , // +build windows

@pmgk07 right, this has to be done, Make sure do it for all runtime.GOOS(wherever possible) instances

[pmgk07]

Thanks @adisky. Raising a simple PR for ./cmd/kubelet/options package to just get the idea of a general PR workflow in this repo

[sorkinl]

Is this still in need of cleanup? If not I'd like to work on this. @pmgk07
/assign

[pmgk07]

@sorkinl Yes, pkg/kubelet contains runtime.GOOS checks within them.

$ grep -rn "runtime.GOOS" pkg/kubelet|wc -l 
      61

[sorkinl]

I'd like to help out on this issue. Needed pointers on if anything more than the below is required:

• I find a file with runtime.GOOS
• Move that piece of code in "file" to "file_linux" / "file_windows" / "file_others"
• Tag the platform specific things with // +build linux , // +build windows

@pmgk07 right, this has to be done, Make sure do it for all runtime.GOOS(wherever possible) instances

Thank you @pmgk07 ! I will try to follow these instructions and post here with updates.

[raphminkyu]

Hi @pmgk07, I'm working on the issue with @sorkinl . We tried making some changes and I was wondering if you could take a look and comment:

We made changes in pkg/kubelet/volume_host.go and changed

func (kvh *kubeletVolumeHost) GetPodVolumeDir(podUID types.UID, pluginName string, volumeName string) string {
	dir := kvh.kubelet.getPodVolumeDir(podUID, pluginName, volumeName)
	if runtime.GOOS == "windows" {
		dir = util.GetWindowsPath(dir)
	}
	return dir
}

to pkg/kubelet/volume_host_others.go with

//go:build !windows
// +build !windows
...
func (kvh *kubeletVolumeHost) GetPodVolumeDir(podUID types.UID, pluginName string, volumeName string) string {
	dir := kvh.kubelet.getPodVolumeDir(podUID, pluginName, volumeName)
	return dir
}

and made pkg/kubelet/volume_host_windows.go:

//go:build windows
// +build windows

package kubelet

func (kvh *kubeletVolumeHost) GetPodVolumeDir(podUID types.UID, pluginName string, volumeName string) string {
	dir := util.GetWindowsPath(dir)
	return dir
}

Is this the right approach? Do we test with go test ./pkg/kubelet/...?

[pmgk07]

@raphminkyu Sorry for the late reply on this PR. Yes, that looks good enough.

So far these are the files that are in need of refactoring. I grepped on "runtime.GOOS" occurences, might have some false positives.

• pkg/kubelet/cm/devicemanager/manager.go
• pkg/kubelet/kuberuntime/kuberuntime_sandbox.go
• pkg/kubelet/kubelet_test.go
• pkg/kubelet/pluginmanager/pluginwatcher/plugin_watcher.go
• pkg/kubelet/stats/cri_stats_provider_test.go
• pkg/kubelet/kuberuntime/kuberuntime_container.go
• pkg/kubelet/lifecycle/predicate_test.go
• pkg/kubelet/lifecycle/predicate.go
• pkg/kubelet/kubelet_pods.go
• pkg/kubelet/nodestatus/setters.go
• pkg/kubelet/volume_host.go
• pkg/kubelet/kubelet.go
• pkg/kubelet/kubelet_node_status_test.go
• pkg/kubelet/kubelet_node_status.go

@raphminkyu & @sorkinl I'll pick up kubelet_pods / kubelet / kubelet_node_status files (& their corresponding test files) for refactoring.

[sorkinl]

@pmgk07 Thank you this helps a lot. We will start with changes going down the list and will reach out if we have any questions.

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

[pacoxu]

/triage accepted

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted