Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

function StringToBytesWithNoCopy(s string) in pkg/util/util.go have some problems in GC scenarios #1432

Open
colin-si opened this issue Dec 13, 2023 · 1 comment

Comments

@colin-si
Copy link

Describe the bug
The return result of the StringToBytesWithNoCopy function will be recycled by GC when a large number of objects are created.

To Reproduce
The following demo can reproduce this problem

var ResultKey []byte

func main() {
	key:=[]byte("123456789")
	ResultKey = StringToBytesWithNoCopy(string(key))
	// Result is normal before gc
	log.Printf("before runtime.gc and makeEscape: %v\n", string(ResultKey))
	runtime.GC()
	for i:=1;i<5000;i++{
		makeEscape(time.Now().String())
	}
	// Result has been recycled after gc
	log.Printf("after runtime.gc and makeEscape: %v\n",string(ResultKey))
}

func makeEscape(input string)string  {
	return  input
}

// method copy from servicecomb-service-center/pkg/util/util.go
func StringToBytesWithNoCopy(s string) []byte {
	x := (*[2]uintptr)(unsafe.Pointer(&s))
	h := [3]uintptr{x[0], x[1], x[1]}
	return *(*[]byte)(unsafe.Pointer(&h))
}

// use this method can solve the problem
func StringToBytesWithNoCopyNew(str string) []byte {
	return *(*[]byte)(unsafe.Pointer(
		&struct {
			string
			Cap int
		}{str, len(str)},
	))
}

Expected behavior
When we executed the above demo, we found that the returned results were recycled after GC. This was unexpected. The memory where ResultKey is located is recycled and reallocated for use elsewhere.

2023/12/13 10:57:20 before runtime.gc and makeEscape: 123456789
2023/12/13 10:57:20 after runtime.gc and makeEscape:  m=+0.017

Platform And Runtime (please complete the following information):

Platform

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Runtime

  • Version [e.g. Go 1.11]

Additional context
Add any other context about the problem here.

@hubeoo
Copy link

hubeoo commented Jun 11, 2024

reflect.StringHeader is unsafe

type StringHeader struct {
	Data uintptr
	Len  int
}

Garbage Collector (GC) : Go's garbage collector automatically tracks all valid Pointers to ensure that the memory they point to is not reclaimed. This means that as long as there is a valid pointer to a memory region, that memory region will not be collected by the garbage collector.

StringHeader which uses uintptr is a typeless integer that can be used to hold pointer values, but it is not really a pointer type. Therefore, the garbage collector will not regard uintptr as a valid pointer to track. Causes StringHeader to be affected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants