Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Unicode in paths #984

Open
xfc1939 opened this issue Dec 8, 2023 · 6 comments · May be fixed by #998
Open

Support Unicode in paths #984

xfc1939 opened this issue Dec 8, 2023 · 6 comments · May be fixed by #998

Comments

@xfc1939
Copy link

xfc1939 commented Dec 8, 2023

I am using log in Windows, and when the path contains a special space (hex code 0xc2a0), the log file cannot be created correctly."

@xfc1939 xfc1939 changed the title Meet log file create failed when the dir of log file contains a special space what the HEX Code is 0xc2a0 When the path contains a special space (hex code 0xc2a0), the log file cannot be created correctly Dec 8, 2023
@sergiud
Copy link
Collaborator

sergiud commented Dec 8, 2023

Your path contains the Unicode character U+00A0 in terms of a UTF-8 code point. AFAIK Windows does not support UTF-8 encoding for path specification in general. If this is still true, such a path would be indeed invalid.

@xfc1939
Copy link
Author

xfc1939 commented Dec 9, 2023

For non-English countries, it's very common that the path contains Unicode. I have checked the source code and found that the _open function cannot work correctly. Replacing it with _wopen may be a good choice.

@sergiud sergiud changed the title When the path contains a special space (hex code 0xc2a0), the log file cannot be created correctly Support Unicode in paths Dec 9, 2023
@sergiud
Copy link
Collaborator

sergiud commented Dec 9, 2023

Replacing ANSI function variants by their Unicode counterparts is not sufficient. The path handling in environment variables, flags etc. must also be completely reworked.

@xfc1939
Copy link
Author

xfc1939 commented Dec 21, 2023

@sergiud sergiud linked a pull request Jan 3, 2024 that will close this issue
@sergiud sergiud linked a pull request Jan 3, 2024 that will close this issue
@sergiud
Copy link
Collaborator

sergiud commented Jan 3, 2024

Thanks! #786 seems to be related.

@xfc1939
Copy link
Author

xfc1939 commented Jan 9, 2024

Thanks! #786 seems to be related.

The solution from #786 cannot resolve this question. it works if I set FLAGS_log_dir to a unicode path,but It cannot work when the path contains the Unicode character 0xc2a0. for example: log dir

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants