Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix busted unicode (emoji? chinese characters?) support #79

Open
bitspittle opened this issue Apr 16, 2022 · 4 comments
Open

Fix busted unicode (emoji? chinese characters?) support #79

bitspittle opened this issue Apr 16, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@bitspittle
Copy link
Contributor

Well, nuts. String.length miscounts unicode characters and we might be using it everywhere.

Something like this should render incorrectly

bordered {
   textLine("Hello")
   textLine("Oh my stars ⭐⭐⭐ this just screws it up")
   textLine("Goodbye") 
}

Add stars to test cases, and probably use text.codePoints.count() instead of text.length. Audit all the code...

@bitspittle bitspittle added the bug Something isn't working label Apr 16, 2022
@bitspittle bitspittle added this to the 1.0 milestone Apr 16, 2022
@bitspittle bitspittle changed the title Fix busted unicode support Fix busted unicode (emoji?) support May 16, 2022
@bitspittle bitspittle removed this from the 1.0 milestone Oct 8, 2022
@bitspittle
Copy link
Contributor Author

bitspittle commented Oct 8, 2022

Not a blocker for 1.0.

It's not great, sure, but emojis (many of them anyway) work, and it's in some ways more cosmetic than fundamental. Plus, we can always fix it in the future and the change should be backwards compatible with older codebases.

@mstrobel
Copy link

FYI, this is an especially tricky/vexing issue—the Unicode standard gets updated all the time, and making sure you measure emojis correctly (especially combining sequences like person + hat + skin color) usually means interoperating with a native library. And then there’s East Asian character sets, but at least those don’t change multiple times a year.

@bitspittle
Copy link
Contributor Author

Even worse than I thought 😂 I have given up on this one at the moment. I looked briefly into the places I was using str.length and I recall some case being tricky to handle. I probably should have left a note on this bug what that was...

But yeah I guess first pass would be finding the right third party library to include which did the counting for me.

@bitspittle bitspittle changed the title Fix busted unicode (emoji?) support Fix busted unicode (emoji? chinese characters?) support Mar 30, 2023
@bitspittle
Copy link
Contributor Author

Report from a user. This looks like it's actively causing a program when trying to repaint Chinese characters:

Not working:
https://user-images.githubusercontent.com/43705986/228932658-9ebe0622-0bf2-4157-b193-c6008eaa01e9.mp4

Working:
https://user-images.githubusercontent.com/43705986/228932683-3fb2a15a-4e70-4e12-91a6-8a485ad23ae8.mp4

Code to reproduce:

var success by liveVarOf<Boolean?>(null)
var errorMsg by liveVarOf<String?>(null)

val blinkTotalLen = 1.seconds
val blinkLen = 250.milliseconds
var blinkOn by liveVarOf(false)

section {
    textLine("Please input:")
    items.forEach { codeData -> textLine("  $codeData") }
    text("> "); input()
    success?.let {
        p {
            if (!it)  {
                red { text("×") }; text(" ")
                scopedState {
                    red(isBright = true) {
                        if (blinkOn) invert()
                        textLine("LKJLKJLJLKJLKLHJKHJSLKDJLKSJDLKSJDLKSNM<ADLKSJKDLJSLKDJKLAJSLKAJKLJLKSKLSHFKLSJDKLAJSDLKAJDKLSJDLKSFLJKSHKLFHKLSJDLKAJSLKAJLKJKLSFKLHSKLFJKLS")
//                            textLine("错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误错误")
                        textLine("caused by: ${errorMsg?: ""}")
                    }
                }
            }
        }
    }
}.runUntilSignal {
    onInputEntered {
        if(input.any { !it.isDigit() }) {
            errorMsg = "error "
            success = false
            rejectInput()
            var blinkCount = blinkTotalLen.inWholeMilliseconds / blinkLen.inWholeMilliseconds
            addTimer(blinkLen, repeat = true) {
                blinkOn = !blinkOn
                blinkCount--
                if (blinkCount == 0L) {
                    repeat = false
                }
            }
        }
    }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants