New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updating to Nom 7 has broken double("NaN")
#1384
Comments
Personally I do not understand why we take Minimal-Lexical there are soo many con, unfortunatly I see the change when it was already done, Rust 1.55 just landed and have a new float parser that is expected to be fast https://blog.rust-lang.org/2021/09/09/Rust-1.55.0.html#faster-more-correct-float-parsing we should maybe considering use it instead. |
I'm the author of that PR, and the author of minimal-lexical. Geal/nom isn't supporting only v1.55, only v1.48+, but ideally, once the MSRV increases, yes, this dependency should be removed. Currently, the performance gap between 1.48-1.54 versions of float parsing is very notable, so minimal-lexical is meant to patch the gap. Also, |
Ok I know where this is a bug: the implementation does not check for special values right now, which minimal-lexical doesn't handle internally. I'll submit a PR shortly. |
I'm concern with perf penalty this do, cause Gael compare bench with different feature it was an unfair benchmark. |
I haven't seen Geal's exact benchmarks, but I can tell you the performance is a fair bit faster prior to 1.55.0 for a relatively comprehensive suite of floats. After 1.55.0, standard should be a bit faster if we don't have to tokenize the float (but this isn't the case), so minimal-lexical should actually still be faster, since core does a lot of duplicate work. This is because there is no support for a partial parser in core. And, prior to 1.55.0, once again, the algorithm for float parsing was substantially worse, which is why I spent a lot of time implementing that. If after these fixes, you want me to benchmark the difference, I'd be happy to do so. |
My problem is that while I don't want to say your work is not amazing but I would like a more neutral argument, since you are the creator of
But lexical-core is now 0.8, mean your list of pro become small add that I don't really agree with some
I don't care rust is slow anyway and
I don't know much so no opinion I expect
Why dependency would be a problem specially https://crates.io/crates/lexical-core/0.8.0/dependencies only dependency are internal tool to split crates.
lexical-core 0.8 require And we end with two cons:
I add one
The main pro was benchmark, but what if are the number versus |
I don't mind either way to use lexical-core vs minimal-lexical (hi 👋👋 I maintain both, and I'm happy to ensure that people don't have to use my crates, which is why I implemented the PR into Rust core, and have made numerous PRs to
minimal-lexical is different than lexical-core. lexical-core v0.8 is certainly faster to compile than v0.7, but it can't compete with a dedicated, small, isolated crate.
Many people do care about compile times.
lexical-core does have a relatively simple API, but once again, it's not possible to have only one function in lexical-core. The high-level API is ~4 parsing functions, although it's relatively moot: the main point is compile times.
Because dependencies add compile time issues? I added
I mean, sure, but it's also a useful potential feature later.
nom7 has already maintained to supporting v1.48+, and lexical-core 0.8 cannot be re-written to support versions older than 1.51, due to the use of const generics integral to the new implementation.
minimal-lexical is actually used in a private fork by serde-json, so it's unquestionably more used. The maintainers are the same for both lexical-core and minimal-lexical, and maintenance plans have been put into motion if I cannot maintain.
Sure, but there is a compile time difference. minimal-lexical compiles on my computer in ~0.5s, while lexical-core v0.8 with only the float-parsing is ~1.5s. This is substantially faster than the nearly 3.5s of v0.7, but it's worth considering there's a tradeoff for both cases. |
Anyway, I'm fine with any of the options:
|
I'm not talking about compile time here, but benchmark in actually doing the job. That in my opinion is what matter the most in crate like nom.
you didn't understand what I wanted to said, I mean that nom 7 would have require a most recent rust version if needed. that was not a big deal.
|
I understood, I pointed out numerous times I don't agree. Although compile time isn't the only factor, minimal-lexical is also fairly performant.
I appreciate that. Anyway, I'll do my best to ensure float parsing is viable for everything. In the future, if the new slow-path algorithms are implemented in fast-float-rust, that might be the most logical option. |
so, to sum up here:
I think we can wait a bit before considering a change of float parsing library. Personally, I am not fond of arguing about the right way to parse floats in nom: the parsers were added for convenience (and for benchmarks, unfortunately), but there's no guarantee that they would parse exactly like people expect in their formats. So they have to provide a minimal feature set for prototyping, then users can make their own tokenizer and pass to something like minimal-lexical or libstd's new float parser, or rely directly on lexical-core and its various format options. |
For the record, I have worked around this issue in our app so there is no urgency on my part to resolve this. Nor do I have a particularly strong opinion on the correct way it should be done. |
#1421 (comment) works around this issue as well. |
Given the following code:
In Nom version 6, this runs and returns:
In Nom version 7, this returns:
Maybe this is intentional? I'm not seeing anything in the release notes about it, however.
The text was updated successfully, but these errors were encountered: