Improving serialization layer for JS applications

[Ihromant]

Hi everyone. I extracted java-js converters I'm using for serialization-deserialization into separate project https://github.com/ihromant/teavm-io .
It's JavaToJS and JSToJava converters inspired by not supported project teavm-flavour. I used metaprogramming to implement functionality for serialization similar to the following Jackson configuration (this is copy of configuration from my server-side):

private final ObjectMapper map = JsonMapper.builder()
            .serializationInclusion(JsonInclude.Include.NON_NULL)
            .enable(SerializationFeature.WRITE_ENUMS_USING_INDEX)
            .configure(MapperFeature.PROPAGATE_TRANSIENT_MARKER, true)
            .build();

For now it's enough for my needs. But I'm thinking about the future. So, the questions are the following

1. Is it possible to make Converters class configurable? Currently using of Converters is possible from static context only (and compilation-based only). I would like to be able to do something similar to ObjectMapper configuration (with possibility to change it in runtime if it's possible):

private final Converters map = Converters.builder()
            .serializationInclusion(JsonInclude.Include.NON_NULL)
            .enable(SerializationFeature.WRITE_ENUMS_USING_INDEX)
            .configure(MapperFeature.PROPAGATE_TRANSIENT_MARKER, true)
            .build();

and then use it wherever I need.
2. I was not able to run the test. I tried to do it based on MetaProgrammingTest in teavm, but this test is just skipped. How to fix it?
3. Is it possible to "hide" not needed classes (actually only "Converters" class is needed)? I tried to make project structure flat and make only Converters class public, but it leads to strange errors during teavm compilation.

[konsoletyper]

As for pt.2, you could have used Gradle, with it this is more convenient. However, since you have a Maven-based build, you should pass additional system properties manually. Please, refer to documentation for specific Maven plugin, e.g. maven-surefire-plugin, to learn how to do that. You can find list of system properties here.

[Ihromant]

Tried suggested configuraition. It worked. But two issues.

1. It creates *** folder every test run when run maven from console. I suppose documentation on teavm.org suggests not the best confguration and correct value for this variable.
2. I was not capable of running tests via IDEA. I tried to put variables with -D and without it ro VM options and environment variables. Nothing helped.

[konsoletyper]

1. Did you just copy everything from example? It was not ready-to-use configuration, just kind of pattern. Instead of *** you should put whatever value you want. Please, refer to Maven documentation. Or better switch to Gradle where everything ready-to-use
2. Have no idea why it's not working. It used to work on my side when I used Maven/IDEA. You can put breakpoints somewhere in TeaVMTestRunner and see what's going on.

[Ihromant]

1. Renamed it to target/js-tests and it helped. Not it is created inside target folder. I suppose improvement for maven plugin could be the following: setting target/js-tests value as default. So the only config needed would be runner.
2. I suppose I made mistake somewhere, managed it to work. Thanks for help.

[konsoletyper]

I suppose improvement for maven plugin could be the following: setting target/js-tests value as default

It's not possible, because tests are run by surefire-maven-plugin, not by teavm-maven-plugin. That's the reason to switch to Gradle. I was ignoring Gradle for many years due to poor integration with IDEA. Now looks like JetBrains fixes most of bugs and reached a good performance, so it's not an issue anymore. In Gradle 'plugin' is completely different thing. 'Mojos' in Maven are something like 'tasks' in Gradle. Maven plugin contributes mostly mojos, nothing more. Gradle plugin just takes reference to Project and is free to do anything with them, e.g. adding new tasks, connecting tasks, configuring existing tasks, etc. This is how teavm gradle plugin works in case of tests: it detects all Test tasks and simply adds all necessary system properties to them, see in the source code.

[konsoletyper]

As for pt.1, I don't fully understand the issue here. Consider you want to configure whether enums are written by index or by name. Then you declare abstraction like EnumSerializer interface, provide implementations for name- and index-based approaches and declare enumSerializer field in Converters. Then, rewrite your serializer generating code to take this field upon constructions of a serializer of class, containing enum. Sure, this would make generated JS grow. If you want avoid latter, then you can configure at compile-time. You can use any kind of configuration you want, for example, based on ServiceLoader, or read TeaVM properties from plugin.

BTW, in my proprietary JSON serializer/deserializer, I don't ever use this Jackson-based approach and configure everything explicitly. I find the way provided by Jackson a bit confusing: when you see at JSON and try to understand why it was generated this way (or rather, why given JSON is not properly parsed), looking on annotations is not enough and you should also find out where particular configuration is created.

One suggestion to Converters API: please, open Serializer/Deserializer interfaces and corresponding Converters methods to construct them. In some scenarios extra lookup in a hashtable may be a performance issue. It's also reasonable to make these interfaces generic.

[Ihromant]

Got it. As I suspected, it is more complicated than I thought. Will return to this topic when this changes will be introduced.
For now it just requires stable work (which it does, now covered by tests), don't think that I'll do big changes in recent future.
Just curious, what is general idea in proprietary JSON serializer? Is it also teavm-only approach?
Marking this as answer.

[konsoletyper]

No, it's not TeaVM-only approach. It's based on bytecode instrumentation/generation and integrated into various builds (TeaVM, JVM, Android, GraalVM). BTW, Metaprogramming API can also be implemented over JVM with bytecode instrumentation techniques, I'm just to busy to do it.

[konsoletyper]

As for pt.3, please, provide instructions to reproduce. I could not.

You can avoid making your project structure flat by introducing another module that contains all code generation stuff. Add this module to impl module as a dependency with provided scope.

[konsoletyper]

Ah, I managed to reproduce 'strange errors'. You should mark all your code generation related classes with @CompileTime annotation.

[Ihromant]

Got it, thanks.