Add simple preferences API

[aevyrie]

Objective

• Partially implements Bevy Preferences API #13311

Solution

// Preferences only require that the type implements [`Reflect`].
#[derive(Reflect)]
struct MyPluginPreferences {
    do_things: bool,
    fizz_buzz_count: usize
}

fn update(mut prefs: ResMut<Preferences>) {
    let settings = MyPluginPreferences {
        do_things: false,
        fizz_buzz_count: 9000,
    };
    prefs.set(settings);

    // Accessing preferences only requires the type:
    let mut new_settings = prefs.get::<MyPluginPreferences>().unwrap();

    // If you are updating an existing struct, all type information can be inferred:
    new_settings = prefs.get().unwrap();
}

• Provide a typed key-value store that can be used by any plugin. This is built entirely on bevy_reflect, so the footprint is pretty tiny.
• Choosing where to store this to disk is out of scope, and is closely tied to platforms and app deployment. That work can be done in follow up PRs. The community can build tools on top of this API in the meantime. It is completely agnostic to file saving strategy or format, and adds no new concepts.
• Use the struct name (from reflect) as the key in the map. This keeps implementation simple, and makes things like versioned preferences trivial for users to implement. This might be the most controversial aspect of the PR.
• While it can be made easier, I've added a unit test that demonstrates how preferences can be round tripped through an arbitrary file type using serde.

Open Questions

• Where should this go? bevy_app seems wrong, but I'm unsure where to put it.

Testing

• Unit tests have been added to the module.

---

Changelog

• Added a Preferences resource to act as a centralized store of persistent plugin state. This does not yet implement serialization of any data to disk, however it does give plugin authors a common target for adding preferences.

[bushrat011899]

I like this, but I'm concerned about how (as the implementer of a storage API) I can take this Preferences resource and save it, while remaining ignorant of the exact preference types. Otherwise, I think this is a good starting point for a preferences API. Simple and elegant.

[bushrat011899]

I believe Preferences needs some kind of de/serialisation support to really be useful. Ideally, a generic FilePreferencesStorePlugin could store/load the entire preferences resource to disk, as an example.

[aevyrie]

That's why this is based on reflect. You can serialize the map however you'd like, in any format you'd like. I can't implement reflect on Preferences as-is, but it possible with some elbow grease for the internal map.

This is also discussed in the OP.

[aevyrie]

I proved this out in a test:

#[derive(Reflect)]
struct Foo(usize);

#[derive(Reflect)]
struct Bar(String);

fn get_registry() -> bevy_reflect::TypeRegistry {
    let mut registry = bevy_reflect::TypeRegistry::default();
    registry.register::<Foo>();
    registry.register::<Bar>();
    registry
}

#[test]
fn serialization() {
    let mut preferences = Preferences::default();
    preferences.set(Foo(42));
    preferences.set(Bar("Bevy".into()));

    for (_k, value) in preferences.map.iter() {
        let registry = get_registry();
        let serializer = bevy_reflect::serde::ReflectSerializer::new(value, &registry);
        let mut buf = Vec::new();
        let format = serde_json::ser::PrettyFormatter::with_indent(b"    ");
        let mut ser = serde_json::Serializer::with_formatter(&mut buf, format);
        use serde::Serialize;
        serializer.serialize(&mut ser).unwrap();

        let output = std::str::from_utf8(&buf).unwrap();
        println!("{:#}", output);
    }
}

which prints

{
    "bevy_app::preferences::tests::Foo": [
        42
    ]
}
{
    "bevy_app::preferences::tests::Bar": [
        "Bevy"
    ]
}

[bushrat011899]

That makes sense, thanks for the clarification! For now, could you mark the inner map as pub? That way if we don't get a follow-up PR we can still get utility out of this?

[aevyrie]

I've added this test and included fully roundtripping a file. The test uses JSON, but you could use whatever format you'd like.

[aevyrie]

I've provided iter_reflect, which gives all the data needed to serialize the data, and set_dyn to deserialize the data, without needing to make the inner map pub.

[aevyrie]

Someone smarter than me can figure out how to implement reflect, because it will require an assembly step to convert between the list of trait objects in the file, to a map in memory.

[MrGVSV]

I can't implement reflect on Preferences as-is, but it possible with some elbow grease for the internal map.

Have you tried this?

Suggested change

	#[derive(Resource, Default, Debug)]
	#[derive(Resource, Reflect, Default, Debug)]
	#[reflect(from_reflect = false)]

DynamicMap doesn't implement FromReflect, so you'll have to opt out of FromReflect for Preferences, but it should still be possible.

Note that when deserializing Preferences, you'll get back a DynamicStruct. Since you can't use FromReflect to convert it back to Preferences, you might need to use Default/ReflectDefault and patch with Reflect::apply.

[aevyrie]

This is pretty outside of my comfort zone with Reflect. I had a sense that I needed to do something along those lines, but I have no idea what they do or what tools to reach for. If you can find a way to do that, I would appreciate a PR against this one!

[tychedelia]

I like building on top of reflect but my intuition is that having an additional untyped key will be necessary to be useful. For example, a user may have the same preferences struct but use it in a few different locations within their app which need to be disambiguated when persisting the preference data. These uses may themselves by dynamic (i.e., providing preferences for user created entities), which means they can't just be solved by getting more generic.

[aevyrie]

If you want to have multiple instances of a type inside your plugin's namespace, you can, but there should be a single canonical version of your preferences struct for your plugin. This is as simple as wrapping all config information into a single type. This is no different from, say, vscode preferences where every extension has its own namespace in the setting map.

Think of the type as the namespace of that plugin's preferences. Though, you aren't locked to a single type. This makes it possible to handle cases like MyPluginPrefsV1 -> MyPluginPrefsV2

[tychedelia]

Think of the type as the namespace of that plugin's preferences.

Sure, that makes sense. Of course you can build any kind of dynamism/metadata into the preference struct itself. I think I was considering something a bit more granular but this totally makes sense to me and keeps things simple. 👍

[inodentry]

I would like to point out a specific design consideration that IMO should be considered.

It should be possible for application developers to use preferences to store things related to how the app/engine itself is set up (rendering-related things are a common example of this):

• Preference for wgpu backend? Render device?
• Preference for window mode / fullscreen?
• Preference for enabling/disabling bevy's pipelined rendering?
• etc ...

All of these are things that must be known at app initialization time, if they are to be customizable via preferences.

This is a tricky problem that IMO needs to be solved, because it is a perfectly normal and reasonable use case to want to store these kinds of things in preferences.

In my projects, I have my own bespoke preferences architecture. I solve this problem by loading the preferences file from fn main before creating the Bevy app, so that I can do things like configure the app itself (such as choosing what plugins to add or disable) based on preferences. And I just insert it into the World via the App, so that it is always available from within bevy, from the very start.

[aevyrie]

That seems like an implementation detail of a storage backend. There's no reason you can't load preferences into the resource on - or even before - the app starts.

Edit: kinda-sorta. you need data from the type registry before you can load preferences. Considering that is an asynchronous task, that suggests that if something needs preference data on startup, it needs to allow the schedule to run to fetch that data.

[inodentry]

Another thing I would like to point out is that we should consider how ergonomic it is to handle changes to preferences at runtime, for those preferences where the application developer wants to support that.

Imagine a complex application. Over the course of development, I add more and more preferences, and I want to handle runtime changes for many of them (but probably not all, there are likely to be some where it is impossible). I don't want the handling of changes to preferences to be scattered all over the codebase in random systems, as that would quickly become unmaintainable. There should be a more organized way to deal with this somehow.

Perhaps some way to run hooks / callbacks / one shot systems associated with the type of each preference? Some sort of trait?

And it should also be possible to control when that gets "applied". There should be a way to change values without applying them immediately, and then applying the changes later.

[inodentry]

Or at the very least, per-preference change detection.

Then it would be possible to easily implement hooks or whatever abstraction on top of that.

[viridia]

An alternative approach would be for each preference item to be a separate resource, and have Preferences be a HashSet of resource ids. This would give you somewhat more granular change detection.

You could have the plugins for individual crates register their preference types at app initialization time. This means that Preferences is built automatically when the plugins are executed. I've done something similar for widget factories for the reflection-based property inspector.

[viridia]

Alternatively, don't register preferences at all, instead register loaders. That is, you have some trait PreferenceLoader, analogous to AssetLoader, and you have a registry of impls of these. The loader can store the data however it wants, resource, component, etc.

[aevyrie]

Re: #13312 (comment)

I think all of that is worth discussing, but I'd like to table that and keep it out of scope for this initial PR. I think that what is here is enough to start toying around with, and for other prototypes built on top of. I'm pretty confident this approach will be flexible enough to accommodate those needs without drastically changing the API.

The current API locks down access behind getters and setters, so it wouldn't be a stretch to imagine adding change flags to each entry, and having a system execute a callback stored alongside any changed entries.

[alice-i-cecile]

This is really well-crafted, and a fantastic starting point for useful functionality.

I do wonder about using a "config-as-resources" pattern like in bevyengine/rfcs#77 in the future. Which is similar to what @valaphee suggested in the linked issue. It would also allow for the granular change detection requested by @inodentry above :) Not worth blocking over though: this is a sensible and unobtrusive design that would be straightforward to refactor.

I'm also not sure what crate this best belongs in. bevy_core feels a bit more appropriate to me? It's "core, useful functionality", not "part of the app abstraction". If you agree, please move it in this PR. If you disagree, just let me know and I'll swap this to an approval regardless.

[NthTensor]

Ditto on bevy_core.

I was on the fence about it, but I think maybe we do need more fine-grained change detection. There may be some preferences (specifically in graphics) which require restarting the window/canvas or even the entire game. It would be nice to collect these together such that if a that specific preferences struct changes the game automatically restarts, but changing other unrelated preferences doesn't trigger a restart.

Not blocking though. Lets merge this and spin out some issues.

[bushrat011899]

On change detection, perhaps a system could take the specific types out of the Preferences resource and insert them into the world, perhaps even bypassing change detection of equivalent values. This would give granularity in reaction to preferences, and also make it easier to access the specific preference values you're after.

But crucially, I think that's something that could be added in a follow-up PR without any changes here.

As for location, bevy_core until we have a need for elsewhere seems sensible. I imagine if Bevy is going to provide browser backed storage on WASM, and file/registry on desktop, etc., that might be well suited in its own crate due to the odd dependencies (bevy_persistance, etc.)

[MrGVSV]

I think one way to achieve fine-grained change-detection-like behavior would be to make Preferences a SystemParam that modifies a resource internally. That way we can send some sort of PreferenceChange event which contains the type/key of the preference that was changed. Users can then react to those changes however they wish.

We could also make a separate PreferencesMut if we're worried about blocking with the mutable reference to Events.

[viridia]

I think that this PR is punting on too many things and isn't opinionated enough. The problem with "leaving things up to the developer" is that it risks each crate/plugin author deciding things differently and incompatibly. If we want multiple crates to be able to store their preferences together, then they need to agree on format.

There are really two parts to the preferences problem, and this PR only addresses one of them: how preferences are stored in memory. That's the easy part. The other part is how they are stored and serialized, and that IMHO is really the more valuable part from the app developer's perspective. The reason it's more valuable is that it's easy to write unit tests for an in-memory data structure, but hard to test serialization on different platforms unless you have the hardware.

This is based on an assumption which we might not share: I think that part of Bevy's value proposition should be to minimize the amount of work needed to get the app to run on multiple platforms. I know that this work can't be reduced to zero, but it should be as small as possible.

Finally, one question: would it be better to develop this as a standalone crate (bevy_mod_preferences)? I can see pros and cons either way. Making it part of Bevy means greater involvement by contributors. On the other hand, because this API is still evolving, and because issues are not settled, I don't want to freeze the design too early by having a whole bunch of users depend on the API not changing.

[viridia]

I'm a bit confused because the comment says "valid JSON map", but it's pushing a bracket, which starts a list.

I believe it's more idiomatic in JSON to use a map type for type/value pairs, where the type name is the key, rather than a list. This doesn't allow duplicate types, but that's the case here anyway.

[viridia]

It's possible, by the way, to use serde to serialize the entire map. I have previously written a serde custom serializer that uses Bevy Reflect internally: https://github.com/viridia/panoply/blob/main/crates/panoply_exemplar/src/aspect_list.rs#L31

[aevyrie]

Yeah, this is a list, the comment is wrong. This was a case of "it's 2AM and I want to build a serialization round tripping test to prove it is possible". As I noted in that section, there is definitely a better way to do it, but this was enough to get it running and working. :)

[cart]

From my perspective, Plugins already have a preferences API:

pub struct AssetPlugin {
    pub file_path: String,
    pub processed_file_path: String,
    pub watch_for_changes_override: Option<bool>,
    pub mode: AssetMode,
}

Notably, these preferences are stored on Apps (and accessible) at runtime:

let asset_plugin = app.get_added_plugins::<AssetPlugin>()[0];

Not the ideal runtime API (not accessible on World, returns a Vec), but thats a solvable problem.

Also, notably, the vast majority of these plugin settings could be serializable.

This is the current "plugin settings" API, so any proposal should describe how it relates to it / justify its existence relative to it. What does Preferences enable that the current plugin settings approach could not support / evolve into? Can (and should) we unify the two concepts? Should we remove plugin settings as they are currently defined?

Note that we moved away from "plugin settings as resources" in this PR in favor of the current approach. This Preferences proposal has a lot in common with that pattern (just using a centralized resource instead of individual resources). Can you read through the rationale in that PR for moving away and give your thoughts?

[aevyrie]

The (unclear) distinction here is that using Preferences gives you a place to declare state that you, as the plugin author want to be globally persistent.

Notably, these preferences are stored on Apps (and accessible) at runtime:

I had no idea this was the case, and I don't know if the ecosystem does either. I was under the impression that plugin state was set on init, and never updatable at runtime. Glad to be made aware of that, but I suppose that highlights a problem if I didn't even know this was a blessed pattern.

Stepping back from the current implementation questions, the user need here is for some idiomatic, easy-to-use place to declare some plugin state is persistent and global to the application. This is distinct from, say, serializing a game save or project file, which is not global.

From that point of view, Plugin is a place to hold in memory state tied to a specific plugin, but there is no distinction or affordances to declare that some part of that state is global and (generally-speaking) hot reloadable. Additionally, there is no way to turn all of that global persistent state into a single, serializable structure, e.g. settings.ron or preference.json. I think that's one of the hardest things to accomplish: gathering all global state into a single place, turning it into trait soup, and being able to serde it to a dynamic format is not easy for most users.

To me, the benefit of something like Preferences, is that I can have my in memory representation of all plugin settings, like

pub struct AssetPlugin {
    pub file_path: String,
    pub processed_file_path: String,
    pub watch_for_changes_override: Option<bool>,
    pub mode: AssetMode,
}

and additionally, I can declare my public-facing preferences, which might look completely different, and require extra processing or runtime steps to apply to my application - or even writing back to the preferences file because a setting is invalid:

pub enum AssetPreferences {
    Default,
    TurboMode,
    Custom(/* ... /*)
}

If I squint, it also seems trivial to add callbacks, so I could update my plugin settings when my preferences entry changes, or vice versa.

Can you read through the rationale in that PR for moving away and give your thoughts?

I don't think this conflicts with that, but it is at least clear that the distinction needs to be made much more obvious to developers.

• "I have state specific to a plugin": great, add it as part of your plugin struct
• "I want some of that state to persist in a global preferences file": define that, and add it to the Preferences.

If this approach is acceptable, we might want to integrate this more closely with plugins, so that adding a preferences entry requires implementing a trait that allows conversion to and from the preference and plugin types. Something vaguely along the lines of:

pub trait PluginPreferences {
    type Plugin: Plugin;
    fn from_plugin_state(plugin: &Self::Plugin) -> Self;
    fn into_plugin_state(&self) -> Self::Plugin;
}

This also aids discoverability, and guides users into these blessed patterns. If they want to add preferences for their plugin, they will immediately see that preferences tie directly to the plugin state, which maybe they (like me) didn't realize was the canonical place to store runtime plugin state.

[NthTensor]

What does Preferences enable that the current plugin settings approach could not support / evolve into?

Plugin settings, as they are used currently, allow the developer to define the specific behavior they want. Stuff like "should the window exit when requested" or "how should we initialize wgpu".

These are distinct from user preferences. We don't want users tweaking them. We shouldn't mix developer settings and user-configurable preferences in the same type, that would be crazy!

Since we were just using the render plugin as an example, let's stick with it. The render plugin absolutely needs to expose preferences for visual fidelity if we want to target low end hardware. San Miguel runs at 10fps on my laptop, and I currently have no way to specify a different performance/fidelity trade off.

Implementing those options is a different problem, but it starts with having a place to put them that can be exposed to the user in a preferences screen or a file, and which isn't contaminated with settings not intended for user consumption. That's what we are missing currently.

[viridia]

Let me offer an alternative proposal for a preferences API:

• Each preference item is a Resource.
• There is a resource named Preferences which looks like:

  struct Preferences(HashSet<EntityId>);

• Plugins can register one or more preference resources using:

  app.add_preference_item::<MyPreferences>()

  The call to add_preference_item automatically inserts the Preferences resource into the world if it doesn't exist. There is also an API to remove/unregister preference items.
• Serialization: iterate through all the registered preference items in Preferences, access the resource (if it exists) and use reflection to serialize the item.
• Deserialization: As you iterate through the serde map, lookup each named type, get the resource, and verify that the resource id is registered in Preferences (otherwise you could have the preferences deserialize arbitrary reflected types, which we probably don't want).

[NthTensor]

This proving so controversial underscores the importance of providing users a workable default solution.

If we do not ship something like this ecosystem authors will be forced to write their own, and bevy apps will be burdened with several config sets scattered about across different files, resources, and components.

I encourage everyone to consider what they can and can't compromise on. An imperfect solution is better than none.

[aevyrie]

Let me offer an alternative proposal for a preferences API:

• Each preference item is a Resource.

Something to note that I wasn't aware of until cart reviewed this - bevy wants to move away from plugin settings in resources, and instead use the state inside the struct that implements Plugin.

I do like that suggestion though, I've been thinking about something nearly identical based on the response to this PR. The only open question for me is whether having resources is desired, or if it needs to be more closely tied to the Plugin structs. Not sure exactly what that would look like, but maybe something where you have a trait PluginPreferences: Plugin, that allows you to read/write to that plugin struct. Very similar to your proposal other than the data living in the existing Plugin structs instead of resources. Edit: that might also help with discoverability of using plugin state as runtime settings?

Edit Edit: one downside of this approach is it would require exclusive world access, the design in this PR does not.
Edit Edit Edit: I think I see a path to make this not require exclusive world access, but would require adding a bunch of systems to the schedule.

[inodentry]

@cart Re: preferences in Plugin

Are you sure that works reliably in all cases?

Yes, if the preferences stored inside of Plugin structs are going to be used for initializing things in Startup system, etc., then of course it works fine.

But I have often used my plugin structs to configure what things are added to the app in Plugin::build(). Like to specify what state or schedule systems should be added to.

Isn't Plugin::build() called as soon as the user does app.add_plugins()? I couldn't figure that out conclusively from Bevy's docs and source code.

If so, then changing the preferences by accessing the plugin struct from the app, after it has been added to the app, like you have suggested in one of your previous comments in this PR, would be ineffective. The systems will have already been added and configured.

If bevy fully wants to embrace "preferences in plugin structs" as a design pattern, then this needs to be addressed. I want to be able to configure plugins based on my stored preferences.

And if you suggest that storing/loading preferences to disk / persistent storage should be done via the Asset system, then that further complicates things. The Asset system is not available / up and running until much later in the app lifecycle. It's not feasible to load preference files via the asset system and then use them to configure plugins.

EDIT: here is a specific example of something, admittedly kinda contrived, but that I want to be able to do in my game. I am already doing this with my own bespoke preferences implementation. I want to enable/disable Bevy's pipelined rendering based on user preference. Like maybe my game's graphics settings menu UI has an option for players to choose if they want the "engine to be optimized for framerate or input latency" (that could be a nice user-friendly way to communicate to players what it does). Then, after restarting the game, when initializing the app, i want to disable the pipelined rendering plugin from DefaultPlugins if the option was set to "latency mode".

[viridia]

@aevyrie @cart @inodentry Let me sketch out a possible synthesis of these approaches: preferences as resources or plugin state.

My assumption is that preferences are ordinarily something that is read once, when the app starts, and then potentially written multiple times. Exactly when a write should occur is something we'll need to determine. We don't want individual plugins to trigger writes directly, because multiple plugins might trigger at the same time, but we do want some way for a plugin to give the app a hint that its preferences are dirty and need to be flushed.

Another assumption is that the reading of preferences can happen after app initialization, possibly immediately after. That is, the app initialization happens in a vanilla state - as if there were no preferences file - and then preferences are read and applied to that state. This solves the problem of preferences being stored in locations that might not exist until after app initialization. In fact, the reading of preferences is something that could potentially happen in a setup system. This does mean that there are some scenarios where things are going to get initialized twice, but I suspect that won't be much of a problem in practice, as most preferences aren't a lot of data.

Having preference-loading functionality be tied to plugins does have one advantage, which is that the number of plugins generally isn't that large (compared with entities, resources or assets), so it's feasible to simply iterate over all plugins, ignoring the ones that have no preference-related functionality. This avoids the need to have some kind of preferences type registry.

Given all that, what immediately comes to mind is very simple: a pair of methods on the Plugin trait, one for reading preferences and one for writing. When preferences are loaded, we simply iterate over all plugins, calling the reader method, which in most cases will do nothing. The argument to the reader is an asset-store-like object which the plugin method can use to read whatever data it wants. The plugin method can read the data, store it within itself, or (assuming we pass in some context) store it in the World using either exclusive access or Commands. It's up to the plugin to decide.

Writing is triggered by a command which, similarly, iterates through all plugins, passing in another asset-store-like object, along with context allowing the plugin access to the world. Again, the plugin is free to write whatever data it wants to the store API. The API abstracts over the details of serialization as in the previous prototype PR.

That's the simple version, there are a couple of bells and whistles that could be added.

• Because some loading of preferences may be asynchronous, we'll need an API similar to an asset handle which gives us a "loaded" state that we can use to decide when setup is complete.
• Similarly, writing preferences may also be asynchronous. To ensure consistency, we'd want a way to synchronously "snapshot" the current state of preferences and then save the snapshots concurrently from the actual game execution. This could be done fairly easily by having the "write_prefs" method return a writer object or possibly a command-like object which represents the state of preferences at the time that the write was triggered. Also, to avoid corrupting preferences in the case of a crash in mid-write, the framework could ensure that prefs were written to a temporary location and then moved to the canonical location in the filesystem once all i/o was complete.
• Currently "preferences" is a singleton in the sense that there's one configuration that has special status. We could generalize this to have multiple "namespaces" of preferences: for example, "screen grabs" might be stored in a separate directory or database from "user settings".
• Another variation is instead of having methods on the plugin, you could have the plugin register read/write preferences trait objects. This requires some kind of registry, but may be more flexible.

[cart]

Ok I'm caught up. Before coming up with solutions we need to agree on "minimal baseline requirements". Here is a list I think we should start with (based on the arguments made above, which have changed my mind on some things). Also sadly I think we are absolutely in RFC territory now. We are building a central Bevy feature that will meaningfully define peoples' everyday experience with the engine (and the editor). This isn't the kind of thing we just build based on a subset of the requirements and then evolve into the "right thing". This needs to be a targeted / holistic / thoughtful design process. I think the rest of this PR should be dedicated to enumerating and discussing requirements (not solutions) and then an interested party (whenever they are ready) should draft an RFC (@aevyrie by default, but if they aren't interested / don't have time, someone else should claim this). I think an RFC is necessary because we're all now writing very large bodies of text with requirements/solutions interleaved, we're often restating requirements in different ways, etc. We need a centralized source of truth that we can easily iterate on (and agree upon).

If that feels like "too much work / too much process" for what people need right now, I think thats totally fine. This is the kind of thing that people can build their own bespoke solutions for while they wait. I do think we might be able to agree on a functionality subset (ex: the in memory data model) more quickly than the "full" design. I know/acknowledge that this was the motivation behind the current PR, which was intentionally very minimal. I think we need a slightly better / agreed upon view of the whole picture first, then we can try to scope out minimal subsets.

Requirements

1. Preferences must be decoupled from Bevy's current "Plugin fields / settings" approach, as Plugin fields are sometimes used for "runtime only" configuration. Likewise, there may be multiple instances of the same plugin type (or different plugin types) that both want to consume the same preferences. This also plays into the desired use case of "settings files / editors" where managing "different settings for multiple instances of the same plugin type" would be untenably complicated.
2. Preferences must be available when a plugin initializes.
   • This cannot/should not be initialized with defaults and then later fixed up when preference file(s) are loaded. Some "app init" things cannot / should not be "fixed up" later (as a "normal init flow"), such as GPU device configuration and window configuration. This would also force all preferences to be hot-reloadable at runtime (aka manually wired up for hot reload, which while nice, is not good UX for preference definers and not feasible in some cases).
3. Preferences must be configurable by "upstream" consumers prior to their use in a given Plugin. Ex: a top-level user GamePlugin should be able to set the default values for WindowPreferences::default_window or GpuSettings::wgpu_features.
4. Preferences must be singleton types. Ex: there is one final GpuSettings instance.
5. Preferences must be able to come from zero to many arbitrary locations (and these locations must configurable per platform). These must be applied in a specific order and they should "patch" each other in that order.
6. A given Preferences file may contain many different preference types (ex: GpuSettings and MySettings)
7. Preferences must be enumerable and Reflect-able, in the interest of building things like a "centralized visual preferences editor" in the Bevy Editor
8. If a Preference is changed (in code or from a file change), interested plugins should be able to track that change and react.
9. Preferences should be easily configurable from code, as they will likely become the preferred way to configure most plugins (especially once editor tooling makes it easy / convenient to configure things there)

A Potential "Preferences Dataflow"

In general these requirements point to a deferred plugin init model (which is something we've been discussing for awhile). We're already kind-of there in that we have plugin.finish(). However it seems like we need something akin to plugin.start() in terms of "plugin lifecycles" (although I'm not sure we should actually have that method).

1. Implement "deferred plugin init". This make plugins specify dependencies ahead of time, allowing us to build the graph of plugin dependencies (and some of their metadata) prior to calling Plugin::build.
2. Plugins would then be able to register what Preference types they need access to ahead of time (ex: a new Plugin::init_preferences(preferences: &mut DefaultPreferences). This allows us to build a full (deserializable) picture prior to building plugins. Plugins could initialize the default values for preferences here (and write on top of preference value defaults defined by plugins "before" them in the dependency tree).
3. The "Preferences loader code" (maybe built on the Asset System, maybe not, depending on how Asset System init ties into deferred plugin init), uses the types registered in DefaultPreferences to deserialize the preference files/sources as patches. Each file's patches are stored in their own "layer" (aka: not yet projected onto a single combined location). This ensures that if a preference file changes, we can hot-reload it and still re-generate the correct "final" output (honoring the file/source application order).
4. The final flattened preference values are computed and inserted into the in-memory preference store (ex: an ECS resource).
5. Plugins are initialized (in their dependency order). They can freely feed off of preferences in the preference store.
6. If a change to a preference file is made, it will be hot-reloaded (into the layered patch data structure). The patches will then be re-projected onto the flattened preference store in the correct order (ideally in such a way that only notifies interested plugins of changes when the final result actually changes).

Some Open Questions

• Using the asset system makes the "preference loading" situation cleaner / reduces redundancies (the asset system was built to cover scenarios like this, a Preferences AssetSource nicely encapsulates cross-platform config code, built-in hot-reloading, support for different asset loaders for different config types, etc). Can we make it work for this scenario? What would need to change?
• Editing preferences visually in the Bevy Editor would probably benefit from additional configuration: Friendly names for preferences categories (Ex: "GPU Settings" vs "GpuSettings") and "hierarchical" categories (ex: Rendering/GpuSettings + Rendering/ImageSettings).
• How do app developers override preferences in code, prior to them being used by Plugins during plugin init?
• We may want to make it possible to dump the current runtime state of preferences to a given file, but should we use that as our normal "preferences editing model"? Doing so would (naively) mix manual user-defined preference values (ex encoded in files) with whatever code-overrides currently exist (ex: an App hard-coding WindowPreferences::fullscreen in code shouldn't necessarily persist that value to user configuration).
• For preferences that are hot-reloadable, should we build into the system the ability to detect what fields have changed on those preferences? What would this look like?
• Preferences are singletons and require change detection. Resources are singletons, have change detection, and have maximally simple ECS access APIs. Given that, unless there is something specific about Resources that makes them a bad fit for preferences, I think they should be the in-memory data model.
• It would be nice if preferences could support arbitrary file formats, but we should probably have one "blessed" format. Given that preferences are "patched" across files, this does complicate the serialization story. Adding support for additional formats could require additional significant manual effort.
• Should Preferences should be scope-able to specific locations (ex: ~/.config/bevy_settings.ron vs my_game_settings.ron)? App developers may want to restrict some settings files to only be able to configure specific settings types (ex: don't allow ~/.config/cool_game/settings.ron to override internal bevy settings)

[aevyrie]

Also sadly I think we are absolutely in RFC territory now.

No worries, I'm glad we started this discussion. :)

I don't have the time to do this justice, someone else will need to pick up the mantle!

[viridia]

Always happen to help out with word-smithing, but I generally don't like to volunteer to write a complete doc until I know roughly the shape of what's going to be in it.

I've been looking a bit at the preferences APIs for Qt, .Net, Java and so on. For example, this section on fallback in Qt.

Change notification: can we make the simplifying assumption that notifications don't start up until the game main loop starts? That is, you can't get notified of changes to preferences before the world starts counting ticks.

I assume asynchronous loading will be required for hot loading. However, for pre-initialization deserialization, I'm assuming we would want to block initialization until the last read request is completed.

It sounds like you might want some kind of static registration for preference "types", similar to the Reflect annotation.

Using the asset system might be possible if we can "bootstrap" parts of it - adding just enough asset loaders and asset sources to be able to deserialize the preferences. We'll need a way to wait for all that i/o to be completed, but we can't use the normal event model (AssetDependenciesLoaded etc.) for this since the main game loop isn't running yet. This may mean adding some kind of blocking awaiter ability for groups of assets.

We need to come up with unambiguous names for all the moving parts:

• A single preferences configuration file.
• An entry within a preferences configuration set.
• A root location for user/system preferences for a given app.

[cart]

Change notification: can we make the simplifying assumption that notifications don't start up until the game main loop starts? That is, you can't get notified of changes to preferences before the world starts counting ticks.

Yeah I think this is totally reasonable.

I assume asynchronous loading will be required for hot loading. However, for pre-initialization deserialization, I'm assuming we would want to block initialization until the last read request is completed.

Yeah I think you're right, although this could be done without actually doing each thing synchronously by using async/parallel APIs and joining on a bunch of load tasks.

Given that preferences of a given type can come from arbitrary files, I don't think theres a good way to eagerly initialize as we load preference files. We need to wait until everything is parsed and loaded.

It sounds like you might want some kind of static registration for preference "types", similar to the Reflect annotation.

When it comes to the "Preference enumeration" problem, I think we can use Reflect directly for this:

#[derive(Resource, Reflect, Preferences)]
#[reflect(Preferences)]
struct MyPeferences;

app.register_type::<MyPreferences>();

fn iter_all_preferences(world: &mut World, type_registry: &TypeRegistry) {
    for (registration, reflect_preferences) in type_registry.iter_with_data::<ReflectPreferences>() {
        // this could be &dyn Reflect, &dyn Preferences, or we could have methods for both
        if let Some(preferences) = reflect_preferences.get(world) {
            println!("{:?}",);
        }
    }
}

I've been pushing for this pattern instead of bespoke centralized stores for Dev Tools as well. Kind of vindicating to see it come up again :)

[cart]

Using the asset system might be possible if we can "bootstrap" parts of it - adding just enough asset loaders and asset sources to be able to deserialize the preferences. We'll need a way to wait for all that i/o to be completed, but we can't use the normal event model (AssetDependenciesLoaded etc.) for this since the main game loop isn't running yet. This may mean adding some kind of blocking awaiter ability for groups of assets.

Yeah I think this can be done without significant changes. We just need to expose an async let preferences: Preferences = asset_server.load_direct("config://my_preferences.ron").await; API that skips insertion into the asset system and directly reads + loads the asset into memory. In parallel to that, if hot reloading is enabled, we could kick off another "normal" load to make the asset system listen for changes.

[cart]

Note that asset_server.load_direct("config://my_preferences.ron").await; essentially already exists, but is not publicly exposed on AssetServer

[viridia]

OK that all sounds good.

I have been looking at the docs for various preference APIs - Java, .Net, Qt. Most of them are fairly opinionated in ways that we might be able to emulate.

Let's take the Qt preferences API as an example. There are four preference namespaces, in ascending order of priority:

• system/organization-id
• system/product-id
• user/organization-id
• user/product-id

The "organization" preferences apply as a fallback to all products produced by that organization. I don't know whether or not we would need this - even a big studio such as Bethesda doesn't share any preferences between, say, Skyrim and Fallout.

However, requiring an organization id - or as in the case of Java, a unique domain name - as part of the preference path would guard against name collisions between similarly-named games. In the case of browser local storage, collisions are impossible since the storage is per-domain, so the organization and game name could be ignored in that case.

This suggests an opinionated wrapper around AssetSource which forces the developer to supply unique key arguments - organization and title - which would be ignored on some platforms, but which would otherwise be used in constructing the root path to the preferences store. Something like PreferencesAssetSource::new(org, title). This would in effect point to a directory, and could be referenced like any other asset source using a prefix such as "preferences://". Individual preferences properties files would be named as desired by the app developer, and would be placed in the asset path just like any other asset, so for example "preferences://config.ron" or "preferences://characters.ron".

For user vs system preferences, on read we could have the asset source transparently handle merging / combining the assets with the fallbacks. However, this doesn't solve the problem of writing, because here the app has to make an explicit choice. One idea would be have the preferences asset source register multiple additional prefixes, one per namespace: "preferences.user://" and "preferences.system://", allowing the app to decide which namespace to write to by choosing the corresponding prefix.

[cart]

I think my take is that we should have a generically usable "config asset source" (config://) with OS-specific roots:

• Linux: ~/.config/APP_NAME
• Windows: ~\AppData\Roaming\APP_NAME
• MacOS: ~/Library/Preferences/APP_NAME

Every OS has a pattern for these types of files. It makes sense to provide a reasonably unopinionated AssetSource for this.

Then for Preferences specifically, we can come up with naming conventions on top of that. Ex:

• config://bevy_settings.ron: built-in bevy plugin settings
  • would expand to something like ~/.config/MyApp/bevy_settings.ron
• config://settings.ron: developer-defined app-specific settings
• /settings.ron: built-in settings defaults (note this doesn't specify an Asset Source, indicating that this is the default /assets source)

The Preferences config could define layers:

PreferencesConfig::new()
  .add_layer("/settings.ron") // apply the `/assets` settings first
  .add_layer("/bevy_settings.ron") // apply the `/assets` settings first
  .add_layer("config://settings.ron") // apply "config" settings last, allowing them to override the `/assets` settings
  .add_layer("config://bevy_settings.ron")

When saving, we always write to a specific location. By default the config:// locations should be preferred.

We could consider extending this API with "scopes" to restrict what settings are allowed in each file:

#[derive(Reflect, Resource, Preferences)]
#[reflect(Preferences)]
#[preferences_scope(GamePreferences)]
struct MyPreferences;

PreferencesConfig::new()
  .add_layer::<GamePreferences>("/settings.ron")
  .add_layer::<BevyPreferences>("/bevy_settings.ron")

[cart]

The resulting system is pretty flexible. We could provide reasonable defaults (like the ones listed above), but it would allow apps that need additional (arbitrary) sources to add them as needed (ex: maybe they want to load settings from the network).

[viridia]

I've been avoiding the word "settings" because it's overloaded - there are plugin settings, asset loader settings and so on.

I'm a bit concerned about collisions with AppName: Since there's no registry of Bevy games, we can't enforce that games have a unique name, which would lead to preferences getting overwritten. This is why Java uses the convention of having the domain name be part of the id: the domain name system gives us a global registry of unique names that we don't have to pay for or maintain.

I suppose the problem could be solved by convention rather than by a mechanism (tell people to include the name of their company in AppName) but we'd have to be careful to ensure that the composite is a valid directory name, so for example viridia:panoply would not be acceptable because of the colon.

This is more of a problem for indie developers, I think, than big companies, since there are a lot more of the former.

[bushrat011899]

I think my take is that we should have a generically usable "config asset source" (config://) with OS-specific roots:

* **Linux**: `~/.config/APP_NAME`

* **Windows**: `~\AppData\Roaming\APP_NAME`

* **MacOS**: `~/Library/Preferences/APP_NAME`

Every OS has a pattern for these types of files. It makes sense to provide a reasonably unopinionated AssetSource for this.

A bit tangential, but this would be an awfully similar arrangement to offering temporary files as an asset source (temp:://). Perhaps a proof of concept for the asset source component of the preferences API could be experimented with there?

[cart]

I've been avoiding the word "settings" because it's overloaded

Yeah I think "preferences" is a good name.

I'm a bit concerned about collisions with AppName

Makes sense. I see no reason not to make this configurable. We could have an app_name: String and an org_name: Option<String>. If org_name is specified, we could do something like ORG_NAME.APP_NAME. Developers that want the simplicity of ~/.config/AppName can have it, and developers that want the collision protection of ~/.config/OrgName.AppName can have it.

A bit tangential, but this would be an awfully similar arrangement to offering temporary files as an asset source (temp:://). Perhaps a proof of concept for the asset source component of the preferences API could be experimented with there?

Yeah a bit tangential. I don't think the Asset Sources feature needs proving out as its already in use, but I do think a temp:// asset source would be useful. Worth implementing!

[viridia]

It seems like we are moving towards convergence, but this conversation is not very discoverable.

I've written up a draft summary of what's been proposed so far on HackMD: https://hackmd.io/@dreamertalin/rkhljFM7R - feel free to add comments and suggestions (you should have permission, let me know if it doesn't work). Also, I can grant edit rights if you want to dive in and start re-writing sections.

This is somewhat of an experiment: I figured it would be faster than creating an RFC in git. I have no objection to copying what I have written into an RFC or Bevy discussion, but I thought I'd wait until things are a bit more fleshed out before doing so.

[viridia]

I'm still a bit confused about the requirement for hot reloading. What's the use case?

At the risk of repeating myself, my assumption is that preferences are unlike other kinds of assets in that they are "authored" by the application itself. Hot reloading makes the most sense when the configuration file is written by a different process - such as exporting a .glb file from Blender. While it's certainly true that a game editor will output various kinds of property-like assets, these kinds of configuration files are not "user preferences" in the sense of being something that is produced through interaction from the end user.

Trying to come up with a hypothetical use case, I suppose you could have a "launcher" app which is separate from the game, whereby you can adjust the game settings through the launcher. However, I don't know of any game which is written this way: while I have seen launchers that have user settings (such as "prod" vs "beta" instances), those settings typically either can't be changed while the game is running, or if they can be changed, they can also be adjusted within the game itself, avoiding the need to switch to another window during play.

The set of serializable entities for a game typically falls into one of the following categories:

• assets, which are produced by the game developer
• mods, which are produced by modders
• saved games, which are generated by the player, but which are only read and written at specific points during play
• screen captures, streams, logs, and other artifacts generated by the player as a side-effect of play
• explicit game options, such as music volume, graphics mode, key bindings, closed captioning and so on
• implicit game options, such as window size/location, "don't show this dialog again" flags, recent colors or files, whether or not the user has seen the tutorial, and so on

Of these, the only ones that I think make sense for hot reloading are the first two categories.

I kind of feel like the desire to have hot reloading is an overgeneralization: "User preferences are config files. Config files are generic and can be used to store all kinds of data. Some of those kinds of data require hot reloading. Therefore all config files should support hot reloading." - my response would be "beware of too much generalization".

[aevyrie]

Vscode settings (preferences), are the most immediate thing that come to mind. Various plugins work together to report they have some persistent configuration. This is merged into single file that is editable by the user or via the settings UI.

[NthTensor]

To me, the use-case for hot reloading is generality and simplicity. Bevy can be used for headless rendering/simulation, or for servers, in other contexts where an in-app preferences UI would be bothersome/impossible. It seems like an important feature.

[tychedelia]

Bevy can be used for headless rendering/simulation, or for servers, in other contexts where people an in-app preferences UI would be bothersome/impossible.

An art installation is a good example of this. Typically creative studios will provide tech support for these in more corporate settings, but telling the client "open textpad.exe and change these lines" when something comes up is potentially a huge benefit over existing platforms.

[viridia]

My concern is that we're making the design more complex in pursuit of some really outlying edge cases.

@aevyrie VSCode is a very unusual program in this respect, in that it's a text editor that can edit its own settings file either via a preferences UI or as raw text. Not many applications do this.

@NthTensor I don't see how that argument is relevant. If the headless app writes out a config file that is the product of user interaction (whether that interaction be via network connection or API) then it is still being written out by the same process, and shouldn't need notification; conversely, if the config is something that is specified externally by some other program, then it's not a user preference and should use a different mechanism. (It also shouldn't store these configurations in ~/.config or ~/AppData/Roaming or wherever user preferences typically go).

@tychedelia Same argument here. An art installation wouldn't have "user preferences", in the sense of remembering the choices of a specific user. It might have a configuration file, which you might want to modify and hot reload, but that is not a user preference, and shouldn't live in the user's home dir. Even in the rare case where an installation maintained some kind of record of interactions, that would probably be stored in a database in a specific directory.

In the case you describe, it's a bit ambiguous who the "user" is: is it the patrons who visit the museum, or the curator? The patrons "interact" with the installation, but there are thousands of them, and we don't maintain home dirs for each one or even remember the choices that they may (except in some aggregate sense). If it's the curator, they don't generally "interact" with the exhibit other than setting it up and configuring it - but again, the curator's home dir is probably not the best place for such configuration.

Look, if you all want to do the extra work of implementing hot reloading for preferences I'm not going to stop you; but I'm trying to minimize the requirements as much as I can.

[NthTensor]

if the config is something that is specified externally by some other program, then it's not a user preference [...] It also shouldn't store these configurations in ~/.config.

I edit preferences files in ~/.config with my text editor almost every day. What are we even talking about here?

In the case you describe, it's a bit ambiguous who the "user" is [...]

I think we can pretty safely say that a "user" is someone with relevant permissions who wants to configure the behavior of the app (a) at some point after compile time (b) using a set of preference options specifically exposed by the engine and app developers for that purpose.

is it the patrons who visit the museum, or the curator? The patrons "interact" with the installation, but there are thousands of them, and we don't maintain home dirs for each one or even remember the choices that they may (except in some aggregate sense).

With all respect, because I really appreciate your contributions here and elsewhere, I feel like discussing having a preferences file for each visitor of a museum is a bit of a straw-man.

Look, if you all want to do the extra work of implementing hot reloading for preferences I'm not going to stop you; but I'm trying to minimize the requirements as much as I can.

You're worried about scope-creep. That's reasonable. I'm worried about a monolithic solution that is too specifically tailored for the designer's (eg, you) personal use and so fails to prevent ecosystem fragmentation. Surely, we can both be satisfied by decomposeability and modularity.

My thesis is: We must trust the app developer to know where and how their users would like to specify their preferences (a term which, to me, is synonymous with "runtime config"). Hot reloading should be an option for them: natively, as an ecosystem crate, or as something they can write for themselves. At the same time, plugin and engine developers should be able to add and consume preferences without needing to worry about the app developer's choices.

I would not block a preferences api for lacking hot-reloading, but I would block for preventing hot-reloading.

[viridia]

I edit preferences files in ~/.config with my text editor almost every day.

Really? The last time I even looked at my .config was about 5 years ago. The only time I ever manipulated files in .config was back before browsers were smart enough to import bookmarks from other browsers.

Anyway, it looks like I'm in the minority here. I still honestly don't get it, but I'll desist.

Also, "hot reloading" is technically not the right term here, because that means loading newly compiled code. I think the right term in this case is "file watching".

Anyway, I added a section in the doc about this.

So what's the next step? It looks like my HackMD experiment is a bust, no one has commented.

[cart]

Also, "hot reloading" is technically not the right term here, because that means loading newly compiled code. I think the right term in this case is "file watching".

In my experience, "hot reloading" is a generic term for changing the state of an app while it is still running (listen for change, apply new state to running app). If seen this in the javascript world and in other engines (ex: Defold)

So what's the next step? It looks like my HackMD experiment is a bust, no one has commented.

Sorry I did a pass when you posted, largely agreed with it, and moved on. Its probably worth posting a link to #engine-dev (with some context / a link to this PR) to to get some more eyes.