Updating plugin config from within the plugin itself #1305

zaughon · 2019-03-06T10:50:51Z

In order to let us help you better, please fill out the following fields as best you can:

I am...

I am running...

Errbot version: 5.1.3
OS version: Ubuntu 18.04
Python version: 3.6
Using a virtual environment: yes

Issue description

I'm trying to figure out whether there's a way to trigger a plugin config update in a different way than the usual !plugin config NAME CONFIG.

I've tried creating this plugin:

from errbot import BotPlugin, botcmd
from itertools import chain


class Example(BotPlugin):
    """
    This is a very basic plugin to try out your new installation and get you started.
    Feel free to tweak me to experiment with Errbot.
    You can find me in your init directory in the subdirectory plugins.
    """

    CONFIG_TEMPLATE = {'ID_TOKEN': '00112233445566778899aabbccddeeff',
                       'USERNAME': 'changeme'}


    def check_configuration(self, configuration):
        pass


    def get_configuration_template(self):
        return self.CONFIG_TEMPLATE

    def configure(self, configuration):
        if configuration is not None and configuration != {}:
            config = dict(chain(self.CONFIG_TEMPLATE.items(),
                                configuration.items()))
        else:
            config = self.CONFIG_TEMPLATE
        print(config)
        super(Example, self).configure(config)


    @botcmd()
    def getconf(self, msg, args):
        print(self.config)

    @botcmd()
    def setconf(self, msg, args):
        print(self.config['USERNAME'])
        # self.config['USERNAME'] = args
        self.configure({'USERNAME': args})
        print(self.config['USERNAME'])

But it doesn't seem to work properly. When you run the setconf command, it shows that it has been updated afterwards, but if you then run !plugin config Example to get the currently config, it's still the old configuration.

[@CHANGE_ME ➡ @errbot] >>> !plugin config Example
[@CHANGE_ME ➡ @errbot] [␍] 
Default configuration for this plugin (you can copy and paste this directly as a command):
 !plugin config Example 
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

 Current configuration:
 !plugin config Example 
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}


[@CHANGE_ME ➡ @errbot] >>> !setconf test
[@CHANGE_ME ➡ @errbot] [␍] 
changeme
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'test'}
test

[@CHANGE_ME ➡ @errbot] >>> !plugin config Example
[@CHANGE_ME ➡ @errbot] [␍] 
Default configuration for this plugin (you can copy and paste this directly as a command):
 !plugin config Example 
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

 Current configuration:
 !plugin config Example 
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

Steps to reproduce

Create the above plugin
Run a !plugin config Example
Run the !setconf command
Run a !plugin config Example

During the !setconf command, it will:

print the self.config['USERNAME']
run self.configure which prints the new config
then !setconf prints self.config['USERNAME'] again where it's printing the updated value, but running !plugin config Example still shows the old config.

Is there any way to update the configuration from a different function? Did i do something wrong?
My plan is to create a poller that checks for updates for a few CMS-systems, and then automatically updates the config.

Additional info

If you have any more information, please specify it here.

The text was updated successfully, but these errors were encountered:

b1rger · 2019-03-13T14:09:07Z

Hi,
i have the same problem- i based my code on the instructions on supplying a partial configuration:

    @botcmd
    def foo_addchannel(self, message, args):
        """ add a channel to the list of channels for this plugin """
        config = self.config
        config['CHANNELLIST'][args] = [1, 1]
        self.configure(config)

    def configure(self, configuration):
       if configuration is not None and configuration != {}:
          config = dict(chain(self.CONFIG_TEMPLATE.items(), configuration.items()))
       else:
          config = self.CONFIG_TEMPLATE
       self.log.info(config)
       super().configure(config)

in the log info i can see, that the config object contains the added value, but when i run !plugin config foo the new value is not listed.

If i add a command:

    @botcmd
    def foo_getconfig(self, message, args):
             self.send(message.frm, "%s" %(self.config))

The new value is listed as part of the config. So !plugin config foo seems to access another object than the plugin itself and the config object of the plugin itself does not persist.

sheluchin · 2019-03-18T15:06:24Z

I think this is related to #910. What kind of storage are you guys using?

b1rger · 2019-03-18T16:09:24Z

I think this is related to #910. What kind of storage are you guys using?

I'm using the default, which is STORAGE = 'Shelf'

sheluchin · 2019-03-18T17:09:07Z

I'm using the default, which is STORAGE = 'Shelf'

@bschacht Could you please check what $ errbot --storage-get <plugin-name> returns under the same test context?

b1rger · 2019-03-18T21:48:17Z

@bschacht Could you please check what $ errbot --storage-get <plugin-name> returns under the same test context?

hm, it returns {'URLLIST': {}}; URLLIST was a config setting i used during development, but switched then to CHANNELLIST. Should i have restarted the bot after changing the Plugins CONFIG_TEMPLATE? I'm pretty sure i deleted the Foo.db.db file it created...

zaughon · 2019-03-19T16:11:20Z

@sheluchin All i get when i try is:
{'domains': {}}
which isn't part of the configuration. It's a key the persistent storage:
http://errbot.io/en/latest/user_guide/plugin_development/persistence.html

b1rger · 2019-03-19T16:24:14Z

hm, it returns {'URLLIST': {}}

Ah, now that @zaughon mentioned the persistent storage, i must admit that i was mistaken- the URLLIST in my case is indeed a persistent storage setting! i mixed that up...

sheluchin · 2019-03-19T17:27:35Z

What about running errbot --storage-get core?

What I suspect is that somewhere along the way, the configuration dict is not being updated correctly, such that the inherited StoreMixin methods are not being called:

errbot/errbot/storage/__init__.py

Lines 43 to 61 in 77231b2

 # those are the minimal things to behave like a dictionary with the UserDict.DictMixin 

 def __getitem__(self, key): 

 return self._store.get(key) 

 @contextmanager 

 def mutable(self, key): 

 obj = self._store.get(key) 

 yield obj 

 # implements autosave for a plugin persistent entry 

 # with self['foo'] as f: 

 # f[4] = 2 

 # saves the entry ! 

 self._store.set(key, obj) 

 def __setitem__(self, key, item): 

 return self._store.set(key, item) 

 def __delitem__(self, key): 

 return self._store.remove(key)

Using the mutable context manager when updating the config dictionary might be helpful, there's even a note about that on master:

errbot/errbot/plugin_manager.py

Lines 343 to 347 in adc13ef

 def set_plugin_configuration(self, name, obj): 

 # TODO: port to with statement 

 configs = self[CONFIGS] 

 configs[name] = obj 

 self[CONFIGS] = configs

For example, doing config_dict.update(...) does not trigger the __setitem__ method on StoreMixin, so it never gets entered into persistent storage. There's a brief warning of this in http://errbot.io/en/latest/user_guide/plugin_development/persistence.html#caveats.

It's worth pointing out that configuration is not stored in the plugin's persistent storage, but rather in the core's storage, in a key called configs, like:

key     | value
--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------
configs | {"Test": {"foo": "bazz"}, "Webserver": {"HOST": "0.0.0.0", "PORT": 3141, "SSL": {"certificate": "", "enabled": false, "host": "0.0.0.0", "key": "", "port": 3142}}}

If you guys are just trying to provision your plugins without having to manually do it through chat, there are instructions on how to do that http://errbot.io/en/latest/user_guide/provisioning.html, but from my testing in #910, it doesn't work. I posted #1311 last night, which seems to resolve the issue and does produce working results when using the approach suggested in the link:

$ echo "{'configs': {"Test": {"foo": "bar"}}" | errbot --storage-merge core

That will save a new configuration for the Test plugin, or overwrite the existing one, while leaving the configuration for other plugins intact.

b1rger · 2019-03-20T11:45:02Z

What about running errbot --storage-get core?

{'bl_plugins': ['HelloWorld', 'Flows', 'UrlCheck', 'VersionChecker'], 'configs': {'Foo': {'CHANNELLIST': {'[email protected]', '[email protected]'}}}}

(i had added ch1 and ch1 using the !plugin config Foo command)

What I suspect is that somewhere along the way, the configuration dict is not being updated correctly, such that the inherited StoreMixin methods are not being called:

Yeah, thats my impression too. But i don't know errbot good enough to find a workaround

Using the mutable context manager when updating the config dictionary might be helpful, there's even a note about that on master:

You mean fixing that in the plugin manager or somehow in our modules?

If you guys are just trying to provision your plugins without having to manually do it through chat,

Actually what i'm trying to do is make configuration of a plugin easier- if i have a list of 30 channels in my config, it would be easier to have a channel_add command than always have to paste a python dict with 30 channels into the chat window. I could just use the persistent storage instead of the plugin configuration though...

zaughon · 2019-03-25T11:15:42Z

@sheluchin For the test bot mentioned in the initial description, that gives me:
{'configs': {'Example': {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}}}

If i try to update that using !setconf which calls self.configure as seen in the initial escalation, i get:
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'test'}
as expected.
However, running !plugin config Example afterwards, or errbot --storage-get core after shutting down the bot, both give the old value:
{'configs': {'Example': {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}}}

I'm also under the impression that it's not being updated correctly in the storage, so it doesn't actually persist, but also not sure how exactly to fix that.
The reason i want it to update automatically is that i have a poller running once a day that checks for the most recent versions for e.g. Wordpress. It would be a lot easier if this automatically updated it's configuration based on that, instead of us having to do it manually every time there's new updates. So the http://errbot.io/en/latest/user_guide/provisioning.html solution won't really solve the issue

sheluchin · 2019-03-25T14:07:51Z

@zaughon I'd like to try to test out your example plugin and try to fix it. I'll see if I can get around to it sometime soon - maybe tonight. If you could confirm that the issue persists with Errbot 6.0.0, which was released over the weekend, it would really be helpful. See https://github.com/errbotio/errbot/blob/master/CHANGES.rst.

zaughon · 2019-03-27T16:16:18Z

@sheluchin Seems to be the same issue:

[@CHANGE_ME ➡ @errbot] >>> !getconf
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

[@CHANGE_ME ➡ @errbot] >>> !setconf test
changeme
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'test'}
test

[@CHANGE_ME ➡ @errbot] >>> !getconf
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'test'}

[@CHANGE_ME ➡ @errbot] >>> !plugin config Example
Default configuration for this plugin (you can copy and paste this directly as a command):
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

 Current configuration:
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

[@CHANGE_ME ➡ @errbot] >>> !about
This is Errbot version 6.0.0

[@CHANGE_ME ➡ @errbot] >>> ^d

$ errbot --storage-get core
{'configs': {'Example': {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}}}

It's still not stored to the actual configuration of the plugin, it's just sort of stored in self.config, but not permanently.

This is my Example plugin atm:

from errbot import BotPlugin, botcmd
from itertools import chain


class Example(BotPlugin):
    """
    This is a very basic plugin to try out your new installation and get you started.
    Feel free to tweak me to experiment with Errbot.
    You can find me in your init directory in the subdirectory plugins.
    """

    CONFIG_TEMPLATE = {'ID_TOKEN': '00112233445566778899aabbccddeeff',
                       'USERNAME': 'changeme'}

    def check_configuration(self, configuration):
        pass

    def get_configuration_template(self):
        return self.CONFIG_TEMPLATE

    def configure(self, configuration):
        if configuration is not None and configuration != {}:
            config = dict(chain(self.CONFIG_TEMPLATE.items(),
                                configuration.items()))
        else:
            config = self.CONFIG_TEMPLATE
        print(config)
        super(Example, self).configure(config)


    @botcmd()
    def getconf(self, msg, args):
        print(self.config)

    @botcmd()
    def setconf(self, msg, args):
        print(self.config['USERNAME'])
        # self.config['USERNAME'] = args
        self.configure({'USERNAME': args})
        print(self.config['USERNAME'])

sheluchin · 2019-03-27T17:16:35Z

@zaughon I made some progress, but not quite done yet.

Try this:

@botcmd()                                              
def setconf(self, msg, args):
    self._bot.plugin_manager.set_plugin_configuration( 
        self.name,                                     
        getattr(self, 'config', self.CONFIG_TEMPLATE)  
    )
    self.configure({'USERNAME': args})

zaughon · 2019-03-28T12:11:38Z

That specific code doesn't help, it still gives the old config:

[@CHANGE_ME ➡ @errbot] >>> !getconf
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

[@CHANGE_ME ➡ @errbot] >>> !plugin config Example
Default configuration for this plugin (you can copy and paste this directly as a command):
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

 Current configuration:
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}


[@CHANGE_ME ➡ @errbot] >>> !setconf TEST
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'TEST'}

[@CHANGE_ME ➡ @errbot] >>> !getconf
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'TEST'}

[@CHANGE_ME ➡ @errbot] >>> !plugin config Example
Default configuration for this plugin (you can copy and paste this directly as a command):
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

 Current configuration:
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}


[@CHANGE_ME ➡ @errbot] >>> ^d

$ errbot --storage-get core
{'configs': {'Example': {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}}}

However, if i move self.configure({'USERNAME': args}) up so it's run before self._bot.plugin_manager.set_plugin_configuration(...) - then it seems to work:

[@CHANGE_ME ➡ @errbot] >>> !plugin config Example
Default configuration for this plugin (you can copy and paste this directly as a command):
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

 Current configuration:
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}


[@CHANGE_ME ➡ @errbot] >>> !getconf
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

[@CHANGE_ME ➡ @errbot] >>> !setconf TEST
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'TEST'}

[@CHANGE_ME ➡ @errbot] >>> !getconf
{'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'TEST'}

[@CHANGE_ME ➡ @errbot] >>> !plugin config Example
Default configuration for this plugin (you can copy and paste this directly as a command):
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'changeme'}

 Current configuration:
 !plugin config Example {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'TEST'}


[@CHANGE_ME ➡ @errbot] >>> ^d

$ errbot --storage-get core
{'configs': {'Example': {'ID_TOKEN': '00112233445566778899aabbccddeeff', 'USERNAME': 'TEST'}}}

sijis added the type: support/question label Jun 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating plugin config from within the plugin itself #1305

Updating plugin config from within the plugin itself #1305

zaughon commented Mar 6, 2019

b1rger commented Mar 13, 2019 •

edited

sheluchin commented Mar 18, 2019

b1rger commented Mar 18, 2019

sheluchin commented Mar 18, 2019

b1rger commented Mar 18, 2019

zaughon commented Mar 19, 2019

b1rger commented Mar 19, 2019 •

edited

sheluchin commented Mar 19, 2019

b1rger commented Mar 20, 2019

zaughon commented Mar 25, 2019

sheluchin commented Mar 25, 2019

zaughon commented Mar 27, 2019

sheluchin commented Mar 27, 2019

zaughon commented Mar 28, 2019

Updating plugin config from within the plugin itself #1305

Updating plugin config from within the plugin itself #1305

Comments

zaughon commented Mar 6, 2019

I am...

I am running...

Issue description

Steps to reproduce

Additional info

b1rger commented Mar 13, 2019 • edited

sheluchin commented Mar 18, 2019

b1rger commented Mar 18, 2019

sheluchin commented Mar 18, 2019

b1rger commented Mar 18, 2019

zaughon commented Mar 19, 2019

b1rger commented Mar 19, 2019 • edited

sheluchin commented Mar 19, 2019

b1rger commented Mar 20, 2019

zaughon commented Mar 25, 2019

sheluchin commented Mar 25, 2019

zaughon commented Mar 27, 2019

sheluchin commented Mar 27, 2019

zaughon commented Mar 28, 2019

b1rger commented Mar 13, 2019 •

edited

b1rger commented Mar 19, 2019 •

edited