Helm: import_datasources.yaml - 'TypeError: 'str' object does not support item deletion'

[ps-aux]

Bug description

With Helm installation with values:

    extraConfigs:
      import_datasources.yaml: |
        databases:
        - allow_file_upload: true
          allow_ctas: true
          allow_cvas: true
          database_name: plm
          sqlalchemy_uri: postgres://my-db:...
          tables: [ my_table ]

superset-init-db job fails with TypeError: 'str' object does not support item deletion

Expected behaviour: Either datasource should be created or in case of invalid configuration the proper error should be logged.

How to reproduce the bug

1. Create new Helm installation with the chart version 0.12.9
2. Define extraConfigs values as above
3. No database is created and superset-init-db fails

Screenshots/recordings

Logs attached below.

Superset version

4.0.0

Python version

Not applicable

Node version

I don't know

Browser

Not applicable

Additional context

Helm chart version: 0.12.9

2024-04-27 09:19:10,385:INFO:superset.security.manager:Syncing role definition
Syncing Admin perms
2024-04-27 09:19:10,587:INFO:superset.security.manager:Syncing Admin perms
Syncing Alpha perms
2024-04-27 09:19:10,621:INFO:superset.security.manager:Syncing Alpha perms
Syncing Gamma perms
2024-04-27 09:19:11,939:INFO:superset.security.manager:Syncing Gamma perms
Syncing sql_lab perms
2024-04-27 09:19:13,319:INFO:superset.security.manager:Syncing sql_lab perms
Fetching a set of all perms to lookup which ones are missing
2024-04-27 09:19:14,694:INFO:superset.security.manager:Fetching a set of all perms to lookup which ones are missing
Creating missing datasource permissions.
2024-04-27 09:19:14,717:INFO:superset.security.manager:Creating missing datasource permissions.
Creating missing database permissions.
2024-04-27 09:19:14,727:INFO:superset.security.manager:Creating missing database permissions.
Cleaning faulty perms
2024-04-27 09:19:14,743:INFO:superset.security.manager:Cleaning faulty perms
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
Creating admin user...
logging was configured successfully
2024-04-27 09:19:18,118:INFO:superset.utils.logging_configurator:logging was configured successfully
2024-04-27 09:19:18,123:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/usr/local/lib/python3.10/site-packages/flask_limiter/extension.py:293: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
  warnings.warn(
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
Recognized Database Authentications.
Error! User already exists admin
Importing database connections.... 
logging was configured successfully
2024-04-27 09:19:22,756:INFO:superset.utils.logging_configurator:logging was configured successfully
2024-04-27 09:19:22,761:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/usr/local/lib/python3.10/site-packages/flask_limiter/extension.py:293: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
  warnings.warn(
Importing dataset from file /app/configs/import_datasources.yaml
2024-04-27 09:19:24,122:INFO:superset.commands.dataset.importers.v0:Importing dataset from file /app/configs/import_datasources.yaml
Importing 1 databases
2024-04-27 09:19:24,122:INFO:superset.commands.dataset.importers.v0:Importing 1 databases
Importing new dbs plm
2024-04-27 09:19:24,133:INFO:superset.models.helpers:Importing new dbs plm
Error running import command
Traceback (most recent call last):
  File "/app/superset/commands/dataset/importers/dispatcher.py", line 57, in run
    command.run()
  File "/app/superset/commands/dataset/importers/v0.py", line 250, in run
    import_from_dict(config, sync=self.sync)
  File "/app/superset/commands/dataset/importers/v0.py", line 213, in import_from_dict
    Database.import_from_dict(database, sync=sync)
  File "/app/superset/models/helpers.py", line 343, in import_from_dict
    child_class.import_from_dict(
  File "/app/superset/models/helpers.py", line 273, in import_from_dict
    del dict_rep[k]
TypeError: 'str' object does not support item deletion
2024-04-27 09:19:24,133:ERROR:superset.commands.dataset.importers.dispatcher:Error running import command
Traceback (most recent call last):
  File "/app/superset/commands/dataset/importers/dispatcher.py", line 57, in run
    command.run()
  File "/app/superset/commands/dataset/importers/v0.py", line 250, in run
    import_from_dict(config, sync=self.sync)
  File "/app/superset/commands/dataset/importers/v0.py", line 213, in import_from_dict
    Database.import_from_dict(database, sync=sync)
  File "/app/superset/models/helpers.py", line 343, in import_from_dict
    child_class.import_from_dict(
  File "/app/superset/models/helpers.py", line 273, in import_from_dict
    del dict_rep[k]
TypeError: 'str' object does not support item deletion
There was an error when importing the dataset(s), please check the exception traceback in the log
Traceback (most recent call last):
  File "/app/superset/cli/importexport.py", line 195, in import_datasources
    ImportDatasetsCommand(contents, overwrite=True).run()
  File "/app/superset/commands/dataset/importers/dispatcher.py", line 68, in run
    raise exc
  File "/app/superset/commands/dataset/importers/dispatcher.py", line 57, in run
    command.run()
  File "/app/superset/commands/dataset/importers/v0.py", line 250, in run
    import_from_dict(config, sync=self.sync)
  File "/app/superset/commands/dataset/importers/v0.py", line 213, in import_from_dict
    Database.import_from_dict(database, sync=sync)
  File "/app/superset/models/helpers.py", line 343, in import_from_dict
    child_class.import_from_dict(
  File "/app/superset/models/helpers.py", line 273, in import_from_dict
    del dict_rep[k]
TypeError: 'str' object does not support item deletion
2024-04-27 09:19:24,134:ERROR:superset.cli.importexport:There was an error when importing the dataset(s), please check the exception traceback in the log
Traceback (most recent call last):
  File "/app/superset/cli/importexport.py", line 195, in import_datasources
    ImportDatasetsCommand(contents, overwrite=True).run()
  File "/app/superset/commands/dataset/importers/dispatcher.py", line 68, in run
    raise exc
  File "/app/superset/commands/dataset/importers/dispatcher.py", line 57, in run
    command.run()
  File "/app/superset/commands/dataset/importers/v0.py", line 250, in run
    import_from_dict(config, sync=self.sync)
  File "/app/superset/commands/dataset/importers/v0.py", line 213, in import_from_dict
    Database.import_from_dict(database, sync=sync)
  File "/app/superset/models/helpers.py", line 343, in import_from_dict
    child_class.import_from_dict(
  File "/app/superset/models/helpers.py", line 273, in import_from_dict
    del dict_rep[k]
TypeError: 'str' object does not support item deletion
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dwierenga]

Any update on this? This is blocking our adoption of Superset.

[rusackas]

Added a couple helm-knowing assignees. I don't know the solution here, but I do know a bit about a hosted version of Superset if you need to unblock your org more expediently. ;)

[ps-aux]

@dwierenga I managed to make it work with adding/settings these properties:

extra: "{\r\n    \"metadata_params\": {},\r\n    \"engine_params\": {},\r\n    \"\ metadata_cache_timeout\": {},\r\n    \"schemas_allowed_for_file_upload\": []\r\n\ }"
tables: []

[dwierenga]

@ps-aux : copying that exact property string works. However, it's entirely un-obvious as to:

• how to properly include a list of tables.
• why that exact incantation in the extra field is needed, or how to modify it successfully
• why the error message has zero information about the actual problem.