Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression / breaking change for RPCs with SET #3242

Open
wolfgangwalther opened this issue Feb 20, 2024 · 8 comments · May be fixed by #3358
Open

Regression / breaking change for RPCs with SET #3242

wolfgangwalther opened this issue Feb 20, 2024 · 8 comments · May be fixed by #3358
Assignees
Labels
breaking change A bug fix or enhancement that would cause a breaking change bug

Comments

@wolfgangwalther
Copy link
Member

Environment

  • PostgreSQL version: 15.x
  • PostgREST version: 12.1-dev
  • Operating system: Linux

Description of issue

#3142 introduced quite a big breaking change - which I would consider a regression / bug. Applying all settings is never going to be correct / expected.

Take this for example:

create function do_something_dangerous() returns something
security definer
set search_path to pg_catalog, pg_temp
language plpgsql as $$
...
$$;

I have a few SECURITY DEFINER functions in my api. Best practice for security reasons is to add pg_catalog, pg_temp as a search_path for those kind of functions. After #3142, this suddenly changes the search_path for the whole transaction. This breaks, for example, computed fields on the returned value - and probably more.

I do have other cases of SETtings that I don't want to have applied on the transaction level. There is a reason I set them on the function and not for the authenticator role or so.

While I like the idea of hoisting some SETtings to the transaction level.. the current implementation is a problem. I don't even see a check for role, so will a CREATE FUNCTION xxx SET ROLE TO y be hoisted, too? Bad.

I think we should do at least one of, but possibly both of:

  • Have an allowlist of explicitly named parameters instead of just setting all of them.
  • Prefix those settings with pgrst.xxx or tx.xxx - and then set it without that prefix on the transaction level.

This needs to happen before we make the next release, otherwise we can't release a minor. In the current form, this is quite a breaking change.

@wolfgangwalther wolfgangwalther added bug breaking change A bug fix or enhancement that would cause a breaking change labels Feb 20, 2024
@steve-chavez
Copy link
Member

How about making it a deny list instead? The only settings we need to ensure are not modified are the role and the search_path because of impersonation + schema isolation. I can make a quick fix for this.

("select " <> intercalateSnippet ", " (searchPathSql : roleSettingsSql ++ roleSql ++ claimsSql ++ [methodSql, pathSql] ++ headersSql ++ cookiesSql ++ timezoneSql ++ funcSettingsSql ++ appSettingsSql))

Have an allowlist of explicitly named parameters instead of just setting all of them.

This will be bad for maintenance because we can't list all extensions settings. For example plan_filter.statement_cost_limit (ref). If new extensions come or if the extension changes the setting name, we would have to patch PostgREST

Prefix those settings with pgrst.xxx or tx.xxx - and then set it without that prefix on the transaction level.

This would be bad for DX. Specially since statement_timeout is standard. Increases users cognitive load plus some support burden.

I think the deny list for search_path and role should be enough.

@steve-chavez
Copy link
Member

Prefix those settings with pgrst.xxx or tx.xxx - and then set it without that prefix on the transaction level.

Btw that won't work either because those pgrst.xxx require SUPERUSER. Unless we require pg 16 and do GRANT SET ON PARAMETER on the placeholder, which is still messy and weird behavior that could break later on.

@wolfgangwalther
Copy link
Member Author

How about making it a deny list instead? The only settings we need to ensure are not modified are the role and the search_path because of impersonation + schema isolation. I can make a quick fix for this.

We can't tell what kind of settings a user might have on their function, which they don't want to have hoisted. We should never just do "all of them" - this is just going to lead to problems.

Have an allowlist of explicitly named parameters instead of just setting all of them.

This will be bad for maintenance because we can't list all extensions settings. For example plan_filter.statement_cost_limit (ref). If new extensions come or if the extension changes the setting name, we would have to patch PostgREST

Then we need a config option to set those settings, that should be hoisted.

The config option can default to some basic PG-builtin settings.

I think the deny list for search_path and role should be enough.

Certainly not. As I said above, I have other cases except those two. For example it broke for me, because I had some timezone settings on some RPCs. Those would affect how the incoming dates were interpreted - but would not affect the transformation of the date to JSON when returning the result. Now, with this change, suddenly my whole output changes. This is a case that can be worked around, might actually just be a question of adjusting my snapshots.

But there are very likely other cases out there that we will just break silently - and it was very hard for me to debug this. It will be even harder for other users, who are not that familiar with postgrest. We 100% need an allowlist approach.

@steve-chavez
Copy link
Member

I see your point. Too bad we can't have this working transparently.

So how about a db-hoisted-tx-settings. For now I think the default will be:

  • statement_timeout
  • plan_filter.statement_cost_limit
  • default_transaction_isolation
  • ...

@steve-chavez
Copy link
Member

@taimoorzaeem Could you take this one? It's pretty much a follow up of #3142

@taimoorzaeem taimoorzaeem self-assigned this Feb 28, 2024
@wolfgangwalther
Copy link
Member Author

It seems to me like the maximum length of a config is 52 characters. Please confirm if that is true.

I don't know any such limit, I assume there's something else going wrong.

@taimoorzaeem
Copy link
Collaborator

@steve-chavez @wolfgangwalther @laurenceisla

Assigning list to config in test_io.py raises this type error. How do I assign the list to the config to test it?

def test_multiple_func_settings(defaultenv):
        "check multiple function settings are applied"
    
        env = {
            **defaultenv,
            "PGRST_DB_HOISTED_TX_SETTINGS": ["work_mem", "statement_timeout"],
        }
    
>       with run(env=env) as postgrest:
.
.
.
filename = ['work_mem', 'statement_timeout']

>   ???
E   TypeError: expected str, bytes or os.PathLike object, not list

<frozen os>:812: TypeError

@laurenceisla
Copy link
Member

laurenceisla commented Mar 25, 2024

How do I assign the list to the config to test it?

Using a string doesn't work?

        env = {
            **defaultenv,
            "PGRST_DB_HOISTED_TX_SETTINGS": "work_mem,statement_timeout",
        }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change A bug fix or enhancement that would cause a breaking change bug
4 participants