bug: unable to decode CustomEmailSender_SignUp encrypted code, got InvalidCiphertextException

[vt-rcheng]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

this is a reopening of bug #9348
I have tried again with the latest localstack ver 3.4.1
So I am trying to implement Cognito lambda trigger

I first sign up
root@7f0236f21f67:/opt/code/localstack# awslocal cognito-idp sign-up --client-id 7h3o0ebyozvd3vjhymdo8bvn0s --username [email protected] --password 12345678Aa! --user-attributes Name=name,Value=+11111111111

the trigger then receives a CustomEmailSender_SignUp trigger
inside the request it has an encrypted code

AgV4RSjfaHPY6cDSoaPQHqQYCDyPQESesN3pMkPvq4wScOsAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREFqdnpyZ2FKMjladVZnRFRSOTBKUXo1TCsranp2SHFGUlg2M2Q4VFM4amJ5b2ZYMk1HK0NrSGpNVkVEQnJod1o5UT09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6a2V5LzY1NjY5OGRhLTZhYTQtNDU4OC05YmZhLWQ4NGYyOTE5YzNkZgB0NjU2Njk4ZGEtNmFhNC00NTg4LTliZmEtZDg0ZjI5MTljM2RmiNnmPzJkxa6K457GXPrcGUAvV50VVZOaylI49kJFvq5VTtzUnunN5Z56+OTBKRffyvn3QnIq1b69o3ARYvmTnnsjG36q6jTXGmXlqU+PYuACAAAQAKs2LNWFjBskSk7IaYj4DURDiqEI0zyZelixydYb0d09rgnGOQixeTl24EVeNNaEov////8AAAABAAAAAAAAAAAAAAABAAAABh4cPQLLvmi4rgpxsAMSLYgkGTwoWsEAZzBlAjEAoUeOu2COlANi9iWqegJ9yCZHeEadgjaPNxedYO9knstLAemrruN7pRL0bkrxs2BgAjBYXbovABctggQj7UQV9uOXKDj1ajCifAC8EBu4moFQMw3K2UIbH4yw2FYIACEakxs=

I try to use kms to decrypt the code

root@7f0236f21f67:/opt/code/localstack# awslocal kms decrypt --ciphertext-blob fileb://<(echo "AgV4RSjfaHPY6cDSoaPQHqQYCDyPQESesN3pMkPvq4wScOsAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREFqdnpyZ2FKMjladVZnRFRSOTBKUXo1TCsranp2SHFGUlg2M2Q4VFM4amJ5b2ZYMk1HK0NrSGpNVkVEQnJod1o5UT09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6a2V5LzY1NjY5OGRhLTZhYTQtNDU4OC05YmZhLWQ4NGYyOTE5YzNkZgB0NjU2Njk4ZGEtNmFhNC00NTg4LTliZmEtZDg0ZjI5MTljM2RmiNnmPzJkxa6K457GXPrcGUAvV50VVZOaylI49kJFvq5VTtzUnunN5Z56+OTBKRffyvn3QnIq1b69o3ARYvmTnnsjG36q6jTXGmXlqU+PYuACAAAQAKs2LNWFjBskSk7IaYj4DURDiqEI0zyZelixydYb0d09rgnGOQixeTl24EVeNNaEov////8AAAABAAAAAAAAAAAAAAABAAAABh4cPQLLvmi4rgpxsAMSLYgkGTwoWsEAZzBlAjEAoUeOu2COlANi9iWqegJ9yCZHeEadgjaPNxedYO9knstLAemrruN7pRL0bkrxs2BgAjBYXbovABctggQj7UQV9uOXKDj1ajCifAC8EBu4moFQMw3K2UIbH4yw2FYIACEakxs=" | base64 --decode) --output text

but got the following error

An error occurred (InvalidCiphertextException) when calling the Decrypt operation: LocalStack is unable to deserialize the ciphertext blob. Perhaps the blob didn't come from LocalStack

Expected Behavior

No response

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

docker run localstack/localstack

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

client.create_key(
    Description='product_master_key',
    KeyUsage='ENCRYPT_DECRYPT',
    Tags=[
        {
            'TagKey': '_custom_id_',
            'TagValue': '00000000-0000-0000-0000-000000000001'
        },
    ],    
)


client.create_user_pool(
        PoolName=user_pool_name,
        Policies={
            'PasswordPolicy': {
                'MinimumLength': 6,
                'RequireUppercase': True,
                'RequireLowercase': True,
                'RequireNumbers': True,
                'RequireSymbols': False,
                'TemporaryPasswordValidityDays': 30
            }
        },
        DeletionProtection='INACTIVE',
        LambdaConfig={
            'PreSignUp': localstack_lambda_router_arn,
            'CustomMessage': localstack_lambda_router_arn,
            'PostConfirmation': localstack_lambda_router_arn,
            'PreAuthentication': localstack_lambda_router_arn,
            'PostAuthentication': localstack_lambda_router_arn,
            'DefineAuthChallenge': localstack_lambda_router_arn,
            'CreateAuthChallenge': localstack_lambda_router_arn,
            'VerifyAuthChallengeResponse': localstack_lambda_router_arn,
            'PreTokenGeneration': localstack_lambda_router_arn,
            'UserMigration': localstack_lambda_router_arn,
            'CustomSMSSender': {
                'LambdaVersion': '$LATEST',
                'LambdaArn': localstack_lambda_router_arn
            },
            'CustomEmailSender': {
                'LambdaVersion': '$LATEST',
                'LambdaArn': localstack_lambda_router_arn
            },
            'KMSKeyID': master_kms_arn
        },
        MfaConfiguration='OFF',
        UserAttributeUpdateSettings={
            'AttributesRequireVerificationBeforeUpdate': [
                'phone_number',
                'email',
            ]
        },
        DeviceConfiguration={
            'ChallengeRequiredOnNewDevice': True,
            'DeviceOnlyRememberedOnUserPrompt': True
        },
        EmailConfiguration={
            'ReplyToEmailAddress': '[email protected]',
            'EmailSendingAccount': 'COGNITO_DEFAULT',
            'From': '[email protected]',
        },
        UserPoolTags={
            '_custom_id_': user_pool_id
        },
        AdminCreateUserConfig={
            'AllowAdminCreateUserOnly': False,
            'UnusedAccountValidityDays': 7,
        },
        UserPoolAddOns={
            'AdvancedSecurityMode': 'AUDIT'
        },
        UsernameConfiguration={
            'CaseSensitive': False
        },
        AccountRecoverySetting={
            'RecoveryMechanisms': [
                {
                    'Priority': 1,
                    'Name': 'verified_email'
                },
                {
                    'Priority': 2,
                    'Name': 'verified_phone_number'
                },
                {
                    'Priority': 3,
                    'Name': 'admin_only'
                }
            ]
        }
    )


client.create_user_pool_client(
        UserPoolId=user_pool_id,
        ClientName=user_pool_client_name,
        GenerateSecret=True,
        RefreshTokenValidity=1,
        AccessTokenValidity=1,
        IdTokenValidity=1,
        TokenValidityUnits={
            'AccessToken': 'hours',
            'IdToken': 'hours',
            'RefreshToken': 'days'
        },
        ReadAttributes=[
            'email',
            'phone_number'
        ],
        WriteAttributes=[
            'email',
            'phone_number'
        ],
        ExplicitAuthFlows=[
            'ALLOW_ADMIN_USER_PASSWORD_AUTH',
            'ALLOW_USER_PASSWORD_AUTH',
            'ALLOW_USER_SRP_AUTH',
            'ALLOW_REFRESH_TOKEN_AUTH',
        ],
        SupportedIdentityProviders=[
            'COGNITO',
            'Facebook',
            'Google',
            'LoginWithAmazon'
        ],
        CallbackURLs=[
            'http://localhost',
        ],
        LogoutURLs=[
            'http://localhost',
        ],
        AllowedOAuthScopes=[
            'phone',
            'email',
            'openid',
            'profile'
        ],
        AllowedOAuthFlowsUserPoolClient=True,
        PreventUserExistenceErrors='ENABLED',
        EnableTokenRevocation=True,
        EnablePropagateAdditionalUserContextData=True,
        AuthSessionValidity=15
    )

Environment

- OS:
- LocalStack:

Anything else?

No response

[vt-rcheng]

>>> import base64
>>> import aws_encryption_sdk
>>> from aws_encryption_sdk.identifiers import CommitmentPolicy
>>> code = "AgV49DDIa0gmpF8ptVztr4W3tBIj40JM74jHpqBgszd+M04AXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREFnZ1p0REJ1SmpmUENRVjZieXpncjFLcm85cVAwV0RXQUptaDVEeG5Zb1NKMjNNcTRmMHFTTjVmQTJyalRsVno0Zz09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6a2V5LzAwMDAwMDAwLTAwMDAtMDAwMC0wMDAwLTAwMDAwMDAwMDAwMQB0MDAwMDAwMDAtMDAwMC0wMDAwLTAwMDAtMDAwMDAwMDAwMDAxhXj8VVMS4crQ0L5paAbfKyr8brLnTVr4nDgEMZyS6SMcSffyHaLtd+AMq8q+IaZmXv3GOPmpm83WwpxtR7xeWyU7wEvYmdU93lUwnc9Rl4ACAAAQAHnzBnp4qzttuCBlvMngrINOCWqu4xj7/cuTlRvjitKd5WBEZAAQnV+FQ5DhNnwAr/////8AAAABAAAAAAAAAAAAAAABAAAABnrRUG8pwekPB+5tdS2Ez480PmaFECIAZzBlAjAJYWdGe/IOrUOoLY+swDhaEIdlu25kEH2MdR/Qf8gt0OabAYjmsV4n5JVBTSf4MsoCMQD1UcVGfR31O/582Iv86fNfKS55l/LIFobUOxoj+2itcmHsWf74ZYdww2pECq+H7ck="
>>> code_bytes = base64.b64decode(code)
>>> kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=["arn:aws:kms:us-east-1:000000000000:key/00000000-0000-0000-0000-000000000001"])
>>> client = aws_encryption_sdk.EncryptionSDKClient(commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT)
>>> decrypt, header = client.decrypt(source=code_bytes, key_provider=kms_key_provider)
>>> decrypt
b'775115'

If I use aws_encryption_sdk to decode locally, then the code decrypts just fine. But if I use boto3 and send it to localstack to decrypt, then it fails!!!

[vt-rcheng]

I am also not sure why the client id of "CustomEmailSender_SignUp" event is "CLIENT_ID_NOT_APPLICABLE"
when sign up is performed, a client id is specified. other trigger event seems to have client_id value in them.

[giograno]

Hi @vt-rcheng, thanks for reporting the issue. Could you give us a few more details on how you generate the ciphertext in input? Have you tried to provide the key id to the awslocal kms decrypt command? Thank you 🙏

[vt-rcheng]

I didn't generate the ciphertext, localstack did when it sends out the CustomEmailSender_SignUp trigger. You will have to setup a cognito user pool and then sign up a user using email address to trigger the event. I have tried kms decrypt command, see my ticket description, and it did not work. The only way for me to decrypt is locally with aws_encryption_sdk. So I feel it is localstack's bug somewhere....