value too long for type character varying(9) when processing checkout.session.completed webhook (possible stripe API version 2024-04-10 issue)

[luc-vocab]

Describe the bug
processing a checkout.session.completed webhook after a purchase results in an error: value too long for type character varying(9)

This is in test mode on a newly opened stripe account which is on API version 2024-04-10. I previously did a very thorough amount of testing on another stripe account which is on version 2023-08-16 and never encountered that issue. I am purposefully keeping a stripe account in test mode (dev/qa/prod environments). It doesn't appear to be possible to downgrade the API version on a stripe account.

Sentry crash report:
https://language-tools.sentry.io/share/issue/f1549f96d5ed42c59af3b21d65e96d01/

Full stack trace:

PaymentIntent.DoesNotExist: PaymentIntent matching query does not exist.
  File "djstripe/models/base.py", line 670, in _create_from_stripe_object
    instance = cls.stripe_objects.get(id=id_)
  File "django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 647, in get
    raise self.model.DoesNotExist(

StringDataRightTruncation: value too long for type character varying(9)

  File "django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)

DataError: value too long for type character varying(9)

  File "django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "django/utils/decorators.py", line 48, in _wrapper
    return bound_method(*args, **kwargs)
  File "django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
  File "django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
  File "djstripe/views.py", line 47, in post
    trigger = WebhookEventTrigger.from_request(
  File "djstripe/models/webhooks.py", line 268, in from_request
    raise e
  File "djstripe/models/webhooks.py", line 249, in from_request
    obj.process(save=False, api_key=api_key)
  File "djstripe/models/webhooks.py", line 363, in process
    self.event = Event.process(self.json_body, api_key=api_key)
  File "djstripe/models/core.py", line 1656, in process
    ret.invoke_webhook_handlers()
  File "djstripe/models/core.py", line 1668, in invoke_webhook_handlers
    webhooks.call_handlers(event=self)
  File "djstripe/webhooks.py", line 98, in call_handlers
    handler_func(event=event)
  File "djstripe/event_handlers.py", line 353, in other_object_webhook_handler
    _handle_crud_like_event(target_cls=target_cls, event=event)
  File "djstripe/event_handlers.py", line 455, in _handle_crud_like_event
    obj = target_cls.sync_from_stripe_data(data, api_key=event.default_api_key)
  File "djstripe/models/base.py", line 1051, in sync_from_stripe_data
    instance, created = cls._get_or_create_from_stripe_object(
  File "djstripe/models/base.py", line 790, in _get_or_create_from_stripe_object
    cls._create_from_stripe_object(
  File "djstripe/models/base.py", line 660, in _create_from_stripe_object
    stripe_data = cls._stripe_object_to_record(
  File "djstripe/models/base.py", line 410, in _stripe_object_to_record
    field_data, skip, is_nulled = cls._stripe_object_field_to_foreign_key(
  File "djstripe/models/base.py", line 536, in _stripe_object_field_to_foreign_key
    ) = field.related_model._get_or_create_from_stripe_object(
  File "djstripe/models/base.py", line 790, in _get_or_create_from_stripe_object
    cls._create_from_stripe_object(
  File "djstripe/models/base.py", line 685, in _create_from_stripe_object
    instance.save()
  File "django/db/models/base.py", line 822, in save
    self.save_base(
  File "django/db/models/base.py", line 909, in save_base
    updated = self._save_table(
  File "django/db/models/base.py", line 1067, in _save_table
    results = self._do_insert(
  File "django/db/models/base.py", line 1108, in _do_insert
    return manager._insert(
  File "django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 1845, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "django/db/models/sql/compiler.py", line 1823, in execute_sql
    cursor.execute(sql, params)
  File "django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
  File "django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)

Here's the the webhook data:

{
  "id": "evt_1P5QPuFotidoaBFMBmfioE9s",
  "object": "event",
  "api_version": "2024-04-10",
  "created": 1713090970,
  "data": {
    "object": {
      "id": "cs_test_b1ih5g9FLiplJDF37Q6oUgaxht8lLGzInwbhL3Yu5hql3AiiVYS3uKS3I4",
      "object": "checkout.session",
      "after_expiration": null,
      "allow_promotion_codes": true,
      "amount_subtotal": 32088,
      "amount_total": 32088,
      "automatic_tax": {
        "enabled": false,
        "liability": null,
        "status": null
      },
      "billing_address_collection": null,
      "cancel_url": "https://app.vocab.ai/prepaid/products/prod_Pv57YH8otEZ176/purchase/cancel/",
      "client_reference_id": "1",
      "client_secret": null,
      "consent": null,
      "consent_collection": null,
      "created": 1713090965,
      "currency": "hkd",
      "currency_conversion": null,
      "custom_fields": [
      ],
      "custom_text": {
        "after_submit": null,
        "shipping_address": null,
        "submit": null,
        "terms_of_service_acceptance": null
      },
      "customer": "cus_PvFP0vcrE3p1SD",
      "customer_creation": null,
      "customer_details": {
        "address": {
          "city": null,
          "country": "HK",
          "line1": null,
          "line2": null,
          "postal_code": null,
          "state": null
        },
        "email": "[email protected]",
        "name": "luc",
        "phone": null,
        "tax_exempt": "none",
        "tax_ids": [
        ]
      },
      "customer_email": null,
      "expires_at": 1713177364,
      "invoice": null,
      "invoice_creation": {
        "enabled": false,
        "invoice_data": {
          "account_tax_ids": null,
          "custom_fields": null,
          "description": null,
          "footer": null,
          "issuer": null,
          "metadata": {
          },
          "rendering_options": null
        }
      },
      "livemode": false,
      "locale": null,
      "metadata": {
        "source": "ecommerce"
      },
      "mode": "payment",
      "payment_intent": "pi_3P5QPtFotidoaBFM1jESvr9b",
      "payment_link": null,
      "payment_method_collection": "if_required",
      "payment_method_configuration_details": null,
      "payment_method_options": {
        "card": {
          "request_three_d_secure": "automatic"
        }
      },
      "payment_method_types": [
        "card",
        "alipay",
        "wechat_pay"
      ],
      "payment_status": "paid",
      "phone_number_collection": {
        "enabled": false
      },
      "recovered_from": null,
      "setup_intent": null,
      "shipping_address_collection": null,
      "shipping_cost": null,
      "shipping_details": null,
      "shipping_options": [
      ],
      "status": "complete",
      "submit_type": null,
      "subscription": null,
      "success_url": "https://app.vocab.ai/prepaid/products/prod_Pv57YH8otEZ176/purchase/success/?session_id={CHECKOUT_SESSION_ID}",
      "total_details": {
        "amount_discount": 0,
        "amount_shipping": 0,
        "amount_tax": 0
      },
      "ui_mode": "hosted",
      "url": null
    }
  },
  "livemode": false,
  "pending_webhooks": 1,
  "request": {
    "id": null,
    "idempotency_key": null
  },
  "type": "checkout.session.completed"
}

To Reproduce
Perform a purchase using the hosted stripe checkout page.

Software versions

• Dj-Stripe version: 2.8.3
• Python version: 3.11
• Django version: 5.0.1
• Stripe API version: 2024-04-10 (just opened the account)
• Database type and version: PostgreSQL 16

[czue]

I am also running into this issue. It appears to be stemming from the capture_method field value of automatic_async which was indeed added in the latest Stripe API: https://docs.stripe.com/upgrades#2024-04-10

I have found that the following migration file, which increases the length of the field to 255 characters temporarily fixes it.

# Generated by Django 5.0.3 on 2024-04-16 12:28

from django.db import migrations, models


class Migration(migrations.Migration):
    dependencies = [
        ("djstripe", "0012_2_8"),
    ]

    operations = [
        migrations.RunSQL(
            "ALTER TABLE djstripe_paymentintent ALTER COLUMN capture_method TYPE varchar(255);"
        ),
    ]

A simple solution in the library would be to bump up the length of the field. I will submit a PR for that.

[agusmakmun]

Hello @jleclanche, any update for this? I'm facing the same issue.

[jleclanche]

Due to the issue I believe we will exceptionally release a 2.8.5 with migrations included.

[agusmakmun]

Thank you for the quick response @Jenselme 👍 we're waiting this updates.

[luc-vocab]

@jleclanche very appreciated, thank you for your work.

[czue]

@jleclanche would it be helpful for me to add the migration to #2040?

[jleclanche]

@czue Yes please go for it.

[czue]

@jleclanche migration added. also created a PR into the stable/2.8 branch (#2041) as I wasn't totally sure where it should go.

[agusmakmun]

@jleclanche I believe we need to include all migration files in the new release version, the reason is for old dj-stripe versions who had migration files need to clear the django_migrations table, and then migrate again.

docker exec -it <postgres_container_name_or_id> psql -U <username> <database_name>
DROP TABLE django_migrations;

then;

python manage.py migrate --fake

[agusmakmun]

Hello @jleclanche sorry to ping you again. When we expect the new release will happens?
as it impacted the important events for the webhook listener, such as:

charge.succeeded
invoice.payment_succeeded
payment_intent.succeeded

[czue]

@agusmakmun if you're needing a patch urgently you can always add a migration like the one mentioned above to any of your Django apps. I would expect it to fix the issue and be compatible with any future migration that ships in djstripe

[czue]

@jleclanche sorry to pester, but just wondering what the steps are to go from #2040 or #2041 to a release, and whether there's anything I can do to help move things along?

At the moment anyone who uses this library with a new Stripe account will get all kinds of errors in webhooks (and maybe other places). Not a great experience.

[luc-vocab]

Note that stripe support can downgrade your API level if you ask them, that's another way of getting dj-stripe working right now.

[jleclanche]

Folks, I'm going to try to get a release to happen ASAP but I'm swamped IRL right now.

If you need to run this in production, you can write the following in your SQL db:

alter table djstripe_paymentintent alter column capture_method type character varying(15);

I haven't tested it but it should work as-is and be forward-compatible with future migrations.

[jleclanche]

This is fixed in 2.9.0a1 which was released recently, please test that release (note: it has not been tested in production yet, please create issues with all feedback).

[czue]

@jleclanche awesome, thanks! What's the easiest way for me to see what else is in that release?

[jleclanche]

It's all here: https://github.com/dj-stripe/dj-stripe/blob/stable/2.9/docs/history/2_9_0.md