MongoDB Outbox + SQS order and delivery retries #5165
Unanswered
felix-v-net
asked this question in
Q&A
Replies: 1 comment 2 replies
-
I don't think so, you'll need to clear the outbox table using SQL most likely. |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
We have just started using MassTransit, very attractive set of features, struggling with configuration a little bit.
The biggish issue came up today when one of the messages was too big to be delivered and the outbox has just stuck with trying indefinitely to deliver this one message.
Is there a way to configure are number of retries to deliver a message and drop it after a number of tries? or setup different outboxes for each of the message type? At the moment it looks like it setup to use the same outbox for all message types, so in this case NONE of the any message types was going out. So ALL the messaging has stopped in the system because of one message was failing to be deliverred.
Here is the error:
{ "@t": "2024-05-08T01:25:00.6147614Z", "@m": "S-FAULT amazonsqs://ap-southeast-2/prod-upserted-invoice?type=topic 07000000-86e7-ba61-f615-08dc6e8de2ab \"MassTransit.Serialization.SerializedMessageBody\"", "@i": "08ba9748", "@l": "Error", "@x": "Amazon.SimpleNotificationService.Model.BatchRequestTooLongException: The length of all the messages put together is more than the limit.\n ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.\n at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage)\n at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)\n at Amazon.Runtime.Internal.HttpHandler
1.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)\n --- End of inner exception stack trace ---\n at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)\n at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)\n at Amazon.Runtime.Internal.ExceptionHandler1.HandleAsync(IExecutionContext executionContext, Exception exception)\n at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)\n at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)\n at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)\n at MassTransit.AmazonSqsTransport.PublishBatcher.SendBatch(IList
1 batch) in //src/Transports/MassTransit.AmazonSqsTransport/AmazonSqsTransport/PublishBatcher.cs:line 42\n at MassTransit.AmazonSqsTransport.Batcher1.ExecuteBatch(IList
1 batch) in //src/Transports/MassTransit.AmazonSqsTransport/AmazonSqsTransport/Batcher.cs:line 139\n at MassTransit.AmazonSqsTransport.Batcher1.Execute(TEntry entry, CancellationToken cancellationToken) in /_/src/Transports/MassTransit.AmazonSqsTransport/AmazonSqsTransport/Batcher.cs:line 42\n at MassTransit.AmazonSqsTransport.AmazonSqsClientContext.Publish(String topicName, PublishBatchRequestEntry request, CancellationToken cancellationToken) in /_/src/Transports/MassTransit.AmazonSqsTransport/AmazonSqsTransport/AmazonSqsClientContext.cs:line 152\n at MassTransit.AmazonSqsTransport.TopicSendTransportContext.Send[T](ClientContext transportContext, SendContext
1 sendContext) in //src/Transports/MassTransit.AmazonSqsTransport/AmazonSqsTransport/TopicSendTransportContext.cs:line 93\n at MassTransit.Transports.SendTransport1.SendPipe
1.Send(TContext context) in //src/MassTransit/Transports/SendTransport.cs:line 93","DestinationAddress": "amazonsqs://ap-southeast-2/prod-upserted-invoice?type=topic",
"MessageId": "07000000-86e7-ba61-f615-08dc6e8de2ab",
"MessageType": "MassTransit.Serialization.SerializedMessageBody",
"SourceContext": "MassTransit",
"AppType": "DOTNET8.API",
"AppName": "HealthNow.API"
}`
Here is how we add Mass Transit with MongoDB Outbox:
`services.AddMassTransit(mt =>
{
mt.AddMongoDbOutbox(o =>
{
o.QueryDelay = TimeSpan.FromSeconds(1);
o.ClientFactory(provider => provider.GetRequiredService());
o.DatabaseFactory(provider => provider.GetRequiredService());
o.DuplicateDetectionWindow = TimeSpan.FromSeconds(30);
o.UseBusOutbox();
});
Much appreciate your insights :-)
Beta Was this translation helpful? Give feedback.
All reactions