Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] EventHubConsumerAsyncClient - Connection Aborted #40104

Closed
3 tasks done
alemag1986 opened this issue May 10, 2024 · 6 comments
Closed
3 tasks done

[BUG] EventHubConsumerAsyncClient - Connection Aborted #40104

alemag1986 opened this issue May 10, 2024 · 6 comments
Assignees
@anuchandy
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Event Hubs needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that

Comments

@alemag1986
Copy link

alemag1986 commented May 10, 2024 •
edited

Describe the bug
Using Event Hub PREMIUM tier, trying to connect from a spring boot application running on a container in a webapp getting connection aborted, and no description of the error:

2024-05-10T05:25:48.547189346Z reactor.core.Exceptions$ErrorCallbackNotImplemented: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A]
2024-05-10T05:25:48.547193246Z Caused by: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A]

Exception or Stack Trace
24-05-10T05:25:48.547189346Z reactor.core.Exceptions$ErrorCallbackNotImplemented: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A]
2024-05-10T05:25:48.547193246Z Caused by: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A]
2024-05-10T05:25:48.547197446Z at com.azure.core.amqp.implementation.ExceptionUtil.toException(ExceptionUtil.java:90) ~[azure-core-amqp-2.9.3.jar!/:2.9.3]
2024-05-10T05:25:48.547201746Z at com.azure.core.amqp.implementation.handler.ConnectionHandler.notifyErrorContext(ConnectionHandler.java:363) ~[azure-core-amqp-2.9.3.jar!/:2.9.3]
2024-05-10T05:25:48.547205646Z at com.azure.core.amqp.implementation.handler.ConnectionHandler.onTransportError(ConnectionHandler.java:259) ~[azure-core-amqp-2.9.3.jar!/:2.9.3]
2024-05-10T05:25:48.547209546Z at org.apache.qpid.proton.engine.BaseHandler.handle(BaseHandler.java:191) ~[proton-j-0.34.1.jar!/:na]
2024-05-10T05:25:48.547213446Z at org.apache.qpid.proton.engine.impl.EventImpl.dispatch(EventImpl.java:108) ~[proton-j-0.34.1.jar!/:na]
2024-05-10T05:25:48.547218246Z at org.apache.qpid.proton.reactor.impl.ReactorImpl.dispatch(ReactorImpl.java:324) ~[proton-j-0.34.1.jar!/:na]
2024-05-10T05:25:48.547221846Z at org.apache.qpid.proton.reactor.impl.ReactorImpl.process(ReactorImpl.java:291) ~[proton-j-0.34.1.jar!/:na]
2024-05-10T05:25:48.547225446Z at com.azure.core.amqp.implementation.ReactorExecutor.run(ReactorExecutor.java:91) ~[azure-core-amqp-2.9.3.jar!/:2.9.3]
2024-05-10T05:25:48.547228746Z at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) ~[reactor-core-3.6.3.jar!/:3.6.3]
2024-05-10T05:25:48.547232346Z at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) ~[reactor-core-3.6.3.jar!/:3.6.3]
2024-05-10T05:25:48.547236346Z at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[na:na]
2024-05-10T05:25:48.547251746Z at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[na:na]
2024-05-10T05:25:48.547255446Z at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
2024-05-10T05:25:48.547258946Z at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
2024-05-10T05:25:48.547262746Z at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
2024-05-10T05:25:48.547266547Z
2024-05-10T05:25:48.547269647Z 2024-05-10T05:25:48.542Z ERROR 15 --- [tor-executor-16] c.a.core.amqp.implementation.RetryUtil : Time out creating management node.
2024-05-10T05:25:48.547273647Z Retries exhausted: 3/3

Code Snippet

public EventHubConsumerAsyncClient eventHubConsumerAsyncClient() {

    EventHubClientBuilder eventHubClientBuilder = new EventHubClientBuilder()
        .connectionString(namespace, eventHubName)
        .consumerGroup(consumerGroupName);

    EventHubConsumerAsyncClient client = eventHubClientBuilder.buildAsyncConsumerClient();
    // Perform a health check to ensure the connection is successful
    try {
        client.receive().blockFirst();
        log.info("Connected to Event Hub successfully.");
        isConnected = true;
    } catch (Exception e) {
        log.error("Error connecting to Event Hub: {}", e.getMessage());
        log.error("Error StackTrace: {}", e.toString());
    }
    return client;
}

This is a health check basically. This same code, with same web app configuration is working on our Development environment that is NOT using EventHub PREMIUM tier.

Expected behavior
Connection to Event Hub and start to recieve messages.

Setup (please complete the following information):
Library versions:

implementation 'com.azure:azure-messaging-eventhubs:5.18.3'
implementation 'com.azure:azure-identity:1.12.1'

Docker Base:
FROM amazoncorretto:21-alpine as corretto-jdk

Java:
plugins {
id 'java'
id 'org.springframework.boot' version '3.2.3'
id 'io.spring.dependency-management' version '1.1.4'
id 'org.sonarqube' version '4.4.1.3373'
}

java {
sourceCompatibility = '21'
}

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • Bug Description Added
  • Repro Steps Added
  • Setup information Added
@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Event Hubs needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels May 10, 2024
@github-actions GitHub Actions
Copy link

@anuchandy @conniey @lmolkova

@github-actions GitHub Actions
Copy link

Thank you for your feedback. Tagging and routing to the team member best able to assist.

@alemag1986
Copy link
Author

Hi there,
We are still troubleshooting the issue from our side... it looks like the issue happens when the docker image is build using Ubuntu.
Our docker build process happens in Azure DevOps Pipeline Ubuntu latest:
https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md

@anuchandy
Copy link
Member

anuchandy commented May 14, 2024 •
edited

@alemag1986, are you saying this connection-aborted only occurs with a specific version of Ubuntu and it's not related to the Event Hub tier (standard, premium)? Is this something we reproduce when running on local docker or only on AKS?

@alemag1986
Copy link
Author

@anuchandy

  • its not happening in Event Hub standard tier.
  • it only happens in premium.
  • it is reproducible from local docker not need to cloud infrastructure (in the morning will share docker info).

@alemag1986
Copy link
Author

alemag1986 commented May 20, 2024 •
edited

The issue is related to Docker Image. The base ubuntu has been tailored for our use case and unfortunately something is missing to support eventhub premium.

Here are the docker file details in case somebody is struggling or want to find what is missing.

# Use the official OpenJDK image as the base image
FROM amazoncorretto:21-alpine as corretto-jdk
WORKDIR /app
COPY ./deps.info /app/deps.info


# required for strip-debug to work
RUN apk add --no-cache binutils


# Build small JRE image
RUN jlink \
         --verbose \
         --add-modules $(cat deps.info) \
         --strip-debug \
         --no-man-pages \
         --no-header-files \
         --compress=2 \
         --output /jre

FROM alpine:3.19

ENV JAVA_HOME=/jre
ENV PATH="${JAVA_HOME}/bin:${PATH}"

COPY --from=corretto-jdk /jre $JAVA_HOME

CMD ["/bin/sh"]

Going to close the issue... since is not related to the library. Most likely, we are missing a requirement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@anuchandy anuchandy

Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Event Hubs needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alemag1986 @anuchandy