Skip to content

Releases: Azure/azure-event-hubs-spark

v2.3.21

17 Aug 04:27
ba7561c
Compare
Choose a tag to compare

Release notes:

  • performance improvement - added data cache to avoid unnecessary fetch operations for multiple readers scenario (#608 )

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.21/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.21/jar

v2.3.20

29 Jun 15:40
e37e127
Compare
Choose a tag to compare

Release notes:

  • Update java SDK dependency which contains a fix for race conditions resulting in timeout error. (#604)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.20/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.20/jar

v2.3.19

17 Jun 18:43
fcb9e26
Compare
Choose a tag to compare

Release notes:

  • Support parameters for Azure AD authentication (#596)
  • Improve logging (#575, #600)
  • Fix bugs (#584, #577)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.19/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.19/jar

v2.3.18

13 Nov 23:58
e627c87
Compare
Choose a tag to compare

Release notes:

  • Support Azure AD authentication (#535)
  • Support partition scale-out scenario (#500, #544)
  • Fix bugs (#550, #505)
  • ThrottlingStatus plug-in update (#555)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.18/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.18/jar

v2.3.17

07 Aug 19:00
b349217
Compare
Choose a tag to compare

Release notes:

  • Spark runtime 3.0 support (#520).
  • Slow partition adjustment support (#512).
  • Fix an issue of retrying receiver close (#523).

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.17/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.17/jar

v2.3.16

26 Jun 02:52
34eeed9
Compare
Choose a tag to compare

Release notes:

  • Fix added quotes to string property values (#516).
  • Add receive watchdog (#517).
  • EventHubs write - allow Properties map to be non-nullable or nullable column (#507).
  • Retry on send failures, improve logging and metrics (#504)
  • Support Apache Kafka key property (#501)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.16/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.16/jar

v2.3.15

14 Apr 17:45
ad3f840
Compare
Choose a tag to compare

Release notes:

  • Add support for different partitioning strategies to deal with reader/executor skew (#478).
  • Add null check for properties key-value and a test case (#483)
  • Fix an issue of closing a receiver for epoch receiver case (#486)
  • Fix IllegalStateException issue and recreate a receiver if the receiver was closed (#490)
  • Encrypt connection string in eventhub conf (#491)
  • Use receiver timeout for underlying API calls to Event Hubs so that connector can retry on failures (#492)
  • Add receive, send listener for getting performance metrics (#473)
  • Fix a few issues for Scala 2.12 and code clean-up (#498)
  • Add SimpleLogMetricPlugin class (#499)

For PySpark scenario, version 2.3.15 assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.15/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.15/jar

v2.3.13

03 Jul 17:50
ad3f840
Compare
Choose a tag to compare

Release notes:

  • add event hub name to the OffsetRange toString function so that it is present in the spark metadata. add simple unit test for validation of the string (#450).

Package link - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.13/jar

v2.3.12

06 May 16:34
081e76b
Compare
Choose a tag to compare

Release notes:

  • Update Azure Event Hubs SDK dependency and use the latest version of the SDK that contains a fix for the issue of reconnect failure after a connection is terminated.

Package link - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.12/jar

v2.3.11

30 Apr 16:59
d9f4fe0
Compare
Choose a tag to compare
v2.3.11 Pre-release
Pre-release

Release notes:

  • performance improvement for EventHubs writer (#441)

Package link - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.11/jar