Kafka fails to start on 1.9.0

[tanenbaum]

Bug Report

Release tag 1.9.0

Symptom

A clear and concise description of what the bug is.

What is the expected behavior?

Full demo should start with all services being healthy.

What is the actual behavior?

The docker compose set up fails with kafka being unhealthy - dependency failed to start: container kafka is unhealthy.

From the kafka logs:

java.lang.OutOfMemoryError: Java heap space
	at java.base/java.nio.HeapByteBuffer.<init>(Unknown Source)
	at java.base/java.nio.ByteBuffer.allocate(Unknown Source)
	at org.apache.kafka.storage.internals.log.SkimpyOffsetMap.<init>(SkimpyOffsetMap.java:84)
	at kafka.log.LogCleaner$CleanerThread.<init>(LogCleaner.scala:354)
	at kafka.log.LogCleaner.$anonfun$startup$2(LogCleaner.scala:158)
	at kafka.log.LogCleaner$$Lambda/0x00007fc45063e578.apply$mcVI$sp(Unknown Source)
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
	at kafka.log.LogCleaner.startup(LogCleaner.scala:157)
	at kafka.log.LogManager.startupWithConfigOverrides(LogManager.scala:648)
	at kafka.log.LogManager.startup(LogManager.scala:570)
	at kafka.server.metadata.BrokerMetadataPublisher.initializeManagers(BrokerMetadataPublisher.scala:292)
	at kafka.server.metadata.BrokerMetadataPublisher.onMetadataUpdate(BrokerMetadataPublisher.scala:127)
	at org.apache.kafka.image.loader.MetadataLoader.initializeNewPublishers(MetadataLoader.java:309)
	at org.apache.kafka.image.loader.MetadataLoader.lambda$scheduleInitializeNewPublishers$0(MetadataLoader.java:266)
	at org.apache.kafka.image.loader.MetadataLoader$$Lambda/0x00007fc4505ad280.run(Unknown Source)
	at org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:127)
	at org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:210)
	at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:181)
	at java.base/java.lang.Thread.runWith(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

Reproduce

Check out tag 1.9.0 and run make start.

This occurs on my machine: Linux joe-fedora 6.8.6-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Apr 13 15:14:23 UTC 2024 x86_64 GNU/Linux - 16 core, 32 GiB RAM.

This also occurs on cloud VMs that I have been testing on, at various configurations.

Additional Context

When observing the memory usage of the kafka container, it quickly climbs to 200MB+ which exceeds the Java heap settings of the container KAFKA_HEAP_OPTS=-Xmx200m -Xms200m.

[darkleaf]

Works for me:

      - KAFKA_HEAP_OPTS=-Xmx300m -Xms300m