Force GC on auto-pause to save RAM

[walking-octopus]

Problem

Auto-pause, despite saving the CPU power, does little to curb the rather enormous RAM usage a Minecraft server, especially with Forge, can impose on the system, being rather problematic for installations where Minecraft lives alongside other services like a home server. Auto-shutdown may seem like a solution, but as far as I can see, it doesn't support automatic startup on new network connections like auto-pause, so isn't considered a viable alternative.

Proposed solution

I propose to amend this feature by forcing the JVM to perform a garbage collection before pausing. One way I found of achieving it would look something like

jcmd $(jps -l | grep forge-1.12.2-14.23.5.2860.jar | awk '{print $1}') GC.run

which would require the following options for the JVM to enable debugging

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9010
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

Perhaps there is a better implementation, but this serves as illustration for my proposal. This simple feature may make a great difference for many RAM-scarce homelabs.

Alternatives considered

• Network-based auto-startup paired with existing auto-shutdown, like https://github.com/gekware/minecraft-server-hibernation.

[itzg]

Thanks, that is a good suggestion.

My concern is that garbage collection will compact the JVM's heap memory, but I don't believe it will release any of that heap allocation back to the operating system. Can you research that with one of your running instances?

I'm also pretty sure that jcmd is not included with the JRE base images, just the JDK (includes full development kit, compiler, etc).

[walking-octopus]

You're right, it seems JVM really doesn't like releasing memory back to the OS. However I found a few flags that may help. I'm not a Java developer, so perhaps there is some performance cost or nuance I wasn't aware of.

• -XX:-ShrinkHeapInSteps in JDK 9 will force the JVM to more aggressively release memory at a performance cost.
• -XX:GCTimeRatio=19 -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=30 can instruct JVM 8 to release memory under a certain idle/low load state.
• JDK 12 seems to by default release unused memory as seen here.

I haven't yet had the opportunity to test any of this but perhaps this can force the JVM to not hold onto memory when paused.

Also, how is the pause feature implemented? Is the process itself paused or is the JVM aware of it? If any of this is enabled, could JVM regard this as idle state and release the unused memory?

I'm anything but an expert here, so please correct me if I am wrong.

[itzg]

Good research. So it might turn out that users adding those options for Java 8 containers and relying on the default G1 for Java 17 etc might the best that can be done anyway.

To answer your question it pauses the process as a whole with a STOP signal

docker-minecraft-server/files/auto/pause.sh

Line 23 in e38cc6e

pkill -STOP java

So I don't believe the JVM will be aware of that or even given a chance to interpret it.

[walking-octopus]

1. I suppose the server already has plenty of time of inactivity, so with these options the GC might just run and compact/free some of the heap itself. The only possible change could be at best making them a default if adding them does change anything about the memory footprint...

2. The kernel understands quite well that these hung processes make for prime swapping candidates. If there's any deficit of RAM, it would already start swapping them as is. But for the purposes of housekeeping, wouldn't it be nice to trigger this sooner than later since we've assumed the server would be off for some time before pausing it? I found one approach to doing so here.

There is a special syscall process_madvise you can use that that lets you select mappings of memory from a foreign process and you can pass the MADVISE_PAGEOUT flat to fault out the memory into swap.

I wonder if it can be called inside the container to instantly swap most of the server's memory usage.

Sadly though, after testing it, it doesn't seem it swaps out more than ~25% of the memory...

3. Maybe it would be better to support knock for server startup? Instead of stopping the container, we can keep the auto-start process running...