Worst Case (underutilization in case of hot-partition )

[dev-badace]

Hi there,
Since dragonfly uses shared-nothing approach, is the memory allocated to each shard during the startup time?
Can shards request for more memory as required during runtime?
If not, then let's say we're running 8 shards each shard is allocated 3gb of memory. some shards get more load than other & 3 shards get full (totally consumed the available memory) while the remaining 5 shards are still under 50%. and a new insert request comes to one of the three partitions that are full. what is going to be the expected behavior in this case.

[romange]

Great question, showing a good understanding of shared-nothing!
Dragonfly does not pre-allocate memory on startup. Instead, we carefully chose a memory allocator that has friendly properties for shared-nothing. Therefore, it's totally possible that some shards will allocate more than others.

So short answer - while we provide shared-nothing at the architectural level, the memory is pooled and it's completely transparent to our app logic.