Replies: 2 comments
-
Hello @NickCis thanks a lot for sharing this detailed report. |
Beta Was this translation helpful? Give feedback.
0 replies
-
I have the same problem, disk read\write IO is too high (iostat util is 100%) |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi people,
tldr; I'm running Meilisearch (on a Linux, centos 7 server) alongside other services and I'm needing to limit Meilisearch resources, mainly disk bandwidth. I've tried using cgroups (via systemd:
BlockIOWriteBandwidth
andBlockIOReadBandwidth
), but, it seems that read / writes are buffered (those limits only work when files are open using theO_DIRECT
flag, see). Is there a way to limit disk resources? Or can meilisearch be configured to use direct disk operations (e.g.: thedd
command line program has theoflags=direct
parameter in order to achieve this).Thanks in advance!
Edit: I'm using Meilisearch 0.23.0
Background: I'm evaluating using Meilisearch in order to perform relevance text searches (mainly there are subtitles or closed captions). This server has several services running (alongside Meilisearch), some of those services send regular additions / deletions to Meilisearch (each of those perform changes on ~700-1400 documents).
(red lines are deletions, green additions. This info is parsed from Meilisearch standard output).
After implemented Meilisearch, I've noticed that when those additions / deletions were done, the server started to have spikes on the IO Wait (I'm measuring this with
top
). This spikes were translated on general server instability (Response times of other services also started to spike, eg.: httpd started having a 500ms / 1000ms response time for static files).I've also measured the disk (write) usage of the process Meilisearch (I'm using
iotop
in order to get this info. This data was recorded after setting Write quotas using the service file at the end of this post. Limit was set on 256K which is ignored):As the server is running Centos 7, I've tried to limit Meilisearch write / read disk bandwidth, but couldn't. This os only implements cgroups v1, which uses blkio to set i/o bandwidth quotas and it's unable to set those write quotas on buffered (non-direct) write operations.
Is there a way to limit disk resources? Or can meilisearch be configured to use direct disk operations (e.g.: the
dd
command line program has theoflags=direct
parameter in order to achieve this).Note: I've ignored cpu / ram data as introducing Meilisearch didn't create any noticeable change on those values. The server has plenty of ram (~32gb, ram; usage is on average ~10gb, so it shouldn't be a problem). Regarding cpu, it has 48 cores and a load average of 10 (again, it shouldn't be a problem).
Service file:
(Meilisearch is storing the database on
/dev/sda2
).Beta Was this translation helpful? Give feedback.
All reactions