Performing search CPU spikes

[markwriter]

Hi, we haven't gone live yet but are very close. I have deployed our app w/Lifti to a QA server and a staging server, and both exhibit CPU spikes when searches are performed. In each case the server is a medium AWS instance with 32 gigs of RAM and a medium AMD processor. They are idling at around 3% when a single search is performed, after which one of server's CPU's can range up into the 70% range while the other one for some reason goes into the 90's. FWIW, there are 4 indexes that get hit asynchronously. They are fairly small, each one under a couple thousand database records comprising a cumulative total of 6 MB. At runtime the user submits a search term and each of the indexes is queried.
When I do the search on my laptop the cpu only spikes to around 60%.

Any thoughts on this? I'll try to get more CPU profiling information to see if I can tell specifically what is happening.

[mikegoatly]

Are the queries that are being run the same? I would expect different operators to exhibit different cpu cost, so if the queries aren't identical then that could be part of it.

[markwriter]

Hi, I discovered that I may have caused the problem by loading the thesaurus incorrectly. Currently I have about 50 items in it and when I comment out all but a few of them the CPU spiking goes away (and that fantastic Lifti-speed returns!). I will take a bit of time to figure out which item(s) are the culprit and get back to you.

[mikegoatly]

Assuming that what you're experiencing isn't a bug in LIFTI, the most likely cause will be due to an increased number of matches that need to be merged in the case of a query containing lots of search terms, or there is a wildcard in the query that's becoming computationally expensive to run.

Let me know what you find out - if there's a concrete example I can work with I'd be happy to find some time to take a look.

[mikegoatly]

@markwriter I'm interested to know if you're able to share where you got to with this?

[markwriter]

Sorry!! I never was able to test it long enough to figure out why the slowdown occurred. I would like to get back to it.
Also, I didn't mention that we finally went to production with this and you have made some users extremely happy with 1 second search times compared to the 10 seconds that it was previously.

[mikegoatly]

That's great to hear, @markwriter. I'll keep this issue open for a while in case you get chance to look into it any more.