Decide on search strategy

[helendduncan]

Summary

A functional search function is implemented in list_public_experiences in views.py
At the moment the search returns results only for exact matches in the story, difference, or title text.
So for the story that begins:

My family recently got a new TV.

Searching for family or TV or recently got will return the story. As will searching for got a new.

However searching for family TV will not return any results.

This is perhaps unexpected behaviour given how most search engines return results

Deadline

July 2023

What needs to be done?

Refactor the function - possible suggestion to assume the search function is a space separated list and iterate. Also deal with phrases within quotation marks as exact matches

• Decide on a couple of key features (suggested above)
• Update function
• Apply function to other areas of code that use search (perhaps move to helper function)

I think this runs a risk of being over-engineered so I think perhaps ensuring that all the examples given in the summary would return the story is a sensible stopping point.

Who can help?

---

Updates

[gedankenstuecke]

A potentially "easy" solution for the MVP might be switching to the fulltext-search functions that Django provides for Postgres, c.f. https://docs.djangoproject.com/en/4.2/topics/db/search/#postgresql-support

I haven't tried it but based on the documentation there it seems like this might already better than limiting ourselves to case-insensitive matches. It will still probably not great compared to some more specialized search solutions, but it looks like this solution should at least come for free with what we have?

[helendduncan]

100% agree

[gedankenstuecke]

We discussed that this might be a fun little project for an intern that will start in July and which could be expanded to include more advanced search (e.g. using an NLP framework that includes search for synonyms).

[gedankenstuecke]

@helendduncan In our co-working we were wondering if this should go into "on hold" or "todo" for the MVP refinements and could not decide, partially as we weren't sure how much progress on this was already made. What's your opinion? 🙂

[helendduncan]

I think there are some ideas in Obinnas branch using the Postgres search which I think could simply be appended to the filtered query set - Will do a quick proof of concept and update this post