The bot knows about Wolf Richter, Wolf Street, and the Wolf Street articles listed in data.json. The oldest known article dates back to 05 Apr 2024.
- Wolf Street articles were saved as markdown files, split into smaller chunks, and embedded in a Chroma database.
- On (almost) every query, the bot embeds your query, identifies the 7 most similar article chunks and 0-3 most relevant article summaries and places them into GPT's context to answer your question. This technique is known as Retreival-Augmented Generation (RAG).
- RAG is far from perfect! I used the open-sourced
all-MiniLM-L6-v2model to create the embeddings. I used it because it's free (I'm cheap) and has good speed and performance for what you're getting. - I removed all those markdown Wolf Street articles from this repo out of respect to Wolf Richter.
data.py is a mess of a codebase that shows how I retrieved, chunked, and embedded the Wolf Street articles
chatbot.py is the UI logic for the Streamlit chatbot.
chatbot_helper.py is the helper functions for the Streamlit chatbot. This is where the magic happens with the GPT chat completions.
This bot was built by Gordon Weakliem. He's been a Wolf Street reader for about 4 years and donates to the site. He wanted to learn more about AI chatbots and RAG and this code was forked from the StratecheryChatbot, a GPT bot that uses RAG to answer questions about the Stratechery weblog.
Disclaimer: This app is not affiliated with, endorsed by, or approved by Wolf Richter or Wolf Street.