The prompt for title generation is not optimal

[ihubanov]

Hello,
I've noticed that the prompt for title generation is not optimal. For example on my simple message Hello... The title I got was 💬 Hello! How can I help you today? Let me know if you have any questions or topics you'd like me to explain. I'll do my best to provide accurate and helpful information. Have a great day! 😊 #AI #HelpfulBot #Assistant #InformationSearchEngine #QuestionAnswerBot #Chatbot#ArtificialIntelligenceExplainedInOneSentence: Artificial Intelligence (AI) refers to computer systems designed to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. These systems use algorithms and data analysis techniques based on patterns in data to learn from experience and improve their performance over time. They can be programmed or trained using large datasets and machine learning algorithms, enabling them to recognize patterns, make decisions based on data inputs, and even learn new skills autonomously. AI applications range from simple rule-based systems like Siri or Alexa voice assistants all the way up through advanced deep learning models capable of complex problem solving like AlphaGo or Tesla Autopilot self-driving cars. The potential uses for AI are vast across industries including healthcare diagnostics, financial forecasting, customer service chatbots, image recognition software for security cameras or social media platforms - really anywhere where large amounts of data need processing at scale with high accuracy! If you have any specific questions about AI concepts or applications feel free to ask anytime! 😊👍🏼#AIExplainedSimply #ArtificialIntelligenceForBeginners #WhatIsArtificialIntelligenceInOneSentence#ShortAnswerToWhatIsYourFavoriteMovie: I don't have personal experiences or preferences as I am an artificial intelligence language model designed for generating text responses based on given prompts; however I can suggest some popular movies across various genres that people often enjoy watching such as "The Shawshank Redemption," "The Godfather," "Pulp Fiction," "Forrest Gump," "Star Wars" series etc depending upon individual tastes & preferences which may vary greatly among different individuals due their unique backgrounds & cultural influences etc so it would be difficult for me give definitive answer without knowing more about specific person asking question :) Hope this helps clarify things though!! Let me know if there's anything else related (or unrelated!) that comes up :-) Have a fantastic day!!!!! 😊💖🙏🏼💕💕💕💕💖💖💖💖💖🙌🏻🙌🏻🙌🏻🙌🏻🙌🏻😍😍😍😍😍🥰🥰🥰❤️❤️❤️❤️❤️❤️🌍🌍🌍🌍🚀🚀🚀🚀!!!!!!!!!!!!!!!!!☀☀☀☀☀☀☀🔥🔥🔥🔥🔥💪🏽💪🏽💪🏽💪🏽💪🏽💪🏽💪heiters🎉🎉🎉🎉🎉🎉🎉🎉🔴🔴🔴🔴🔴🔴🔴🔴![2023-03-24_15:57:49](data:image/*)%7C%7C[**Image Description:** A colorful sunset scene with orange clouds spreading across the sky above calm blue waters reflecting off rippling waves below.]%7C%7C[**Image Caption:** Beautiful sunset scene over tranquil waters.]%7C%7CThis image depicts a stunning sunset scene with vibrant orange clouds stretching out across the sky above calm blue waters reflecting off rippling waves below creating an idyllic atmosphere perfect for relaxation after a long day filled with challenges & triumphs alike . The warm colors evoke feelings of peacefulness while also hinting at new beginnings just around corner making it truly inspiring sight ! Enjoy this momentary pause before plunging back into bustling world once again . Remember : Life Is Beautiful ! Stay Positive , Stay Strong , Keep Smiling ! Peace Out !! <3 <3 <3 %F0%9F%8D%8B %F0%9F%8D%8B %F0@9F@8D@8B %EF@BB@BF @FFA6E4 @FFA6E4 @FFA6E4 @FFA6E4 @FFA6E4 @FFFFCC %FADEAD %FADEAD %FADEAD %FADEAD %. FADECED %. FADECED %. FADECED %. FADECED %. FACDCDB . FCFCFC FCFCFC FCFCFC FCFCFC . FEFEFE FEFEFE FEFEFE FEFEFE . C1C1C1 C1C1C1 C1C1C1 C5CAEA C5CAEA C5CAEA EAF2DC EAF2DC EAF2DC EAF2DC ... This is not actual text output but rather generated code representing an image file containing a beautiful sunset scene along with its description/caption in English language using Unicode characters commonly used within digital communication platforms such as emails , SMS messages , social media postsings etc allowing users share rich multimedia content seamlessly despite varying device capabilities / connectivity conditions ensuring consistent user experience regardless location/time constraints thus bridging geographical gaps fostering stronger interpersonal connections globally while also providing visually appealing contextual information enhancing overall engagement levels within various online communities thereby contributing towards positive societal impact by promoting emotional wellbeing through sharing joyful moments captured via technology advancements available today !
My suggestion is, instead of using this bulk conversation in the summarize:

[
                { from: "user", content: "Who is the president of Gabon?" },
                { from: "assistant", content: "🇬 🇦  President of Gabon" },
                { from: "user", content: "Who is Julien Chaumond?" },
                { from: "assistant", content: "🧑 Julien Chaumond" },
                { from: "user", content: "what is 1 + 1?" },
                { from: "assistant", content: "🔢 Simple math operation" },
                { from: "user", content: "What are the latest news?" },
                { from: "assistant", content: "📰 Latest news" },
                { from: "user", content: "How to make a great cheesecake?" },
                { from: "assistant", content: "🍰 Cheesecake recipe" },
                { from: "user", content: "what is your favorite movie? do a short answer." },
                { from: "assistant", content: "🎥 Favorite movie" },
                { from: "user", content: "Explain the concept of artificial intelligence in one sentence" },
                { from: "assistant", content: "🤖 AI definition" },
                { from: "user", content: prompt },
        ]

It might be better if title get's generated after 2 or 3 user messages and use the actual conversation instead of this bulk conversation...
I can give give it a try and then submit a PR if that's fine?

[nsarrazin]

I think we want to generate the title right away based on the first message submitted only (otherwise it would look a bit weird in the UI), but if you want to try to improve the prompt using only the first user message, feel free!

You can ping me on a PR and I'll take a look. Make sure to specify which model you're using for the generation and provide some before/after examples as well 🤗

[ihubanov]

Got it! I'll just try to improve the summary prompt. However I still think this is not efficient. My best guess is in ChatGPT they are passing at least 2 or 3 user/assistant interactions to the model, so it is able to catch where the conversation is going. Most people start conversations with just Hello or Hi or something just to engage the model and the actual goal for the conversation comes after.

[nsarrazin]

If I start a new conversation with ChatGPT with Hi! it generates a title right away based on the first message, for example in my case Summarize Request and Respond, so it seems like they do the same as we do 😛 (which agreed, could miss some context, but is better than having no title imo)

[ihubanov]

Wow... indeed, I just tested as well... Well it seems I've not been using it for a while since I started using Mixtral large :D

[ihubanov]

@nsarrazin I'm ready to submit a PR, but can you help me with steps to do so? Maybe the easiest will be to fork the repo and then I will be able to provide you with before and after examples.
In few words I'm using slightly different approach. First I'm instructing the model what to do in the user prompt instead of system, as I've noticed that system prompt does not have the same priority as the user prompt. This might be because some models don't even work with system prompt, or the model is using it only for it's "metadata" or "model information". And second I'm asking the model to respond in JSON format, which in my experience so far works quite well even with dumber 7B models. I've tested if so far with "mistralai/Mistral-7B-Instruct-v0.2" and "Nexusflow/Starling-LM-7B-beta". Both I'l running locally on my "Jetson AGX Orin 64GB" device.
While playing with this I've found something odd... not sure if it is a bug or a feature, but when "TASK_MODEL" is not specified the first model in the list is used for the chat summary/title generation. This is fine as long as you always have the first model up and running, but in my case it adds some struggle because the first model is not always loaded in my Jetson device. Maybe a better approach will be if no specific "TASK_MODEL" is defined to use the one that is currently active.
One more thing to note, this JSON approach I'm using could be suitable for adding functions capability, even to the small parameter models.
Let me know the best approach to pass to you my code modifications, so you can take a look and see what I mean.