-
Notifications
You must be signed in to change notification settings - Fork 204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Literary illustrator bot Done by CTRL_Pix-A-Thon #318
Closed
Closed
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
# StableDiffusion | ||
📌 Overview | ||
|
||
Stable Diffusion is a platform that lets you create realistic images from text prompts using latent text-to-image diffusion models. Our AI-powered platform empowers you to generate stunning, photo-realistic images based on any text input. | ||
Our system uses only one agent, the AI Model Agent, for handling image description requests and interfacing with the Hugging Face API. | ||
|
||
# Instructions to run the project: | ||
Step 1 Account Setup:Create an account on Hugging Face to obtain your API key. | ||
|
||
Step 2 Imgur Integration: Create an account on Imgur to obtain your client key. | ||
|
||
Step 3 Configuration:Replace the placeholders in the code with your API key and client key as instructed. | ||
|
||
Step 4 Generate Illustration:Use the provided function to input the desired literature, paragraph, or note for illustration. | ||
|
||
Step 5 Accessing the Image:Click on the link generated after the illustration process to access your image. | ||
|
||
|
||
# Use case Examples: | ||
1.Artistic Rendering: Creating artistic images from textual descriptions or prompts. | ||
|
||
2.Poem Visualization: Converting poems into visually appealing images. | ||
|
||
3.Content Creation: Generating images for social media posts, blogs, and articles. | ||
|
||
4.Educational Material: Creating visual aids for educational content. | ||
|
||
5.Storytelling: Enhancing storytelling by visualizing scenes or characters from stories. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. empty file? |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,91 @@ | ||
import requests | ||
from pydantic import BaseModel, Field | ||
from ai_engine import UAgentResponse, UAgentResponseType | ||
from uagents import Agent, Context, Protocol, Model | ||
|
||
# Define the API keys and URLs | ||
BERT_API_URL = "https://api-inference.huggingface.co/models/google-bert/bert-base-uncased" | ||
BERT_HEADERS = {"Authorization": "Bearer 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"} | ||
|
||
STADIF_API_KEY = "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" | ||
Maithiliii marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# Define the models for prompt generation and image generation | ||
class PromptGenModel(Model): | ||
paragraph: str = Field(description="Given a paragraph prompt describing a scene or concept, your task is to identify and extract useful and relevant information or tokens that can be utilized for image generation. Focus on key details such as objects, settings, actions, emotions, and any other elements that contribute to the visual representation of the described scene. Your output should provide a structured representation of the extracted information, facilitating the generation of coherent and visually compelling images. Please ensure that the extracted tokens accurately reflect the content and context of the paragraph prompt. Ignore all other irrelevant information") | ||
|
||
class ImgGenModel(Model): | ||
symbol: str = Field(description="Enter prompt to generate high quality, realistic images with dynamic lighting that envoke strong emotions in a visually appealing and interesting way") | ||
|
||
# Function to query BERT model for relevant information extraction | ||
def query_bert(paragraph: str): | ||
try: | ||
response = requests.post(BERT_API_URL, headers=BERT_HEADERS, json={"inputs": paragraph}) | ||
response.raise_for_status() # Raise exception for HTTP errors | ||
return response.json() | ||
except requests.exceptions.RequestException as e: | ||
raise RuntimeError(f"Error from Hugging Face API: {e}") | ||
|
||
# Function to query image generation model | ||
def query_img_gen(symbol: str): | ||
API_URL = "https://api-inference.huggingface.co/models/runwayml/stable-diffusion-v1-5" | ||
headers = {"Authorization": f"Bearer {STADIF_API_KEY}"} | ||
try: | ||
response = requests.post(API_URL, headers=headers, json={"inputs": symbol}) | ||
response.raise_for_status() # Raise exception for HTTP errors | ||
return response.content | ||
except requests.exceptions.RequestException as e: | ||
raise RuntimeError(f"Error from Hugging Face API: {e}") | ||
|
||
# Function to upload image to Imgur | ||
def upload_image_to_imgur(image_bytes): | ||
headers = { | ||
"Authorization": "Client-ID ff9ffff4455eadf" # Replace "your-client-id" with your actual Imgur client ID | ||
} | ||
files = { | ||
'image': ('generated_image.jpg', image_bytes, 'image/jpeg') | ||
} | ||
try: | ||
response = requests.post("https://api.imgur.com/3/image", headers=headers, files=files) | ||
response.raise_for_status() # Raise exception for HTTP errors | ||
return response.json()["data"]["link"] | ||
except requests.exceptions.RequestException as e: | ||
raise RuntimeError(f"Error uploading image to Imgur: {e}") | ||
|
||
# Define protocols for handling prompt generation and image generation | ||
PromptGen_protocol = Protocol("PromptGenProtocol") | ||
ImgGen_protocol = Protocol("ImgGenProtocol") | ||
|
||
# Function to handle prompt generation | ||
@PromptGen_protocol.on_message(model=PromptGenModel, replies={ImgGenModel}) | ||
def handle_promptgen(ctx: Context, sender: str, msg: PromptGenModel): | ||
ctx.logger.info(f"Received message from {sender}, session: {ctx.session}") | ||
try: | ||
relevant_info = query_bert(msg.paragraph) | ||
ctx.logger.info(f"Relevant information extracted: {relevant_info}") | ||
ctx.send(sender, ImgGenModel(symbol=relevant_info)) | ||
except Exception as ex: | ||
ctx.logger.error(f"Error occurred during relevant information extraction: {ex}") | ||
ctx.send(sender, UAgentResponse(message=str(ex), type=UAgentResponseType.ERROR)) | ||
|
||
# Function to handle image generation | ||
@ImgGen_protocol.on_message(model=ImgGenModel, replies={UAgentResponse}) | ||
def handle_imggen(ctx: Context, sender: str, msg: ImgGenModel): | ||
ctx.logger.info(f"Received message from {sender}, session: {ctx.session}") | ||
try: | ||
image_bytes = query_img_gen(msg.symbol) | ||
image_link = upload_image_to_imgur(image_bytes) | ||
ctx.logger.info(f"Image link from Imgur: {image_link}") | ||
ctx.send(sender, UAgentResponse(message=image_link, type=UAgentResponseType.FINAL)) | ||
except Exception as ex: | ||
ctx.logger.error(f"Error occurred during image generation: {ex}") | ||
ctx.send(sender, UAgentResponse(message=str(ex), type=UAgentResponseType.ERROR)) | ||
|
||
# Initialize the agent | ||
agent = Agent() | ||
|
||
# Include ImgGen_protocol | ||
agent.include(ImgGen_protocol, publish_manifest=True) | ||
|
||
# Include PromptGen_protocol if ImgGen_protocol is successfully included | ||
if ImgGen_protocol in agent.protocols: | ||
agent.include(PromptGen_protocol, publish_manifest=True) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. please remove empty files |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really nice project, could you add a project.json file?
see https://github.com/fetchai/uAgents/blob/main/integrations/CONTRIBUTING.md, thanks!