Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jasper #5

Open
wants to merge 106 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
0a66f6c
updated auth class
CccrizzZ Oct 23, 2023
c292987
updated auth function
CccrizzZ Oct 24, 2023
dce1489
updated permission and auth
CccrizzZ Oct 25, 2023
ea0d1e0
added admin controller
CccrizzZ Oct 26, 2023
0f4bf05
updated utilities and controllers
CccrizzZ Oct 27, 2023
af245f6
updated authentication method
CccrizzZ Nov 1, 2023
6126184
updated authentication flow
CccrizzZ Nov 2, 2023
463d8f1
fixed controllers
CccrizzZ Nov 4, 2023
d07316b
added admin only auth route
CccrizzZ Nov 8, 2023
733a496
added image uploading function
CccrizzZ Nov 12, 2023
86e6329
fixed image and inventory service
CccrizzZ Nov 14, 2023
590cd9e
updated views
CccrizzZ Nov 17, 2023
8cf7cfc
fixed minor issue
CccrizzZ Nov 18, 2023
47c9f2b
fix bug, added utility
CccrizzZ Nov 19, 2023
4387326
fixed routes, added invitation code mech
CccrizzZ Nov 21, 2023
28377ff
Add a field to QA record obj, working on paged query
CccrizzZ Nov 22, 2023
e118649
added docker stuff
CccrizzZ Nov 22, 2023
bbd9e71
prepare for container deployment
CccrizzZ Nov 23, 2023
6f838dd
prepare for deployment
CccrizzZ Nov 24, 2023
bceaee1
Create docker-image.yml
CccrizzZ Nov 26, 2023
4ee84f4
solved cookie problem, working on docker cert problem
CccrizzZ Nov 29, 2023
62728c2
Add or update the Azure App Service build and deployment workflow config
CccrizzZ Nov 30, 2023
6abbf76
deployment
CccrizzZ Nov 30, 2023
38ae1de
updated requirements.txt
CccrizzZ Nov 30, 2023
d24c472
fix cookie not delete
CccrizzZ Dec 1, 2023
dcbdc10
update admin route
CccrizzZ Dec 1, 2023
1fe16c4
updated routes
CccrizzZ Dec 5, 2023
f7126c8
fixed timestamp problem
CccrizzZ Dec 7, 2023
99765b3
updated admin controller
CccrizzZ Dec 7, 2023
667be2a
working on admin controller
CccrizzZ Dec 9, 2023
3f6704d
fixed admin route
CccrizzZ Dec 13, 2023
a56bb4f
fixed date problem
CccrizzZ Dec 13, 2023
205102a
added QARecords function
CccrizzZ Dec 14, 2023
8c7bf83
fixed previous logic, added admin stuff
CccrizzZ Dec 16, 2023
7eb00f3
added admin route
CccrizzZ Dec 17, 2023
8967059
Update settings.py
CccrizzZ Dec 18, 2023
216e12e
worked on admin controller
CccrizzZ Dec 22, 2023
0e22f66
update admin controller
CccrizzZ Dec 22, 2023
d4af3d9
tried to fix image controller
CccrizzZ Dec 31, 2023
2dc6874
fixed time and problem record route
CccrizzZ Jan 4, 2024
985af51
working on Q&A Record filters
CccrizzZ Jan 5, 2024
63cfac5
fixed image controller
CccrizzZ Jan 10, 2024
5710c48
fixed image controller
CccrizzZ Jan 11, 2024
00f53d7
tried to fix time range problem
CccrizzZ Jan 14, 2024
d4d4e8a
added chat gpt api integration
CccrizzZ Jan 17, 2024
169a7fd
working on scrape functions
CccrizzZ Jan 18, 2024
92ea106
worked on scraping logic
CccrizzZ Jan 21, 2024
a453c23
fixed amazon scrape code
CccrizzZ Jan 24, 2024
213320c
migrated QA records
CccrizzZ Jan 25, 2024
43a41dd
remove example image, fixed imageController
CccrizzZ Jan 27, 2024
ad3ac2b
fixed time problem, added csv logic
CccrizzZ Jan 31, 2024
0cce6cd
migrated records, added inventory routes
CccrizzZ Feb 1, 2024
d81c013
updated filters and utils
CccrizzZ Feb 4, 2024
363dd50
fixed inventory filter
CccrizzZ Feb 9, 2024
e694460
fixed scrape utils and filter utils
CccrizzZ Feb 9, 2024
1b64b31
fixed older inventory logics
CccrizzZ Feb 10, 2024
c5a0cab
fixed bugs and time format
CccrizzZ Feb 11, 2024
d09f74a
added image uploading method, fixed bugs
CccrizzZ Feb 14, 2024
aedac46
added image compression method
CccrizzZ Feb 15, 2024
f2cae78
fixed login problem, worked on chat gpt api
CccrizzZ Feb 18, 2024
3f4bca2
fixed date filter
CccrizzZ Feb 21, 2024
5132c4c
updated filters and fixed bugs
CccrizzZ Feb 21, 2024
ded0933
instock update and delete function
CccrizzZ Feb 23, 2024
06f2541
worked on instock routes
CccrizzZ Feb 23, 2024
4189634
fixed filter
CccrizzZ Feb 23, 2024
142d55a
fixed bug, added qa overview route
CccrizzZ Feb 28, 2024
26b9618
added QA inventory method
CccrizzZ Feb 28, 2024
29e6fa8
fixed bugs, updated packages
CccrizzZ Mar 8, 2024
d958fef
updated CSV stuff
CccrizzZ Mar 10, 2024
9662686
worked on auction routes
CccrizzZ Mar 12, 2024
61db112
updated auction function
CccrizzZ Mar 14, 2024
e5dcd8c
updated csv logic
CccrizzZ Mar 15, 2024
8e8b4e5
updated auction record logic
CccrizzZ Mar 16, 2024
058f197
updated auction logic
CccrizzZ Mar 21, 2024
665042c
fixed csv image problem
CccrizzZ Mar 22, 2024
d508b02
updated auction logic
CccrizzZ Mar 23, 2024
87ea12c
worked on auction manager
CccrizzZ Mar 27, 2024
b862f16
fixing auction logic
CccrizzZ Mar 30, 2024
8799c92
prepare for deploy
CccrizzZ Apr 9, 2024
f61e37f
added throttle setting
CccrizzZ Apr 9, 2024
434f785
added restock route, fixed bugs
CccrizzZ Apr 10, 2024
ac9da96
fix bugs
CccrizzZ Apr 11, 2024
729b02c
fixed update qa inventory
CccrizzZ Apr 12, 2024
1a2fe77
minor bug fix
CccrizzZ Apr 16, 2024
02b08b4
fixed remaining
CccrizzZ Apr 19, 2024
b82dd55
update chat gpt route
CccrizzZ May 10, 2024
4b87247
added image rotation route
CccrizzZ May 11, 2024
7ba4b31
fixed rotation
CccrizzZ May 14, 2024
0b5165d
worked on firebase auth
CccrizzZ May 16, 2024
f84213d
fixed admin firebase authentication
CccrizzZ May 17, 2024
36a34bc
fixed auth methods
CccrizzZ May 18, 2024
4ce39c7
added recorded filter for QA table
CccrizzZ May 21, 2024
0abc438
fixed auction logic
CccrizzZ May 22, 2024
109e6fb
fixed bugs, fixing user info update function
CccrizzZ May 23, 2024
858652f
fixed user update own info
CccrizzZ May 24, 2024
826b314
fixed minor bug
CccrizzZ May 30, 2024
cbf9353
fixed bug, tried to add selenium
CccrizzZ May 31, 2024
b7c11f5
fixed minor bugs
CccrizzZ Jun 12, 2024
c55461c
implemented proxy when requesting Amazon pages
CccrizzZ Jun 13, 2024
40ac6a5
added daily bar chart in qa user info
CccrizzZ Jun 14, 2024
6333d84
worked on auction controller
CccrizzZ Jun 18, 2024
6301fda
worked on remaining & audit work flow
CccrizzZ Jun 20, 2024
6447b8e
work on import to existing auction
CccrizzZ Jun 21, 2024
5e17583
fixed add items to existing auction logic, fixed bugs
CccrizzZ Jun 22, 2024
b2707cc
fixed create remaining record
CccrizzZ Jun 28, 2024
5d54c52
worked on admin settings
CccrizzZ Jun 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
*.pyc
*.pyo
*.mo
*.db
*.css.map
*.egg-info
*.sql.gz
.cache
.project
.idea
.pydevproject
.idea/workspace.xml
.DS_Store
.git/
.sass-cache
.vagrant/
__pycache__
dist
docs
env
logs
src/{{ project_name }}/settings/local.py
src/node_modules
web/media
web/static/CACHE
stats
Dockerfile
# .env
18 changes: 18 additions & 0 deletions .github/workflows/docker-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Docker Image CI

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:

build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Build the Docker image
run: docker build . --file Dockerfile --tag cccrizzz/ccpd-django-service:latest
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ wheels/
*.egg-info/
.installed.cfg
*.egg
staticfiles/
static/

# PyInstaller
# Usually these files are written by a python script from a template
Expand Down
4 changes: 3 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,7 @@
"[python]": {
"editor.defaultFormatter": "ms-python.python"
},
"python.formatting.provider": "none"
"python.formatting.provider": "none",
"python.analysis.typeCheckingMode": "off",
"python.analysis.autoImportCompletions": true
}
99 changes: 81 additions & 18 deletions CCPDController/authentication.py
Original file line number Diff line number Diff line change
@@ -1,34 +1,97 @@
import jwt
from rest_framework.authentication import get_authorization_header, TokenAuthentication
from firebase_admin import auth
from rest_framework.authentication import CSRFCheck, TokenAuthentication, BaseAuthentication
from bson.objectid import ObjectId
from django.conf import settings
from rest_framework.exceptions import AuthenticationFailed
from rest_framework.exceptions import AuthenticationFailed, PermissionDenied
from CCPDController.utils import get_db_client

# pymongo
db = get_db_client()
collection = db['User']

# check for csrf token in request
def enforce_csrf(request):
check = CSRFCheck(request)
check.process_request(request)
reason = check.process_view(request, None, (), {})
if reason:
raise PermissionDenied('CSRF Failed: %s' % reason)

# customized authentication class used in settings
class JWTAuthentication(TokenAuthentication):
def verifyToken(self, token):
class JWTAuthentication(TokenAuthentication):
# run query against database to verify user info by querying user id
def authenticate_credentials(self, id):
# if id cannot convert into ObjectId, throw error
try:
# decode
payload = jwt.decode(token, settings.SECRET_KEY)
uid = ObjectId(payload['id'])

print(token)
uid = ObjectId(id)
except:
raise AuthenticationFailed('Invalid ID')

# get only id status and role
user = collection.find_one({'_id': uid}, {'userActive': 1, 'role': 1})

# check user status
if not user:
raise AuthenticationFailed('User Not Found')
if user['userActive'] == False:
raise AuthenticationFailed('User Inactive')

# return type have to be tuple
return (user, user['role'])

# called everytime when accessing restricted router
def authenticate(self, request):
try:
# check for http-only cookies
raw_token = request.COOKIES.get('token') or None
if not raw_token:
raise AuthenticationFailed('No token provided')

# query mongo db for user
user = collection.find_one({'_id': uid})

# expect errors
except jwt.DecodeError:
# decode jwt and retrive user id
payload = jwt.decode(raw_token, settings.JWT_SECRET_KEY, algorithms='HS256')

except jwt.DecodeError or UnicodeError:
raise AuthenticationFailed('Invalid token')
except jwt.ExpiredSignatureError:
raise AuthenticationFailed('Token has expired')

# if not activate throw error
if not user['userActive']:
raise AuthenticationFailed('User inactive or deleted')
return payload
# TODO
# check the reason why csrf token cannot be fetch on logout
# enforce_csrf(request)
return self.authenticate_credentials(payload['id'])

class FirebaseAuthentication(BaseAuthentication):
def authenticate(self, request):
# read token in header
auth_header = request.META.get("HTTP_AUTHORIZATION")
if not auth_header:
print("No auth token provided")
raise PermissionDenied("No auth token provided")

# decode token
id_token = auth_header.split(" ").pop()
decoded_token = None
try:
decoded_token = auth.verify_id_token(id_token)
except Exception as e:
raise PermissionDenied("Invalid Auth Token")

if not id_token or not decoded_token:
return None

# pull user form mongo
user = collection.find_one(
{'email': decoded_token.get('email').lower()},
{'_id': 0, 'userActive': 1, 'role': 1}
)

# check user status
if not user:
raise PermissionDenied('User Not Found')
if user['userActive'] == False:
raise PermissionDenied('User Inactive')
uid = decoded_token.get("uid")
if not uid:
raise PermissionDenied("Firebase Server Error")
return (user, user['role'])
159 changes: 159 additions & 0 deletions CCPDController/chat_gpt_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# send
import os
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv()

# chat gpt's natural language processing model engine's name & key
model_engine = "gpt-3.5-turbo-instruct"
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))

# short description (lead on auction flex max 40 char )
def generate_title(title, template) -> str:
if template == '':
template = "You are an Auctioneer, based on the information, create a short product title. The character limit for the product title is 50 characters. " + title + "."
else:
template = template + ' ' + title

res = client.chat.completions.create(
messages=[
{
"role": "user",
"content": template,
}
],
model="gpt-3.5-turbo",
max_tokens=40
)
return special_characters_convert_l(res.choices[0].message.content.strip())

# full description
def generate_description(condition, comment, description, template) -> str:
if template == '':
template = "Please generate a product description in the format '[Item Condition] - [Item information]', The character limit for Item information is 250 characters."

res = client.chat.completions.create(
messages=[
{ "role": "user", "content": f"Item Condition: {condition}, {comment}." },
{ "role": "user", "content": f"Item information: {description}." },
{
"role": "user",
"content": template
},
],
model="gpt-3.5-turbo",
max_tokens=100,
temperature=0.4
)

desc = res.choices[0].message.content.strip()
desc = desc.replace("SAME", "")
desc = desc.replace("SCANNED", "")
desc = desc.replace("ALL MAIN PARTS IN", "")
desc = special_characters_convert_d(desc)
return desc

def special_characters_convert_d(description):
try:
if description[0] == '"' and description[-1] == '"':
description = description.strip()
description = description[1:-1]
except Exception as e:
print(e)
description = description.replace("°"," Degree ")
description = description.replace("™","")
description = description.replace("®","")
description = description.replace("©","")
description = description.replace("w/ ","With ")
description = description.replace("w/","With ")
description = description.replace("W/ ","With ")
description = description.replace("W/","With ")
description = description.replace(",",",")
description = description.replace("“",'"')
description = description.replace("”",'"')
description = description.replace("’’",'"')
description = description.replace("’","'")
description = description.replace("‘","'")
description = description.replace("″",'"')
description = description.replace("【","[")
description = description.replace("】","]")
description = description.replace("×","x")
description = description.replace("–","-")
description = description.replace("℉"," Fahrenheit scale ")
# description = description.replace(" "," ")
description = description.replace("Φ"," ")
description = description.replace("’","'")
description = description.replace('Additional Condition" - "',"")
description = description.replace("Additional Condition - ","")
description = description.replace("Additional Condition: ","")
description = description.replace("[Additional Condition] - ","")
description = description.replace("Item Info: ","")
description = description.replace("é","e")
description = description.replace("(","(")
description = description.replace(")",")")
description = description.replace("≤"," less than or equal than ")
description = description.replace("ӧ","o")
description = description.replace("ö","o")
description = description.replace("î","i")
description = description.replace('"Additional Condition" - ',"")
description = description.replace("150 Byte: ","")
description = description.replace("150 byte: ","")
description = description.replace("150 Byte - ","")
description = description.replace("150 byte - ","")
description = description.replace("[Additional Condition] ","")
description = description.replace("ñ","n")
description = description.replace("New - ","")
description = description.replace("[Item information]","")
description = description.replace("Item information: ","")
description = description.replace("²","^2")
description = description.replace("³","^3")
description = description.replace("à","a")
return description

def special_characters_convert_l(lead):
try:
if lead[0] == '"' and lead[-1] == '"':
lead = lead.strip()
lead = lead[1:-1]
if "(" in lead:
index1 = lead.find("(")
index2 = lead.find(")")
leadH = lead[:index1]
leadB = lead[index2+1:]
lead = leadH + leadB
if "," in lead:
index3 = lead.find(",")
lead = lead[:index3]
except Exception as e:
print(e)
lead = lead.replace("°","Degree")
lead = lead.replace("™","")
lead = lead.replace("®","")
lead = lead.replace("©","")
lead = lead.replace("w/ ","With ")
lead = lead.replace("w/","With ")
lead = lead.replace(",",",")
lead = lead.replace("“",'"')
lead = lead.replace("”",'"')
lead = lead.replace("″",'"')
lead = lead.replace("’’",'"')
lead = lead.replace("’","'")
lead = lead.replace("‘","'")
lead = lead.replace("【","[")
lead = lead.replace("】","]")
lead = lead.replace("×","x")
lead = lead.replace("–","-")
lead = lead.replace("℉"," Fahrenheit scale")
lead = lead.replace(" "," ")
lead = lead.replace("Φ"," ")
lead = lead.replace("î","i")
lead = lead.replace("é","e")
lead = lead.replace("(","(")
lead = lead.replace(")",")")
lead = lead.replace("≤"," less than or equal than ")
lead = lead.replace("ӧ","o")
lead = lead.replace("ö","o")
lead = lead.replace("ñ","n")
lead = lead.replace("²"," Squared")
lead = lead.replace("à","a")
return lead
50 changes: 50 additions & 0 deletions CCPDController/permissions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
from rest_framework import permissions
from CCPDController.utils import get_db_client, getIsWorkingHourEST

# pymongo
db = get_db_client()
collection = db['User']

# user object will be pass into here from authentication
# request.auth = ROLE
# request.user = {
# '_id': ObjectId(xxx),
# 'userActive': xxx,
# 'role': xxx
# }

# QA personal permission
class IsQAPermission(permissions.BasePermission):
message = 'Permission Denied, QAPersonal Only!'
def has_permission(self, request, view):
# if not in work hours, return false
if getIsWorkingHourEST() == False:
return False

# mongo db query
# grant if user is qa personal and user is active
if request.auth == 'QAPersonal' and bool(request.user['userActive']) == True:
return True

# admin permission
class IsAdminPermission(permissions.BasePermission):
message = 'Permission Denied, Admin Only!'
def has_permission(self, request, view):
if request.auth == 'Admin' and bool(request.user['userActive']) == True :
return True
elif request.auth == 'Super Admin':
return True

class IsSuperAdminPermission(permissions.BasePermission):
message = 'Permission Denied, Super Admin Only!'
def has_permission(self, request, view):
if request.auth == 'Super Admin':
return True

# user blocked by IP black list
# class BlockedPermission(permissions.BasePermission):
# message = 'You Are Blocked From Our Service'

# def has_permission(self, request, view):
# ipAddress = request.META['REMOTE_ADDR']

Loading