HeyBagh

Find and Discover Visually Similar Content - Images

---

Technicals

• Feature Extraction / Image embeddings::
  • Pretrained ResNet-18, to extract features from images.
  • CNN adept at capturing the semantic meaning and visual characteristics of an image.
  • Vector of size 1000
• Vector Database:
  • Milvus Benchmark
  • Milvus utilizes advanced indexing techniques and optimizations to perform blazing-fast searches for similar vectors.
  • Most of the vector index types supported by Milvus use approximate nearest neighbors search (ANNS) algorithms.
  • Well-structured documentation with Helpful practical examples.
  • configuration used for HeyBagh
    • im-memory index
    • Similarity Metric Type: COSINE
    • Index Type: IVF_FLAT - Quantization-based index, High-speed query and Higher Recall
    • Top_k: 4
• User Interface:
  • Interface built with Gradio.
  • This interface allows users to easily upload images or provide image URLs and visually explore the retrieved results.

Objective:

Build a image search engine using deep learning techniques to find visually similar images.

Plan:

• Image Data - We can use Caltech-101, for starters.
• DB - Store images and extracted features/embeddings.
• UI - Design UI, we can start with Gradio.

TODO:

• Data Pre-processing

  • resize images to a uniform size
  • augmentation??

• Model Selection and Feature Extraction

• Benchmark all options below for SPEED and ACCURACY

  • CNN
  • Siamese Networks?
  • Triplet Networks
  • Autoencoders
  • CLIP

• Indexing

  • Select Vector DB
  • to efficiently store and retrieve feature vectors associated with each image
    • Approximate Nearest Neighbors (ANN) algorithms

• Query Processing

  • Use same pre-processing technique as used in indexing

• Deployment

• Diagrams

  • Flow of the application