You want to improve your search engine algorithms for image search. What are the most effective techniques?

Content based image retrieval(CBIR) techniques can be used for comparing images based on the features. CBIR uses global and local features for image comparison. Global features can be defined by color and histogram technique can be used to compare the color spectrum. The local features are defined at pixel level e.g. points, lines and edges in the images. The CBIR techniques can be used for using the following ways 1) Remove noise from the images so that features are effected by the noise. 2)Align images by performing affine and scale transformation 3) Use scale and rotation invariant algorithm for feature extraction. So that, features are not dependent on image size & alignment. 4) Use appropriate algorithm for comparing features.

Imagine you have a big box of toys. To find the toy you want, you can look at each toy one by one and compare it to the toy you're looking for: 1. If the toys look the same, you find the one you want. 2. If they don't look the same, you keep looking until you find the right one. This is how content-based image retrieval (CBIR) works. The computer takes the image you want to find and looks at its colors, shapes, and patterns. It then compares these features to the features of other images present inside the data store. It finds images that look similar to the image you want to find and shows them to you.

Start by considering if the images are represented in the most appropriate way. Classical CV-based features based on color histograms, edge detectors or similar algorithms could be replaced by modern DL models. Post processing steps and comparison methods should be updated to correctly use the models' output - for example by applying dimensionality reduction after the initial feature extraction step.

Imagine you have a big box of toys, each with a label that tells you what it is, like "red truck" or "blue train." If you ask for a "yellow airplane," you will look at the labels to find toys that match your words. That's what MBIR does with pictures on a computer. Computer takes the words you type in the search box and looks at the information that is written about each image in its image store. 1. It then finds images that have the same words or similar words in their information. 2. It shows you these images because they are related to the words you typed. MBIR is a useful tool for finding images that are related but it may not always find the right. i.e, if you search for "cat," MBIR might show you images of "cat food" or "lion".

Review if modern CV approaches can provide new, useful metadata. For example, DL-based object detectors or classifiers can provide metadata per image or frame to extend the filtering capabilities. Similarly, combining the initial step, as in #1, with a language model, can allow you to query the dataset with simple text prompts.

FBIR learns by paying attention to what we do when we use it. 1. Clicking: When we click on a picture, FBIR knows that we like it. It adds that picture to our list of favorites and tries to find more pictures like it. 2. Looking: If we spend a long time looking at a picture, FBIR knows that we're really interested in it. It makes a note of that and tries to find more pictures that are similar. 3. Describing: When we use words to describe a picture, FBIR learns what kinds of pictures we like. For example, if we say that we like "cute puppies" it will try to find more pictures of cute animals for us. The next time we come, it still remembers our past data, and before we even ask, it will show us the image/video. i.e IG Reels, YT shorts

When we search for pictures online, we want the best ones to show up first. Ranking-based image retrieval (RBIR) is used that help us find the pictures with the most "likes" or the ones that other people say are good. RBIR also considers things like how old a picture is (freshness), how many different kinds of pictures there are (diversity), and whether the pictures are clear and easy to understand (quality). Example - The images that appear on Google Search

When building your system, consider who will use it and how. Will a text-prompts be easier to use or a image-based search? Is metadata required, and what kind? What volume of image and queries is expected? Once defining the system, review modern CV approaches to support metadata generation, combinations of language and vision models, and comparison options.