Woman taking a picture with her cell phone Woman taking a picture with her cell phone Using visual search, a person could snap a photo of a plant and find out more information about this plant species. (Photo: Chervonaya / Shutterstock)

How Google and others are taking the typing out of searching

Photo-based visual search can help you find out more information about everyday objects you come across.

Google made waves recently when it announced that it is working on technology that could theoretically look at a photo of food on Instagram and let you know how many calories are in that item.

The technology is known as "visual search." More often than not, when you want to search on the Internet, you type in a few keywords ("how to change a tire"), hit search ... and voila! A wealth of unlimited information (some useful, some not) appears on your screen.

But what if you don't know what to type? Let's say you're taking a hike and see a beautiful plant on the side of the path. You're curious to know what it is, but you have no clue what to type into Google. ("Super cool green plant thingy" just won't do the trick.)

That's where visual search comes in. Think of it as facial recognition technology without the faces. Snap a photo of the item in question, and artificial intelligence does the rest, taking the guesswork and confusion out of the equation.

JustVisual, an Israel-based company formerly known as Superfish, is at the forefront of the visual search revolution and told From The Grapevine that it has a laundry list of things it's working on. The scientists at their R&D lab are toying with all sorts of potential uses for the technology: You could find a photo of a beach, and the algorithm can find you hotel rooms that have the same view. You could take a selfie of yourself, and Superfish technology could show you other people who look like you. Instagram a dinner and instantly find the calorie count along with recipes for preparing that dish.

Google spokesman Jason Freidenfelds says its similar product, codenamed Im2Calories, is only in the research stage and that there are "no actual product plans at this stage."

JustVisual has unveiled one tool employing visual search that consumers can already get their hands on: LikeThat Pets. Using the app, formerly known as PetMatch, you can take a picture of a dog at the park, and JustVisual's servers can not only tell you what breed of dog it is, but also the nearest adoption agency where the breed is available.

The real magic is that people can take these pictures in a real-world environment. Advanced visual search technology can weed through bad lighting, odd angles, imperfect pictures and – down to the strands of hair on the back of the dog – give you detailed information about that breed. In total, their computer servers slice the photo 200 different ways in just milliseconds.

Now that sounds like a magic trick.


Photos and SlideshowsPhotos and Slideshows

Related Topics: Apps