Some of the more advanced video indexing technology is capable of indexing by images in the video, including characters and actors, using a form of facial recognition software:

Elsewhere, VideoSurf Inc. is analyzing the actual visual content of videos using technology known as “computer vision algorithms,” which produces more relevant search results, says Lior Delgo, the company’s chief executive. Computer vision is the science of programming computers to process and analyze images and video.

For example, VideoSurf’s technology can identify characters within search results. A search for the television show “Lost” brings up results for the show and also a thumbnail photo for each character. Clicking on the thumbnail of “Lost” actress Evangeline Lilly will bring up clips from the TV show and also other clips of Ms. Lilly, like her appearance on the “Late Show With David Letterman.” The site has indexed 10 million videos from 50 different online video sources.

Six weeks ago, Google premiered Gaudí GAudi, a video search engine that uses speech recognition to find words in clips and cue up the clips in SERPs to the uses of the keyword. So far, GAudi is only processing election videos, but the technology is being refined through this process and will hopefully be applied to other categories.

Mefeedia is also taking into account user ratings/votes in its ranking schemes. OVGuide relies on people in a different way—editors select the sites to be indexed and says they also have editorial control over ranking.

These two engines are meta search engines for video, pulling results from several different sources (licensed, possibly bootleg and UGC included)—and plenty of them don’t host original video content at all. Also in this category is the familiar name Blinkx, which indexes content from Hulu, CBS and Showtime. Blinkx also uses external clues, including links from Wikipedia or social media, to identify potential viral hits.

What advancements in video search are you most looking forward to?

Comments

Perhaps it was the hype of video search engine VideoSurf (I’m still awaiting my invitation on that btw) that got this kicked into gear. Those who have had a chance to use that seem to think quite highly of it. Basically, it uses visuals throughout the videos to identify relevant results. Whether or not a certain company will try to acquire VideoSurf down the road remains to be seen (but you can imagine what it would be like if they did…eh? eh?).

But that’s neither here nor there. Google’s dealing with audio right now and seems to be making strides, though it is still far from perfect. The transition (or translation, depending on how you want to look at it) from audio to text is a bit rocky as some excerpts from this Barack obama speech illustrate:

For example, "sex" is really "since" (I wonder how it would pick up Tiki Barber’s recent slip [may contain unsuitable language]). That’s why the project is in Google Labs though. It’s not an official release, and it will likely be improved upon. Frederic Lardinois Read Write Web suggests some additional improvements:

It would be extremely useful, for example, to be able to sort videos by date. Right now, they are always sorted by relevance, though in this fast-paced political season, it is often more important to know when somebody last used a certain phrase.

Also, while the variation on the YouTube video player Google uses here is well suited for this task, you cannot embed the videos on your own site. All you get is a link to the video on the Audi Indexing site.

The nice part about the interface is that you can jump directly to certain parts of the video based on the search results. As you can see in the above image, there are play buttons next to each one.

Despite the current kinks in the project, it is evident that video search on the whole is on the way up in terms of usefulness. I just wonder how long it will be before video and image search produce just as relevant results as regular text search. It might be a while before it gets to that point, but the light at the end of the tunnel seems a little closer these days.

Update: I have now received my invitation for VideoSurf. I’ll have to do some experimenting with it.