WebProNews » Semantic

Yahoo Geeks Out Over Semantic Search

WebProNews Staff — Wed, 23 Jul 2008 12:06:09 +0000

One word may have two different meanings, depending on the context of their usage. Yahoo hopes to find the key in search that enables its engine to understand the difference.

Does orange mean a fruit or a color? The word ‘bass’ could be a tasty fish or a rocking musical instrument. When it comes to the written word, people enjoy a vast advantage over a computer, as we easily pull the context out of the text to understand the meaning of the word.

Yahoo Search wants to enable such semantic recognition in their engine. Despite the much-discussed travails on the advertising side, the tech side of Yahoo Search seems a livelier place, especially with the recent unveiling of BOSS, their opening of the index to all developers.

Impediments lay in the roadway to better semantic search. Yahoo’s Peter Mika listed an assortment of them at Devx:

It is almost impossible to return search results that relate to the secondary sense of a term

Selling a Company on Enterprise Search

Mike Moran — Fri, 30 Nov 2007 21:08:36 +0000

I spoke at the Gilbane conference yesterday (you can download my slides on semantic search).

I fielded a number of interesting questions, but the one that made me stop and think for a minute was this one, "What’s the hardest objection you ever overcame to sell a company on enterprise search?"

I was glad the other panelist answered first, because it took me a moment to think of something. It was several years ago, but I remember it like yesterday. (OK, OK, I’m old. I remember it a lot better than yesterday.)

A very well-run company had an unusually sane way of making technology decisions. They developed real business cases based on improved revenue or reduced cost, and they actually tracked whether the projects delivered on the promise one and two years later. (It was not great for your career if they didn’t.) So, they wanted to subject their decision to license an intranet search engine to the same scrutiny.

They asked a simple question: "How can we prove that this search technology will increase revenue for our company?" It was a simple question, but the toughest one I’d ever gotten. There would be no productivity savings business cases here. No "If we assume that we save just six minutes each week for every knowledge worker…" No! This one needed a real business case. I promised the customer I’d put on my thinking cap and get back to him. (Since then, I’ve decided to wear my thinking cap all the time, despite all those snarky comments about my appearance.)

But how do you claim that anything you do to improve employees finding information will generate revenue? At first, I couldn’t think of anything. I mean, if the HR people save some time, would a nickel roll in under the door from a customer? What about the executives? Or the accounting department? I couldn’t come up with how I could stitch together revenue stories for every different usage of search across an entire enterprise full of employees on the intranet.

So I took a different approach. I started to ask myself which employees really did have an impact on revenue. Of course! It’s the sales force. But how could I justify revenue based on making sales people more productive? I needed to go back to the customer for more information.

So, I asked my customer how they had justified that spiffy sales force automation package they used. And their spanking-new customer realtionship management system. At first, he was puzzled about why I wanted to know, but he showed me the studies they’d done that showed how their sales people spent their time (18% was finding information) and the revenue impact of saving time for their sales people.

That was all I needed. I put together a business case that showed that the extra revenue stemming from the increased productivity of sales people more than paid for the search facility for the entire company. In fact, making information available to busy sales people on smart phones (where search is one of the few effective interfaces for a small screen) was enough to justify the search investment all by itself.

It was a tough question, but it had a good answer. You can’t always justify technology investments based on revenue, and search is one of the toughest, but if you think about it a little bit, sometimes you can pull it off.

Comments

Tag:

Trying Semantic Search Yourself

Mike Moran — Tue, 30 Oct 2007 19:33:57 +0000

Most of you know that my job focuses on IBM’s OmniFind enterprise search and text analytics products. And I’ve written before about semantic search—I’ve even written about what semantic search isn’t. I keep talking about it because semantic search is probably the easiest to understand application of text analytics. But maybe you need to stop hearing about it and actually see it for yourself.

If you’ve never seen semantic search in action, check out a free facility to search your e-mail—OmniFind Personal E-Mail Search. Maybe you already use a desktop search product, but using this one will give you a clue as to how semantic search is better.

Try to find the PowerPoint file that Rob sent you with your desktop search engine. Then type "PPT from Rob" into our e-mail search and see the difference.

Can’t find a phone number that you know someone e-mailed to you? Try " Pat phone" and find all the phone numbers for people named "Pat."

No matter how good your desktop search engine is, it finds only keywords. So it will find that PowerPoint file only if the e-mail has the words "PPT from Rob" and it will find the phone number only if the e-mail literally says "Pat phone" in it. But usually they don’t. Usually they don’t have the words "PPT" or "phone" in them. They just have the PPT attached. And they say "Call Pat at 332-456-5624."

I could talk about this until the cows come home. (I’m not sure when that is, but trust me that it’s a long time from now.) It’s better that you try it yourself. Go ahead. It’s free. Download it and try it out—and let me know what you think.

Comments

Tag:

Calacanis: Web 3.0 is Whatever I Say it is

Mathew Ingram — Thu, 04 Oct 2007 19:41:47 +0000

Humpty Dumpty: “When I use a word, it means just what I choose it to mean – neither more nor less.”

Alice: “The question is, whether you can make words mean so many different things.”

You have to hand it to Jason Calacanis, the diminutive Web entrepreneur behind Mahalo, for completely ignoring all the ink and electrons that have been spilled writing about the concept of Web 3.0 — including conversations like the one I had with Sir Tim Berners-Lee, the guy who invented the Web — and just coming up with his own definition. Not only that, he has the audacity to call it the “official” definition. Official according to whom? Why, to Jason, of course.

Not surprisingly, as Fred Wilson points out in his post, Jason’s definition is also effectively a thumbnail description of Mahalo, the people-powered search/directory service he is trying to build. Web 3.0, he says, is:

“the creation of high-quality content and services produced by gifted individuals using Web 2.0 technology as an enabling platform.”

That’s funny, because every time I’ve heard anyone who actually knows anything describe it, they use terms like “semantic Web,” and talk about adapting the way the Web is built so that information can be aggregated and linked in different ways automatically, as Josh Kopelman describes here. But that kind of definition wouldn’t suit Jason’s purposes, so in effect it doesn’t exist. I think I like the definition Jemima Kiss came up with better.

Comments

SiloMatic – Latent Semantic Indexing

Bill Hartzer — Mon, 10 Sep 2007 21:21:09 +0000

The days of keyword stuffing, single phrase optimization and concentrating only on incoming links to gain traffic are slowly being phased out as a more holistic approach to judging website content comes online. This new concept has many webmasters hopping, and it should. Latent semantic indexing is quickly becoming the wave of now.

Latent semantic indexing, is a Google driven creation that’s meant to better gauge the content of a web page in relation to the entire site to discover the overall theme. It is a more sophisticated measure of what sites and their pages are all about. While it doesn’t mean webmasters need to completely retool all of their keyword optimization efforts, it does mean depth needs to be a greater consideration.

The history behind latent semantic indexing is rather interesting. Google’s current ranking system, which relies on incoming links (or votes) and keywords to scan pages for relevancy when surfers do searches has been known for penalizing perfectly good sites. The system was set up to scan for relevance and quality. In the process, it has a habit of knocking new sites and those which add too much content too quickly. Although some of these sites, naturally, are those that result from link farming and quick keyword stuffed content generators, not all are unplanned fabrications.

Google wanted a better way, and found one. Latent semantic indexing is meant to scan the overall theme of a site, so as not to penalize those sites that have fresh, relevant and good content even if they do happen to pop up over night.

This new focus puts an emphasis on quality and freshness of content to help sites gain higher ranking position. In essence, latent semantic indexing is meant to give a searcher the best possible site to meet their needs based on relevant keywords and comprehensive coverage and not just incoming links.

This system basically presents a more fair way to give search engine users the pages they really want. It does what Google has always tried to do – provide higher quality, more relevant results.

The old days of Google putting 80 percent of its emphasis on incoming links and 20 percent on the actual site itself are coming to an end. Incoming links will always have relevance, especially in regard to breaking search “ties,” but they may not carry the same weight as before. This can make it a bit easier for those who work on their sites with an emphasis on quality to see real results.

What all of this means to web publishers is that those who have done and continue to do their jobs correctly will have a better chance of shining with latent semantic indexing. Those who keyword stuff, create nonsensical content and spend a lot of time using link farms likely will not.

The key to getting ahead in the new age of Google search falls on quality. Sites that provide useful and relevant information in regard to their content will be likely to do better on searches. Those that cut corners could find themselves at the bottom of the search totem pole.

If you’re looking to build a perfectly-structured, highly-optimized site or even improve your existing site there is a great new software tool that just came out called (http://bhartzer2.silomatic.hop.clickbank.net) SiloMatic.

This software guarantees all of your web pages will be properly structured to rank high on Google and other major search engines.

You can read all about it right here at the SiloMatic site.

Avatar Seeks Semantic Search

WebProNews Staff — Fri, 27 Jul 2007 17:40:26 +0000

Researchers at IBM Almaden have been developing a semantic search process that can delve into unstructured text to retrieve structured information.

Avatar Seeks Semantic Search

While a lot of attention has been heaped upon Powerset and its almost-here natural language search, IBM has been working on a similar technology that may or may not be as close to public debut.

IBM calls their effort Avatar Semantic Search. Right now it doesn’t even have the nice minimalist home page Powerset has for early peek signups, but since everyone’s done reading ‘Harry Potter and the Deathly Hallows’, a little text to read is a good thing.

“Ongoing research in Avatar is at the cusp of a number of disciplines ranging from search and information retrieval to machine learning, information extraction, and probabilistic databases,” IBM announced on the project’s page.

We’ve looked at earlier IBM efforts to pull information out of unstructured resources. Their UIMA developments now occupy a place in the freely available IBM Omnifind Yahoo Edition enterprise search product, for example.

But UIMA is so 2005. While Powerset has drawn upon research performed by the Palo Alto Research Center, aka PARC, IBM reached out to the academic community to complement Avatar’s internal team.

They have approached the semantic search issue in three ways. Developing an information extraction system will allow Avatar to plunge into mounds of raw text, and emerge with structured data based on rules-based annotators.

IBM claimed this extraction system will permit unsophisticated users to build an annotator with Avatar and pull out the desired information from email, web pages, business reports, etc.

Through semantic search, the researchers think they can interpret queries people make, and model the real intent behind a query.

The real challenge comes from an effort they refer to as managing uncertainty and probabilistic databases. They’ve stepped deeply into theory here, well beyond any help Douglas Adams can provide for me.

IBM built momentum with UIMA starting well before I’d interviewed Marc Andrews about it in December 2005. It led to the co-branded, freely available Omnifind product I mentioned earlier, and I have to think Avatar may be on a similar track today.

What Semantic Search is Not

Mike Moran — Wed, 18 Jul 2007 14:47:25 +0000

You may have heard the term "semantic search," but do you really know what it is? Some people have very big ideas of how computers will understand the meaning of text, but today’s semantic search falls far short of that. Regardless, what’s possible today is still very useful.

To understand how hard it is for computers to really understand the meaning of text, let’s not look at understanding entire documents or even paragraphs. Let’s not even look at sentences. No, let’s start with something extremely simple: noun phrases.

Here’s a simple noun phrase: bath soap. It has a simple meaning, too—soap used in the bath. Let’s look at different phrase now—wood soap. It means soap used to clean wood. And one more: glycerine soap—soap made of glycerine.

Three noun phrases about soap and the modifying noun means something different each time. It’s not easy for software to interpret them correctly, as you might imagine. I don’t think you’ll see software that can correctly interpret most noun phrases for quite awhile.

So what kind of semantic search is possible?

Today’s keyword search can be vastly improved using mere part-of-speech analysis. Consider the law enforcement officer looking for a report on someone driving a Neon car. If it is an old Neon, it was a Dodge Neon. Newer models are Chrysler Neons. It’s likely the police reports that should be found contain neither the words Dodge nor Neon. So how do you do a keyword search? Searching for "neon" alone finds neon signs, neon lights, and other spurious results. Searching for "neon car" likely finds nothing.

Enter semantic search. With a semantic search facility, looking for "neon car" causes the system to look for occurrences of the word "neon" that denote cars. Simply knowing that a car is a noun eliminates almost all spurious results ("neon" is a modifier in the phrase "neon lights"). A bit more smarts, such as looking for forms of the word "drive" in the same sentence improves the results even more.

So even though semantic search is a big idea, practical implementations exist today to improve search results. Does your search facility have the smarts that semantic search proivides?

Comments

Tag:

Xerox Takes A Stab At Semantic Search

Doug Caverly — Thu, 21 Jun 2007 22:16:21 +0000

Every day, we bring you news of the latest comings and goings in the search engine industry. The names Google and Yahoo come up a lot . . . Xerox, not so much. But it’s that last company that is preparing a semantics-based search engine.

“Xerox Corp. says its new search engine based on semantics will analyze the meaning behind questions and documents to help researchers find information more quickly,” reports the AP’s Stephen Singer. “FactSpotter promises to help by returning a specific portion of a search document that is relevant to the query.”

Singer then relays an example of what semantic search is all about. “For example, common searches using keywords ‘Lincoln’ and ‘vice president’ likely won’t reveal President Abraham Lincoln’s first vice president. A semantic search should yield the answer: Hannibal Hamlin.”

But here’s where Xerox may have a problem, as, following a Google search for “Google+Search”>Lincoln vice president,” I can see Hamlin’s name in the second, sixth, ninth, and tenth results – without even bothering to click or anything. Whatever Xerox can do, Google may be able to do better.

Then there are the companies that only do semantic search. Powerset’s engine hasn’t yet been released, but a lot of people have been impressed by some early previews.

Still, Xerox’s product isn’t yet “out there,” either, so it’s too early to judge how things will go. “FactSpotter was introduced in Grenoble on Wednesday and will launch next year,” states Singer, “initially to help lawyers and corporate litigation departments plow through thousands of pages of legal documents. Xerox expects the technology to eventually be used in health care, manufacturing and financial services.”

Semantic Images & SEO

Barry Hurd — Mon, 18 Jun 2007 19:41:45 +0000

Very rarely do I come across software that makes me go “WOW”

Keep in mind that I read almost everything covering new social software.

Microsoft has a project team working on a piece of software called SeaDragon.

What is SeaDragon?

SeaDragon is awesome. I caught a video review a month or so ago and finally wandered back across it and need to share.

It is a way to link images to other images across the net. Using pixel information and meta data, SeaDragon has the ability to capture and infinite number of related images and being them together. Beyond that, it has the ability to connect angles.

What does that mean?

It can take 500 images of the Space Needle taken by 500 different people who posted them on Flickr and create a virtual image… pasting them together to form a comprehensive detail of entirely different angles. It can take a newpaper view and correlate one low resolution image to a hundred high resolution ones. It can take satellite images of the earth and overlay street and highway information.

While this may sound like “just another widget” for online software, capabilities like this could revolutionize how we browse data on the web.

For the specifics of social media and search optimization, it also means that images will have to be tracked and updated on a pixel level. If you want to be found for the keyword “Coke” you may soon have to have an image of a Coke logo properly indexed and keyworded on your site.

Take a few minutes and watch this presentation. It is impressive. You can also visit the Microsoft Labs site for SeaDragon.

SMX: Cutts on Themes and Latent Semantic Indexing

Andy Beal — Mon, 11 Jun 2007 21:45:32 +0000

Having worked closely with latent semantic indexing, during my time at FI, I’ve become a big advocate of making sure you have structured themes in your content, and that you include a supporting cast of semantically connected keywords.

In this clip from SMX Advanced, Matt Cutts shares how Google is continually testing the use of LSI, and keyword themes.

Click To Play

Comments