According to Sean Gourley, the co-founder and CTO of quantitative analytics firm Quid, the FBI wants to track social sites to not only detect potentially catastrophic events such as bombings and weather related incidents, but it also wants to create a predictive model that would analyze the “publicly available” information that users put on social networks to help it detect “bad actors.”

How do you feel about the FBI’s request for help in monitoring social media sites? We’d love to know.

In other words, the FBI is looking for an equivalent to what a company such as Radian6 does for brands. As Gourley explained, most of these types of services do brand monitoring and sentiment analysis. The FBI, however, hopes to piece together multiple data streams that could indicate that something was happening before it reaches the mainstream media.

For example, if they see activity pertaining to “explosions” or “smoke,” they could potentially jump in and minimize the situation before it gets completely out of hand.

“What they’re trying to do is go a little bit beyond just… ‘Is this a good tweet or a bad tweet?’ to say, ‘What actually happened in the world?’ and ‘When can we know it?’ and ‘Can we be the first to know it?’” said Gourley.

While the idea is simply nothing more than that, at this point, there are challenges that the FBI faces. For starters, it will be difficult to understand what is real versus what is fake, if it merely relies on keywords. Take the word “bomb” for instance. It can be used to say someone or something is “the bomb,” which does not mean that either is literally an explosive device.

Another challenge that the FBI is facing is the backlash from privacy groups, including the Electronic Privacy Information Center, over its request. Even though the FBI has said that it would only use the information that users make “publicly available,” these groups believe that such monitoring would prevent free speech and violate privacy and civil rights.

“Anytime you are tracking people’s activities to infer a type of behavior that has significant consequences, people are gonna start to get worried,” said Gourley.

As he explained, technology has advanced, and new services such as Facebook, Twitter, and Foursquare have developed in a way that has made sharing very easy. Our society, as a whole, enjoys checking-in to places, writing reviews on Yelp, and simply sharing about themselves on the other social sites.

Through this sharing cycle, Gourley said that we have essentially entered into a “deal” with social sites that allows them to serve us with search ads if we share data.

“We’ve sort of entered this bargain as a society to share information in return for being served ads,” he said. “What we haven’t really, at least not consciously, entered into is sharing information in return for perhaps being identified as a bad actor by the FBI,” he said.

While this is true, what most consumers may not realize is that the government has been monitoring information for many years. The Internet, and namely social networks, is now just the newest form with which to do it.

“Indeed, the whole concept of open source intelligence has been around with us since the second World War… it’s just kind of evolved now to take advantage of the data streams we have in front of us,” pointed out Gourley.

Since consumers are using social sites as its primary mode of communication, Gourley believes that the FBI should be using the information in order to protect the country to the best of its ability.

“If a company like Radian6 can know more about a brand than you can know about a bomb in Afghanistan, the FBI looks pretty stupid,” he said. “So, this is absolutely the kind of things they want to be using and needs to be doing to do their job properly.”

According to Gourley, the interesting part in this concept is the fact that the FBI isn’t developing the technology themselves. He told us that, when he graduated, the top jobs were with the government and the National Security Agency. Now, the tables have turned since the crash of Wall Street, and the current top jobs are in the Silicon Valley. Instead of working on government intelligence matters, the talent is working to figure out how to serve better ads to encourage consumers buy more.

“The sad truth is there’s more money in selling ads to Facebook users at the moment,” said Gourley. “The kind of dollars they’re [FBI] looking at here really do pale in comparison to what can be earned from serving social media ads.”

He went on to say that the FBI would likely receive some proposals but that the demand for solutions that sell more ads is much higher.

Incidentally, a House Homeland Security Committee hearing was held last week in which the Department of Homeland Security testified that its efforts were not violating any laws. Representative Jackie Speier, however, spoke out against the agency over some accusations regarding the monitoring of reporters’ activities. She referenced statements filed by EPIC and even asked that the nonprofit’s recommendations be considered.

At this point, it’s still very unclear how this saga will play out.

Should the FBI be allowed to monitor social networks, or should it be filtered? Would you feel safer or worse knowing the bureau was monitoring this activity? Please let us know what you think in the comments.

IQ tests are based on two types of problems: progressive matrices, which test the ability to see patterns in pictures, and number sequences, which test the ability to see patterns in numbers. The most common math computer programs score below 100 on IQ tests with number sequences. For Claes Strannegård, researcher at the Department of Philosophy, Linguistics and Theory of Science, this was a reason to try to design “smarter” computer programs.

“We’re trying to make programs that can discover the same types of patterns that humans can see,” he says.

The research group believes that number sequence problems are only partly a matter of mathematics – psychology is important too. Strannegård demonstrates this point:

“1, 2, …, what comes next? Most people would say 3, but it could also be a repeating sequence like 1, 2, 1 or a doubling sequence like 1, 2, 4. Neither of these alternatives is more mathematically correct than the others. What it comes down to is that most people have learned the 1-2-3 pattern.”

The group is therefore using a psychological model of human patterns in their computer programs. They have integrated a mathematical model that models human-like problem solving. The program that solves progressive matrices scores IQ 100 and has the unique ability of being able to solve the problems without having access to any response alternatives. The group has improved the program that specializes in number sequences to the point where it is now able to ace the tests, implying an IQ of at least 150.

“Our programs are beating the conventional math programs because we are combining mathematics and psychology. Our method can potentially be used to identify patterns in any data with a psychological component, such as financial data. But it is not as good at finding patterns in more science-type data, such as weather data, since then the human psyche is not involved,” says Strannegård.

The research group has recently started collaborating with the Department of Psychology at Stockholm University, with a goal to develop new IQ tests with different levels of difficulty.

“We have developed a pretty good understanding of how the tests work. Now we want to divide them into different levels of difficulty and design new types of tests, which we can then use to design computer programmes for people who want to practice their problem solving ability,” says Strannegård.

We might find out in the coming weeks. The Navy SEALs who carried out the mission that resulted in the death of Bin Laden left the Pakistani compound with more than a body. According to Politco, the SEALs were able to recover hard drives, CDs, and flash drives from the raid that U.S. officials are calling “the motherlode of intelligence.”

Currently, the data is being combed through by hundreds of people at a secret location in Afghanistan. Everyone in Washington is understandably excited to see what kind of information, if any, is usable. “It’s going to be great even if only 10 percent of it is actionable,” officials said to Politico.

The first thought has to be about the contents. Exactly what kind of information could be on these drives? It has been reported that Bin Laden’s compound lacked phone lines or internet in order to keep a lower profile – so it’s hard to guess the contents.

About 5 years ago it was famously unearthed that Osama Bin Laden had a soft spot for some aspects of American pop culture. A woman named Kola Boof wrote a tell-all book where she claimed to be a former sex slave/ girlfriend of Bin Laden. In the book, she detailed the Al Qaeda leader’s obsession with Whitney Houston among other things such as The Wonder Years.

I guess it’s possible that all they could find is the finale to The Wonder Years, you know, the one in the barn where Winnie…ah I can’t talk about it. Talk about depressing on multiple levels.

But hopefully something of use can be taken from the drives, some sort of data that can be used effectively.

The second thought has to be about encryption. Gizmodo has an interesting take on accessing the data:

We don’t know if “the mother lode of intelligence”—as that official called it—in the hard drives and memory sticks is encrypted or not. Given Osama’s level of confidence—the circumstances appear to indicate that he was very comfortable and felt very secure in his lair—the idea of some of the data not being encrypted is not that crazy.

But even if it is, the US intelligence agencies have the necessary computing power and the expertise to crack the information open, even if the terrorists are using the AES-256 standard. You can be sure that, if there are any encrypted files, they are now being processed by supercomputers at CIA’s headquarters. The only question is how fast they can access the information. That’s the critical part: the fastest they get it, the more actionable that information would be, leading to the fast capture or killing of other leaders and operatives in the al Qaeda network.

Yesterday, video and photos surfaced from the raid of the inside of the compound. The graphic images were of blood-spattered rooms in total disarray. Today, we await the possible release of photos of Bin Laden’s body. The White House is currently debating the pros and cons of that action.

It’s difficult to look at your competitors without leaving tracks. It can be a challenge, but is extremely important to manage your own reputation. This panel will look at a number of issues surrounding competitive intelligence being probed by the competition and probing the competition yourself.

Moderator:

• Jake Baillie

Speakers:

• Jake Baillie, Managing Director, STN Labs
• Andy Beal, Internet Marketing Consultant, Marketing Pilgrim LLC
• Larry Mersman, Vice President, Trellian

First up is famed Marketing Pilgrim, Andy Beal of Marketing Pilgrim LLC. Andy wants to discus tools for spying on your competition (Sounds great!).

1. DomainTools.com: Gives a great deal of background information. It tells whether the site is listed in Yahoo Directory, Open Directory, registration details, etc. It also gives information about other sites in the same IP.
2. ranks.nl/tools/spider.html: Great tool as it breaks up keyword density into 2, 3 and 4 word combos.
3. sitexplorer.search.yahoo.com: Yahoo! places the most important backlinks at the top of the list.
4. seomoz.org/tools: Great way to find out about competition’s page strength. Gives information on how many times they have been in del.icio.us/digg etc.
5. SoloSEO.com/tools/indexRank.html: This is great to find how many pages Google has indexed in the last year and how far they have grown.
6. copernic.com: Use to track site changes. Anytime your competition has an update you will be aware.
7. Technorati.com: Find out about the kind of feedback people are giving about your competition.
8. google.com/alerts: Use this to alert yourself on competition activities.
9. searchanalytics.compete.com: Lets you know which keywords are bringing in significant traffic to your site.
10. touchgraph.com: Show you competition’s link clusters. Know where the important links are coming from.
11. google.brand.edgar-online.com: Shows you your competitor’s SEC filings.
12. seekingalpha.com/transcripts: View transcripts.
13. google.com/patents: Information on your competition’s patents.
14. Oodle.com: Shows whether your competition is hiring or not.

Keep a track on your competition’s employee blogs. Especially if the company has no inkling about the blog. Who knows what you could stumble upon.

Next up is Larry Mersman, Vice President of Trellian.

Competitive intelligence can mean many things depending on the channel we are dealing with. For the most part, it is the gathering of information about your competitors.

Sources of information:

• Articles
• Blogs
• Online articles
• Internet Service Provider
• User Panels (User Installed Software)
• Website Search Patterns/History

How to find your competition?

• HitWise
• CoMScore
• Use search engines
• Find out who is bidding on your keywords

How did your competition get there?

• Referring Domains/Backlinks. Who is sending them traffic?
• Keyword Data: What keywords are actually being clicked on to get the user to your competition?
• Where is traffic coming from? search engines, banner ads, blogs, etc.

Final points:

• Knowing which keywords are performing best for your competitor will help you understand their strengths and weaknesses
• Know the performance of both paid and unpaid keywords (helps in optimizing your site around proven data, possibly streamline your spending and increase your ROI)
• Many companies will optimize their website around kws they think they will be found under, or where the end user will find a link to their site

Finally, it’s the users who make the choices that drive the traffic and the money to your site.

Last speaker is Jake Baillie, Managing Director, STN Labs. The best webmasters already investigate their competition. Search engine optimization is a game. Know more than your competition and you will win. Most novice webmasters have no idea. Use this to your advantage.

Mistakes made by novice webmasters:

• Put sensitive data on their web servers
• Use competitive research tools from their own company Ips

WHOIS (www.whois.sc) my competition:

• Designed in the 80s
• WHOIS was originally intended to be contact point for technical issues
• Evolved to be the “legal documentation” of internet domains
• Forge-abe with very little technical knowledge or even anonymized

Regional IP Databases:

• Use nslookup to find the IP address of the domain
• Plug the IP into a regional database IP and see what company that IP is registered to. Worst case scenario, you’ll find the ISP

Social Engineering Targets:

• ISP employees
• Spouses, significant others of employees or ex-employees.
• Marketing departments/sales people
• PR firms

Next, Jake has a script on how to get information out of people!

The Script:

• Introduce yourself as someone you’re not.
• Be friendly. People love friendly people. Never become confrontational
• If you don’t get what you want the first time, hang up and call back to talk to someone else.

Allinanchor:

• Use it to return webpages linked to with that target term
• Good for discovering networks
• Google the links: Search Facebook, follow people on Twitter, search MySpace etc
• You can pretty much tell an SEO’d site these days by a visual link inspection
• Keep an eye on unnatural traffic
• Users who come in to your website after using allinanchor are not your target visitors
• Those who come 20 times in 2 minutes through the same cache aren’t your target visitors
• People from whois.sc are competitors

Tracking and Logging:

• Track their referrer and do something cool with competitors via mod-rewrite:
• Send all incoming traffic from that specific referrer to a porn site.
• Serve them a 403 access forbidden message.
• Make them think the site is down.

Defence against social engineering:

• Instruct your employees not to talk to no one about your site.
• Find a trustworthy ISP – most intelligence is gathered at this point.
• Tell your Significant other to not take business calls at home.

Comments

Tag:

The newspaper industry has always been more interested in keeping money in-house rather than sharing even a little with others. Publishers have big egos, and depending on someone else to bring home the bacon doesn’t have much appeal, even with the newsprint industry facing financial troubles.

Even the newspaper-savvy duo of Yahoo’s Sue Decker and Hilary Schneider, architects of Yahoo’s HotJobs alliance with the print classified industry, aren’t appealing enough to keep the publishers’ eyes from wandering to other girls and fantasizing about their revenue undergarments.

Classified Intelligence recounted the recent deal with Kaango by Hearst and MediaNews, a $20 million infusion that gave them an 80 percent stake in the company. Kaango has deals with about a hundred newspapers for community-oriented classified sites.

There is no mincing of words when it comes to why Hearst and MediaNews bought into Kaango, as well as cutting Lee Enterprises in for a ten percent stake. They want to take on Craigslist as well as eBay’s classifieds product, Kijiji.

Details of the deal came as news to Yahoo, according to Classified Intelligence. A Yahoo spokesperson said the company was not aware of the Kaango announcement and requested more details about it from Classified Intelligence.

follow me on Twitter

Visual analytics will be the key. Science Blog said “analysts must meld the encyclopedic eye of Google with the inductive genius of Sherlock Holmes” to accomplish this.

The inspiration of Edward Tufte, visual data display legend, motivates the ongoing work by DHS and its associated research partners:

The centers

]]> http://www.webpronews.com/dhs-emulating-doctorows-scroogled-tale-2007-11/feed 0 http://www.webpronews.com/craigslist-lists-new-job-posting-fees-2007-10 http://www.webpronews.com/craigslist-lists-new-job-posting-fees-2007-10#comments Thu, 04 Oct 2007 23:03:37 +0000 WebProNews Staff http://www.webpronews.com/?p=40883 After deliberation in the Craigslist forums, the massively popular online classifieds site will begin imposing a fee on job postings in a quartet of new markets.

Famed for its modest pricing model and ubiquitous presence, Craigslist only charges fees in a handful of places for job postings. That list will grow by four markets: Chicago, Orange County, Portland, and Sacramento, effective November 1, 2007.

Classified Intelligence cited the announcement, which was posted on the Craigslist forums:

Over the last 6 months we have received a lot of feedback in this forum and by email, the consensus of which is that a $25 fee for posting jobs in these four cities would be beneficial, with many recent comments to the effect that we may have waited too long to implement such a fee.

Though some have balked at the new fee, many Craigslist devotees welcomed the announcement. Classified Intelligence noted the impact of adding job posting fees in markets previously by the site:

Job ads are a cash-cow for Craigslist; the number of job listings in New York and Los Angeles dropped substantially when the fee began, “but the volume quickly rebounded and by 2007 had more than tripled in each city,” the company said.

“After an initial drop-off in ad volume (in Boston, San Diego, Seattle and Washington) similar to that experienced in NYC and LA, the volume of legitimate job ads in these four cities has more than doubled.”

In addition to the job fees, Craigslist charges $10 for brokered apartment listings in New York City. Classified Intelligence believes the site’s job listings perform better, as fees drive out the junk postings.

]]> http://www.webpronews.com/craigslist-lists-new-job-posting-fees-2007-10/feed 2 http://www.webpronews.com/ebay-facing-serious-dropoff-in-germany-2007-09 http://www.webpronews.com/ebay-facing-serious-dropoff-in-germany-2007-09#comments Fri, 14 Sep 2007 16:42:23 +0000 WebProNews Staff http://www.webpronews.com/?p=40419 The online marketplace is in danger of losing out on the lucrative German market, and eBay has entered full panic mode in an effort to staunch the bleeding.

]]> The online marketplace is in danger of losing out on the lucrative German market, and eBay has entered full panic mode in an effort to staunch the bleeding.

A 50 percent drop in listing fees, and the replacement of inexpensive email-based customer support with pricier telephone customer service, at eBay’s German division represents the company’s try at recovering from a downturn.

The Fear is in full effect at eBay.

Giving up revenue while adding costs shows this, as a new report from Classified Intelligence described eBay’s German woes:

The announcement came in the wake of signs that all was not well with the core business, which had prompted observers to ask whether the impressive growth phase of the 12-year-old marketplace might be drawing to a close.

In July, EBay Inc. said the number of products listed worldwide dropped 6 percent in the second quarter compared with the year-ago period. In Germany, the online publication iBusiness reported “a clear drop in the number of concurrent auctions running on EBay.de” in 2007. Axel Gronen, an EBay.de analyst, suggested the number of active new members might be declining from year to year.

With eBay Germany’s newly-rediscovered love for consumer-to-consumer business, Classified Intelligence suggested the growth potential on the professional seller side could be hitting a ceiling.

German sellers can place a listing, with photo, on eBay for about 67 cents. An earlier <a href=http://investing.reuters.co.uk/news/articleinvesting.aspx?type=media&storyID=nL04854337>Reuters</a> report about this said Germany and the US are eBay’s top two markets.

Profitability losses from their upgraded support, which lowered phone question costs to 14 cents from 59 cents per minute, and reduced listing fees will need to be offset with a larger volume of participants.

That will be the real trick for eBay.

Sellers should be eager to capitalize on the lower fees with more competitive pricing for buyers, and that could set up a nice cycle of growth for eBay to recoup these new costs.

eBay has been spending money on promotion on the US side of their business. Forbes cited one analyst who thinks the impact on margins will be offset by sellers using more auction features.

Listings have been rising since late August, after a year of declining, according to Forbes. eBay’s strategy looks like it’s working.

]]> http://www.webpronews.com/ebay-facing-serious-dropoff-in-germany-2007-09/feed 0 http://www.webpronews.com/when-spies-do-search-2007-09 http://www.webpronews.com/when-spies-do-search-2007-09#comments Fri, 07 Sep 2007 23:39:36 +0000 WebProNews Staff http://www.webpronews.com/?p=40293 The Central Intelligence Agency launched CIA Wire, a service that will help users of the Library of National Intelligence sift through available information from participating agencies.

]]> The Central Intelligence Agency launched CIA Wire, a service that will help users of the Library of National Intelligence sift through available information from participating agencies.

The Iraq war and the 9/11 terrorist attacks have been the most recent examples of situations where better availability of intelligence may have led to better choices being made, ones that could have preempted the losses of thousands of lives.

Government agencies are no strangers to insularity and a fiefdom mentality. What’s theirs is theirs, and they have never been real fans of sharing. That long-standing issue could get in the way of what should be a promising development for intelligence analysts throughout the federal bureaucracy.

ResourceShelf cited the opening of CIA Wire, the CIA’s contribution to the Library of National Intelligence (LNI). It may be intentional or happenstance that it opened four days ahead of the sixth anniversary of the 9/11 tragedy.

As a top-secret resource, access to the CIA Wire will be limited. A report at FCW.com said the full LNI initiative will launch by October 31st, and described the process of how requests for documents will be fulfilled:

Mike Wertheimer, assistant deputy director of national intelligence for analytic transformation and technology in the Office of the Director of National Intelligence, said that when intelligence community employees request a document from LNI, one of three things can happen:

They will receive a copy of the document. They will need to be cleared before obtaining a copy of the document. They will not be able to view the document.

“We will measure attribute-based access for the first time and then we can start to automate,” Wertheimer said. “LNI is just the starting point for us to get a handle on what we know and how we use it.”

Minimal information will be available in the form of metadata arranged in a "card catalog listing." Requesting the linked document starts the aforementioned verification process.

We found the mention of "available information" on LNI a telling phrase. It refers to the data an agency chooses to make available to the project, not just what the agency possesses.

To give our analysts the best chance of heading off future threats, agencies need to do their utmost to make as much information available as possible. An analyst with proper clearances who could make a critical connection between disparate events should have the opportunity to do so.

Tuesday the 11th should be a grim reinforcement of that idea.

]]> http://www.webpronews.com/when-spies-do-search-2007-09/feed 0 http://www.webpronews.com/newspapers-sniping-over-careerbuilder-2007-06 http://www.webpronews.com/newspapers-sniping-over-careerbuilder-2007-06#comments Thu, 21 Jun 2007 22:50:30 +0000 WebProNews Staff http://www.webpronews.com/?p=38664 McClatchy, Gannett, and Tribune own parts of the CareerBuilder job listings site, but McClatchy is starting to feel like it’s just heard the song stop during musical chairs, and all the seats are taken.

Newspapers Sniping Over CareerBuilder

Online job classifieds at McClatchy’s existing newspapers have performed well in 2007. But the Knight Ridder papers it acquired, which use CareerBuilder as their employment presence on the Internet, have seen their job revenue drop 14.4 percent.

Classified Intelligence reported the awful truth McClatchy CEO Gary Pruitt discussed at a recent Newspaper Association of America event. “We would prefer to stay with CareerBuilder … but we have to consider all options, which would include selling our share,” Pruitt said at the NAA’s Mid-Year Review.

At one point, Knight Ridder owned a third of CareerBuilder, as did Gannett and Tribune. When McClatchy bought Knight Ridder, the papers renegotiated the ownership. McClatchy ended up with 15 percent, a figure that has reduced its revenue opportunity, according to McClatchy VP of interactive media, Christian Hendricks.

Classified Intelligence indicated that neither CareerBuilder nor Gannett are too concerned about McClatchy crying foul now. CareerBuilder’s CEO said his firm would be successful “whether McClatchy’s in or McClatchy’s out.”

Gannett CFO Gracia Moore told Dow Jones that Pruitt “knew what the deal was when he signed up for it.” If comments were music, Moore’s statement would be a very tiny violin playing for Pruitt, it seems.

]]> http://www.webpronews.com/newspapers-sniping-over-careerbuilder-2007-06/feed 0