The algorithm is claimed to predict with 95% accuracy the topics that will show up on Twitter’s trending topics list. It can make these predictions an average of an hour and a half before Twitter lists the topic as a trend, and can sometimes predict trends as much as four or five hours in advance.

Devavrat Shah, associate professor in the electrical engineering and computer science department at MIT, and MIT graduate student Stanislav Nikolov will present the algorithm at the Interdisciplinary Workshop on Information and Decision in Social Networks in November.

Shah stated that the algorithm is a nonparametric machine-learning algorithm, meaning it makes no assumptions about the shape of patterns. It compares changes over time in the number of tweets about a new topic to the changes over time seen in every sample in the training set. Also, training set samples with statistics similar to the new topic are more heavily weighted when determining a prediction. Shah compared it to voting, where each sample gets a vote, but some votes count more than others.

This method is different from the standard approach to machine learning, where researchers create a model of the pattern whose specifics need to be inferred. In theory, the new approach could apply to any quantity that varies over time (including the stock market), given the right subset of training data.

For Shah and Nikolov’s initial experiments, they used data from 200 Twitter topics that were listed as trends and 200 that were not. “The training sets are very small, but we still get strong results,” said Shah. In addition to the algorithm’s 95% prediction rate, it also had only a 4% false-positive rate.

The accuracy of the system can increase with additional training sets, but the computing costs will also increase. However, Shah revealed that the algorithm has been designed to execute across separate machines, such as web servers. “It is perfectly suited to the modern computational framework,” said Shah.

“It’s very creative to use the data itself to find out what trends look like,” said Ashish Goel, associate professor of management science at Stanford University and a member of Twitter’s technical advisory board. “It’s quite creative and quite timely and hopefully quite useful.

“People go to social-media sites to find out what’s happening now. So in that sense, speeding up the process is something that is very useful.”

(Image courtesy MIT)

Well, the blame may not rest entirely on the radio stations – they may, in reality, truly have no choice in the quality of music they play. That’s because new research has confirmed that as the years have gone by, popular music has gotten worse and worse. Specifically, louder and less original, which is a longer way of saying it sucks.

The research comes from an AI specialist named Joan Serra at the Spanish National Research Council (CSIC). CSIC is the largest public institution of research in Spain, and the third largest in all of Europe – so let’s throw any arguments about legitimacy out the window.

Serra and the team used “complex algorithms” to process pop music from the last 55 years (1955-2010) – from Elvis to Lady Gaga. To do this, they used the Million Song Dataset, a “freely-available collection of audio features and metadata for a million contemporary popular music tracks,” whose purpose is to aid in research projects just like this one.

“We found evidence of a progressive homogenization of the musical discourse,” Serra told Reuters. “In particular, we obtained numerical indicators that the diversity of transitions between note combinations – roughly speaking chords plus melodies – has consistently diminished in the last 50 years.”

Translation: It’s all the bloody same.

Reuters says that the team also found that the “timbre palette” has gotten worse over the years.

Translation: The actual sounds are all the bloody same.

Not only is modern pop music swimming in a vomitous sea of sameness, but it’s also getting louder.

Now, when you’re having an argument with your 14-year-old cousin about her Bieber fever and his intrinsic talent, or how he’s fulfilling Kurt Cobain’s legacy or some shit, all you have to do is say that science has proven that he sucks.

Now, to be fair, one could make the argument that pop music has all been the same for the last half a century. Maybe the latest generation is worse, but in reality it really is all the same. For your consideration:

It all starts with an algorithm proposed by Dr. Łukasz Kaiser of Universite Paris Diderot. The algorithm proposes that machines can learn how anything works just by watching how it works. It’s like if Google’s cat loving computer learned what a cat was and then learned how to care for a cat by watching more videos of these actions.

This algorithm is not being tested on cats though. The tests are currently centered around games and learning how to play said games. The hope is that a computer can watch people playing a game like Connect 4 and then learn how to beat a human opponent just by watching them. Sure, machines can beat humans in Chess, but the machine has to be programmed by a human with all the potential moves available to it.

What makes Dr. Kaiser’s research so fascinating, and terrifying, is that machines would only have to watch to learn. We as humans learn by observing things around us and we’re giving machines that same ability. Of course, now we have to discuss machine rights and whether or not I’m a machinist. Look, I don’t hate machines, but I would not like to be killed by something that can’t feel basic emotions.

If you want to learn more about the downfall of humanity, check out Dr. Kaiser’s presentation on his research at the Third Conference on Artificial General Intelligence.

Lukasz Kaiser-Playing General Structure Rewriting Games from Raj Dye on Vimeo.

[h/t: Wired]

Nowadays, if you want something built, you take wood or other materials and build or cut it out of that. But, what if you could have a computer model of what you want, and have that thing magically appear out of a box of sand?

That is the very vision the brains at the Distributed Robotics Laboratory (DRL) at MIT’s Computer Science and Artificial Intelligence Laboratory are pursuing. It involves a lot of programming and seemingly-simple twiddling, but it could change the way things are made with the same kind of promise that 3-D printing has people so excited about. The development is called “smart sand”.

From the MITNews:

At the IEEE International Conference on Robotics and Automation in May — the world’s premier robotics conference — DRL researchers will present a paper describing algorithms that could enable such “smart sand.” They also describe experiments in which they tested the algorithms on somewhat larger particles — cubes about 10 millimeters to an edge, with rudimentary microprocessors inside and very unusual magnets on four of their sides.

Unlike many other approaches to reconfigurable robots, smart sand uses a subtractive method, akin to stone carving, rather than an additive method, akin to snapping LEGO blocks together. A heap of smart sand would be analogous to the rough block of stone that a sculptor begins with. The individual grains would pass messages back and forth and selectively attach to each other to form a three-dimensional object; the grains not necessary to build that object would simply fall away. When the object had served its purpose, it would be returned to the heap. Its constituent grains would detach from each other, becoming free to participate in the formation of a new shape.

Of course, ten-millimeter cubes is hardly what you would call “sand”, but the idea is to get the functionality and algorithms in place, then shrink the size of it over time.

“Take the core functionalities of their pebbles,” says [Robert] Wood, who directs Harvard’s Microrobotics Laboratory. “They have the ability to latch onto their neighbors; they have the ability to talk to their neighbors; they have the ability to do some computation. Those are all things that are certainly feasible to think about doing in smaller packages.”

“It would take quite a lot of engineering to do that, of course,” Wood cautions. “That’s a well-posed but very difficult set of engineering challenges that they could continue to address in the future.”

This video gives you an idea of the “subtractive” methods of building that the MIT folks are working on.

Their findings, published in the March 22 issue of Neuron, could help patients and their families confirm a diagnosis of dementia and prepare in advance for future cognitive declines over time. In the future — in an era where targeted drugs against dementia exist — the program might also help physicians identify suitable brain targets for therapeutic intervention, says the study’s lead researcher, Ashish Raj, Ph.D., an assistant professor of computer science in radiology at Weill Cornell Medical College.

“Think of it as a weather radar system, which shows you a video of weather patterns in your area over the next 48 hours,” says Dr. Raj. “Our model, when applied to the baseline magnetic resonance imaging scan of an individual brain, can similarly produce a future map of degeneration in that person over the next few years or decades.

“This could allow neurologists to predict what the patient’s neuroanatomic and associated cognitive state will be at any given point in the future. They could tell whether and when the patient will develop speech impediments, memory loss, behavioral peculiarities, and so on,” he says. “Knowledge of what the future holds will allow patients to make informed choices regarding their lifestyle and therapeutic interventions.

“At some point we will gain the ability to target and improve the health of specific brain regions and nerve fiber tracts,” Dr. Raj says. “At that point, a good prediction of a subject’s future anatomic state can help identify promising target regions for this intervention. Early detection will be key to preventing and managing dementia.”

The computational model, which Dr. Raj developed, is the latest, and one of the most significant, validations of the idea that dementia is caused by proteins that spread through the brain along networks of neurons. It extends findings that were widely reported in February that Alzheimer’s disease starts in a particular brain region, but spreads further via misfolded, toxic “tau” proteins. Those studies, by researchers at Columbia University Medical Center and Massachusetts General Hospital, were conducted in mouse models and focused only on Alzheimer’s disease.

In this study, Dr. Raj details how he developed the mathematical model of the flow of toxic proteins, and then demonstrates that it correctly predicted the patterns of degeneration that results in a number of different forms of dementia.

He says his model is predicated on the recent understanding that all known forms of dementia are accompanied by, and likely caused by, abnormal or “misfolded” proteins. Proteins have a defined shape, depending on their specific function — but proteins that become misshapen can produce unwanted toxic effects. One example is tau, which is found in a misfolded state in the brains of both Alzheimer’s patients and patients with frontal temporal dementia (FTD). Other proteins, such as TDP43 and ubiquitin, are also found in FTD, and alpha synuclein is found in Parkinson’s disease.

These proteins are called “prion-like” because misfolded, or diseased, proteins induce the misfolding of other proteins they touch down a specific neuronal pathway. Prion diseases (such as mad cow disease) that involve transmission of misfolded proteins are thought to be infectious between people. “There is no evidence that Alzheimer’s or other dementias are contagious in that way, which is why their transmission is called prion-like.”

Dr. Raj calls his model of trans-neuronal spread of misfolded proteins “very simple.” It models the same process by which any gas diffuses in air, except that in the case of dementias the diffusion process occurs along connected neural fiber tracts in the brain.

“This is a common process by which any disease-causing protein can result in a variety of dementias,” he says.

The model identifies the neural sub-networks in the brain into which misfolded proteins will collect before moving on to other brain areas that are connected by networks of neurons. In the process the proteins alter normal functioning of all brain areas they visit.

“What is new and really quite remarkable is the network diffusion model itself, which acts on the normal brain connectivity network and manages to reproduce many known aspects of whole brain disease patterns in dementias,” Dr. Raj says. “This provides a very simple explanation for why different dementias appear to target specific areas of the brain.”

In the study, he was able to match patterns from the diffusion model, which traced protein disbursal in a healthy brain, to the patterns of brain atrophy observed in patients with either Alzheimer’s disease or FTD. This degeneration was measured using MRI and other tools that could quantify the amount of brain volume loss experienced in each region of the patient’s brain. Co-author Amy Kuceyeski, Ph.D., a postdoctoral fellow who works with Dr. Raj, helped analyze brain volume measurements in the diseased brains.

“Our study demonstrates that such a spreading mechanism leads directly to the observed patterns of atrophy one sees in various dementias,” Dr. Raj says. “While the classic patterns of dementia are well known, this is the first model to relate brain network properties to the patterns and explain them in a deterministic and predictive manner.”

Starting today, YouTube’s algorithm for determining what videos appear on your suggested or recommended sections is morphing. Instead of picking these videos according to view count only, YouTube will now populate this suggestions section based on time spent viewing each video as well.

Here’s the reasoning behind this, courtesy of the Official YouTube Partners and Creators blog:

The last time you went channel surfing, did you enjoy (or remember) the 20 TV shows you flipped through, or just the shows you watched all the way through? Would you recommend the 20 you surfed through to a friend, or the ones you actually watched? To make the videos you watch on YouTube more enjoyable, memorable, and sharable, we’re updating our Related and Recommended videos to better serve videos that keep viewers entertained.

This makes sense. How engaged with a video the average user is should determine whether it is passed along as a recommended video a particular user. Another, possibly more important point of this move is to cut back on people gaming the system. YouTube elaborates on the YouTube Help site

Previously, the YouTube algorithm suggested videos (whether related videos on the watch page or recommended videos elsewhere on the site) based on how many people clicked to watch a video. This was a helpful way to promote channels, but issues like misleading thumbnails kept this system from bringing videos with deeper engagement to the top. Starting March 14th 2012, the algorithm for suggesting videos will also be based on which videos contribute to a longer overall viewing session rather than how many clicks an individual video receives. This is great for viewers because they’ll be able to watch more enjoyable content; moreover, this is great for creators because it can help build more focused and engaged audiences.

They mention “misleading thumbnails,” which is specification of a number of techniques creators use to boost the views on their particular videos. One of the most prominent examples is the “reply girl.” Users (usually female) will upload their videos as replies to popular YouTube videos, and most of the time show something sexually suggestive or otherwise unrelated image as the thumbnail in order to draw clicks.

Basically, what YouTube is saying here is you have to make your content relevant and it has to be something that people want to watch. If a user clicks your video and only stays for a few seconds because it doesn’t satisfy what they were looking for, you’re going to have a harder time making it to the recommended and suggested video sections.

“How can you adapt to these changes?” they say. “The same as you always have — create great videos that keep people engaged. It doesn’t matter whether your videos are one minute or one hour. What matters is that your audience stops clicking away and starts watching more of your videos.”

“We are asking for examples, and may use data you submit to test and improve our algorithms,” the company says on a “Report Scraper Pages” form, found here.

Google’s head of web spam, Matt Cutts, tweeted about the new initiative:

This testing comes after months of iterations of Google’s Panda Update, designed to improve the quality of search results, though there has been no shortage of complaints about scrapers ranking over original content in that time.

The testing also follows a recent, big refresh of Google’s spam submission process, discussed here.

This past week, Google shared an interesting video, providing an inside look at the search algorithm tweaking process. While no earth shattering information was necessarily contained, it did provide a rare visual glimpse into the process. Watch it below.

I don’t think everyone would agree that they’ve all been improvements, as we see complaints about this every day, but Google makes a lot of changes aimed at improving search results, nevertheless.

“There are almost always a set of motivating searches, and these searches are not performing as well as we’d like,” says Engineering Director Scott Huffman. “Ranking engineers then come up with a hypothesis about what signal, what data could we integrate into our algorithm.”

The video does provide some unique behind the scenes footage of Google engineers plugging away on their computers, presumably working on the algorithms.

Google briefly talks about the process of raters. “These are external people that have been trained to judge whether one ranking is more relevant and higher quality than another,” says software engineer Mark Paskin.

“We show these raters a side-by-side for queries that the engineer’s experiment might be affecting,” explains Google Search Scientist Rajan Patel. “We also confirm these changes with live experiments on real users.”

“We do this in something called a sandbox. We send a very small fraction of actual Google traffic to the sandbox. We compute lots of different metrics,” says Paskin.

“In 2010, we ran over 20,000 different experiments. All the data from the human evaluation and the live experiment are then rolled out by a search analyst,” says Huffman.

Sangeeta Das, a quantitative analyst says, “For each project, it’s usually one analyst assigned from the moment that we’re talking to the engineers, trying to learn about their change.”

“We then have a launch decision meeting where the leadership of the search team then looks at that data and makes a decision,” says Huffman.

“Ultimately, the goal of the search eval analyst team is to provide informed, data-driven decision, and present an unbiased view” says Das.

“If our scientific testing says this is a good idea for Google users, we will launch it on Google,” says Google Fellow Amit Signhal.

The video then looks at the “did you mean” and “showing results for” features as an example.

Clearly there’s still room for humans on the web. In search, that’s good news for Blekko, which brings the old human-edited approach back into the mix of an industry that has largely been dominated by the algorithm for the last decade, though the jury’s still out on whether it will ever be as effective as Google.

In terms of content creation, we’ve already seen the beginnings of what the algorithm can do. Look at Demand Media’s business model (at least for the content portion) – it’s largely algorithm based, though it still uses humans to write and edit the content.

The future content farm may be a different story though. We’ve also seen the absence of human intervention in content creation. Look at what Narrative Science is doing. The company, run by a former DoubleClick executive, describes itself in the following manner:

“We tell the story behind the data. Our technology identifies trends and angles within large data sources and automatically creates compelling copy. We can build upon stories, providing deeper context around particular subjects over time. Every story is generated entirely from scratch and is always unique. Our technology can be applied to a broad range of content categories and we’re branching into new areas every day.” 

Look at what IBM has been able to accomplish through machine learning with its robot Watson. How long until a bunch of Watsons are creating content for the web (and creating other Watsons, for that matter)?

The good news is it might still be a while before robots replace us all. Back to the points made in the Harvard Business Review, by Eli Pariser, he notes that algorithms aren’t yet good at predicting future news, to the extent that humans are.

As far as risk-taking, “Chris Dixon, the co-founder of personalization site Hunch, calls this “‘he Chipotle problem,’ he writes. “As it turns out, if you are designing a where-to-eat recommendation algorithm, it’s hard to avoid sending most people to Chipotle most of the time. People like Chipotle, there are lots of them around, and while it never blows anyone’s mind, it’s a consistent three-to-four-star experience. Because of the way many personalization and recommendation algorithms are designed, they’ll tend to be conservative in this way — those five-star experiences are harder to predict, and they sometimes end up ones. Yet, of course, they’re the experiences we remember.”

Would you trust content created by algorithms or do you put your trust in humans?

Have you noticed recent changes in your ranking? Tell us about it.

Algorithm Change

Google makes changes to its algorithm all the time, but when a change comes with an announcement, you know people are going to talk. On Friday, Google announced a tweak designed to surface multiple pages from a single site for relevant queries.

"For queries that indicate a strong user interest in a particular domain, like [exhibitions at amnh], we’ll now show more results from the relevant site," says Google software engineer Samarth Keshava. "Prior to today’s change, only two results from www.amnh.org would have appeared for this query. Now, we determine that the user is likely interested in the Museum of Natural History’s website, so seven results from the amnh.org domain appear. Since the user is looking for exhibitions at the museum, it’s far more likely that they’ll find what they’re looking for, faster. The last few results for this query are from other sites, preserving some diversity in the results."

Not all webmasters have been thrilled with this. "Brace yourselves! Another Mayday disaster coming," one person commented on our story about it.

What do you think of this algorithm change? Comment here.

Experimenting

Just as the company frequently changes its algorithm, it also frequently experiments with different features, showing them to small sets of users before either turning them into full-fledged features or throwing them away. The jury’s still out on this one, but a new experiment has been spotted, which alters search results as you type your query.

Think of this like autosuggest taking over the entire SERP. The video demonstrates:

Again, this is only an experiment at this stage, and it may never make its way to the mainstream Google experience, but people are already expressing a great deal of concern about it (particularly with regards to queries that begin with words that could yield undesired NSFW results).

My guess is that Google would have ways around that issue, but it remains to be seen if users/webmaters will have to deal with it. If the feature does come to fruition, this is something SEOs are going to have to consider, as it could have a big impact on the habits of searchers. You may, for example, want to optimize more for the earlier words in a longer key phrase, in addition to the key phrase itself. But, we’ll see.

Should Google change search results as you type? Comment here.

Google Crawling Sites From Numerous IPs

Barry Schwartz at Search Engine Roundtable points to some discussion from SEOs in Webmasterworld, who have found for the first time that Googlebot is now crawling from several different IP addresses at the same time. One webmaster said, ". their fast activity notified me so I took a peek to see who was scraping the site… I’ve never seen Google spider so fast and from so many IP addresses, they were all valid Google ip’s but there was like 10-20 of them running at once."

Acquisition

The other day, it was officially announced that Like.com has been acquired by Google. Like.com is a shopping search company offering visual search technology and an automated cross-matching system for clothing and other merchandise.

At this point, it’s unclear what Google has planned for this technology, but it could very well affect search results for shopping queries, which means it could affect small business visibility for better or for worse. Shopping search is going to be an area of Google to keep an eye on.

Have you noticed anything else interesting happening with Google search within the last week or so? Are you seeing things that are impacting your rankings? Let us know.