In Google's latest Webmaster Help video, Matt Cutts discusses page speed and whether or not it's a more important factor for mobile than for desktop.
The video is a response to the submitted question:
Is load speed a more important factor for mobile? Is it really something that can change your rankings, all the things being equal?
"Let's start with the second part of that question: all things being equal," Cutts begins. "If your site is really, really slow, we've said that we do use page speed in our rankings. And so all of the things being equal, yes, a site can rank lower. Now, we tend not to talk about things in terms of like an absolute number of seconds because websites do work differently in different parts of the world, and there's different bandwidth and speeds in different parts of the world. However, it's a good way to think about it to say, 'Okay, look at your neighborhood of websites. Look at the sites that are returned along with you, and then if you're the outlier. If you're at the very bottom end because your site is really, really slow, then yes, it might be the case that your site will rank lower because of its page speed."
"Now, what's interesting is that that factor applies across the board," he continues. "It's not specific to mobile...it's not that in mobile we apply that any more or less than we do for desktop search, but if you're using your mobile phone, you do care a lot about whether it will load in a reasonable period of time. So we'll continue to look at ways to improve the ways that we find out how fast a site is, the page speed for a particular page, and then try to figure out whether it makes sense...okay, if we want users to be less frustrated, then maybe it does make sense to incorporate that more into our rankings or more for mobile Something along those lines."
About two years ago, Cutts said page speed affects rankings in about one out of a hundred searches, and that you shouldn't overly stress about it. It's unclear how much it matters now compared to then.