According to Google Webmaster Trends Analyst Gary Illyes, Google is planning to rank mobile site speed separately from desktop. This means that if your mobile site loads quick and the desktop version of your site is slow you will rank better for the same query in a search done on a mobile device than you would on a desktop Google search.
Jennifer Slegg of The SEM Post noted that Illyes mention of this at Search Marketing Summit Sydney was the first time that Google confirmed that it indeed plans to make page speed a factor in its next mobile friendly update. Illyes told Jennnifer that the update will be in a matter of months. Illyes has been hinting at mobile friendly sites ranking higher for months.
Google added a mobile friendliness to their search ranking algorithm well over a year ago, but this is the first time that they are actually going to use mobile speed as a factor in their ranking and specifically for searches occurring on mobile devices.
Google offers a rich amount of information on how webmasters and web developers can make a site speedier and mobile friendly in general. Start here.