According to Google, just because your site is manually penalized, it does not mean that your site will vanish from the rankings. In fact, it may not hurt it much at all in some cases. A lot of that depends on you, the webmaster, and the other signals you've managed to send the search engine.
In a Google Webmaster Central forum thread (via Search Engine Roundtable), a webmaster discussed being denied his 4th reconsideration request, and Google Webmaster Trends analyst John Mueller had some interesting words. For that guy's specific story, you can view the thread, but that's not really the point. Here's what Mueller said:
The primary manual action that is affecting your site is that these unnatural links are being ignored. This is more or less in line with the spreadsheet that you have submitted, and would generally not be affecting the other links to your site. That said, while these things may have been counting for your site in the past, they no longer are -- so it's possible that you'd see some effect in your site's crawling, indexing, and ranking. Past that, keep in mind that the manual action here might not be the strongest element affecting your site's performance, we use over 200 factors in our crawling, indexing, and ranking, and regularly announce updates (such as http://googlewebmastercentral.blogspot.ch/2012/04/another-step-to-reward-high-quality.html ). My recommendation would be to not focus so much on this specific manual action, but instead to work to make sure that your site (and how it interacts with users and the rest of the web) is the best it can possibly be. Emphasis added.
The manual action is just one thing Google is taking into consideration. If you have enough other positive signals going your way (or can at least gain some), it's possible that Google's manual penalty will have little effect on your site's rankings.
Speaking of signals, Google is about due to put out its monthly (at least it used to be monthly) list of algorithm changes. We have yet to see August's changes, and September is almost over. Last time, they released two months' worth of lists at the same time. Perhaps we'll see them do that again. It seems likely at this point. Or who knows, maybe they'll wait and give 3 months' worth next time.