There are a number of reasons your site might have been hit by Google's recent Penguin update (formerly known as the Webspam update). Barring any unintended penalties, the algorithm has wiped out sites engaging in webspam and black hat SEO tactics. In other words, Google has targeted any site that is violating its quality guidelines.
One major thing you need to avoid (or in hindsight, should have avoided) is cloaking, which is basically just showing Google something different than you're showing users. Google's Matt Cutts did a nice, big video about cloaking last summer. He calls it the "definitive cloaking video," so if you have any concern that you may be in the wrong on this, you'd better watch this. It's nearly 9 minutes long, so he packs in a lot of info.
Cloaking is "definitely high risk," Cutts says in the video.
With Penguin, there's been a lot more talk about bad links costing sites. Link schemes are specifically mentioned in Cutts' announcement of the Penguin update, and Google has been sending webmasters a lot of messages about questionable links recently. That's definitely something you don't want to ignore.
But while Google didn't mention cloaking specifically in the announcement, it did say the update "will decrease rankings for sites that we believe are violating Google’s existing quality guidelines." Cloaking fits that bill. Google divides its quality guidelines into basic principles and specific guidelines. Cloaking appears in both sections.
"Make pages primarily for users, not for search engines,' Google says in the Basic Principles section. "Don't deceive your users or present different content to search engines than you display to users, which is commonly referred to as 'cloaking.'"
In the Specific Guidelines section, Google says, "Don't use cloaking or sneaky redirects." This has its own page in Google's help center. Specific examples mentioned include: serving a page of HTML text to search engines, while showing a page of images or flash to users, and serving different content to search engines than to users.