For manual Penalty you can check Google Webmaster Tools account. Here they can see whether the problem has been detailed within the account and what the solution is.
"... It could very well be the case that, 'hey we thought there was some keyword stuffing or cloaking or whatever going on,' and that's a clear-cut case, you'll get a notification, you'll get a message, and then you can start to figure out and investigate, 'OK, where can I start to improve things or what can I change to make things better.'"
--> Start from Google Webmaster Tool
--> Check your website Back links
--> Analysis your website back links
--> Filter out all bad links
--> Try to remove bad links
--> Check any mail/notification from Google
--> If you have received any mail then you will Disavow your links.