Google's Adam Lasnik clarifies what Google looks for in regards to penalties involving outbound linking.
First, he discounts a relationship between outbound links and the supplemental index, by saying,
It's unlikely that your outbound linking is causing your pages to be listed in the supplemental, rather than main index.
He then discusses the fact that Google looks for patterns of linking to bad neighborhoods, rather than penalizing for the occasional bad link when he says,
Also, be assured that we're not looking to penalize folks for a "bad" link here and there. Rather, our algorithms are tuned to look for patterns of "egregious" linking behavior... both on individual sites and in the aggregate.
He sums it up by encouraging webmasters to consider their users' interests, rather than Google's, when auditing outgoing links.
With that said, it's certainly in your users' interest that you regularly audit outgoing links on your site (especially prominent ones) to ensure that you're not losing folks' trust by sending them to inappropriate places or 404 pages. Sure, it's great to keep Google happy, but it's usually more important (long term) to have your users be return visitors.
Now, I have found that having a handful of links to 404's can cause ranking problems, but cleaning up those links have quickly resolved the problems. So, even if you don't have a pattern of bad outbounds, I would still recommend that you audit your links occasionally. But it is nice to know that Google is more concerned with the big picture, rather than the occasional "oopsie".