It also prevents the possibility of some unauthorized program (virus, rootkit, just a misbehaving one) accepting connections without you knowing about it.
I used to configure my firewall to also block outgoing connections except for some ports, as a social service of a kind, but it turned out to be quite a hassle, for a relatively secure Linux box. On a Windows system it's a must, and all Windows firewalls do it (that I know of).
For the truly paranoid, you also have intrusion detection systems like Tripwire, rootkit scanners and such. Gentoo has an extensive security handbook
that you might want to read sometime.