Is it reasonable to have multiple SSH keys?

So far I've created a separate SSH key for each server I need to login to (for each purpose, to be more accurate). I did it out of a sense of security, just like different passwords to different sites.

Does having multiple SSH keys actually improve security? All of them are used from the same machine, are located in the same ~/.ssh, most even have the same passphrase.

So... should I give up the whole system and just use one SSH key for everything?

[UPDATE 2015-08-05] Github publishes your public key, and your SSH client may send all of your public keys to every server, depending on configuration, thusly, if you are concerned with a 3rd party SSH server knowing your identity when connecting, you should use multiple SSH keys, though in my opinion it is paranoid.

Answer

SSH keys use public-key cryptography. That means that what you're installing on all those servers is just your public key, which you want the whole world to know. The only actual secret is your private key that you keep locked down on your own machine. So yeah, I'd say you're wasting your time.