openssh - How can I force SSH to ignore the IdentityFile listed in "Host *" for one specific host?

My SSH config file is set up to use a specific IdentityFile in the Host * section as the default key. For one particular host, I want to offer a different key (and only that key, never the default key).

My ~/.ssh/config file looks like this:

Host specialhost
HostName specialhostname.mydomain.com
IdentityFile %d/.ssh/special-key
LogLevel debug
IdentitiesOnly yes 

Host *
IdentityFile %d/.ssh/regular-key
IdentitiesOnly yes 

Regardless of what I do, or regardless of the order of sections in the config file, my SSH client (OS X OpenSSH_6.2p2) offers the 'regular-key' first (which the server accepts, which is not what I wanted).

The ssh_config man page states the following about IdentityFile:

It is possible to have multiple identity files specified in configuration files; all these identities will be tried in sequence.

The "sequence" it refers to is unclear to me, since the IdentityFile listed in the Host * section is always offered first, even if "specialhost" is listed prior to it (as in the example above). The IdentitiesOnly trick suggested here only seems to be relevant to keys offered by ssh-agent. A few other similar questions have not yielded any answers.

I want to leave the regular-key in the Host * section because I use that key with dozens of other hosts and I don't want to duplicate the IdentityFile in every host entry.

Is there any way to force SSH to offer only the 'special-key' using just the SSH config (and without copying my IdentityFile directives to every other host)? I can see how I might achieve this by taking the keys out of the Host * section, adding them to the agent, and then using IdentitiesOnly yes for specialhost, but I would really prefer to avoid using ssh-agent unless there is no other way.

Answer

A Host line can have more than one pattern. And it's not really spelled out in the documentation, but a "!" (exclamation) at the beginning of a pattern means "if a host matches this pattern, then don't apply the section". In other words, you can do this:

Host * !special1 !special2
IdentityFile etc...

And it should match any host except "special1" and "special2".

I don't think the order of the patterns is important. The hostname being checked has to match one non-exclamation pattern, and it has to not match any exclamation patterns.