Why is a football match between two UK teams in the same town called a 'derby'?

I know it originated with the Earl of Derby and the Derby race, but I still do not see the connection. It is the same with other sports, I believe.