I can't speak for others, I do not support the man because he is not particularly bright, is dishonest (even by Washington standards), and is not proving to be very effective as a leader.Third, having said that I think it is extremely naive to think that there is not a significant amount of dislike of BO stemming from his race.
To test the theory, I would be pleased as punch to replace him with Alan Keyes (who, although a little bit kooky, has forgotten more about the Constitution than Obama will ever know) or Colin Powell (with whose positions I don't always agree, but whose judgment I do trust).
Schmucks like Olberman would spin that into racism somehow, but such an argument would be more a statement of Olberman's inability to reason coherently than a statement on my political beliefs or views on race.