First of all, I guess we'd have to define "college town." I'm not so sure I would classify Baton Rouge or Nashville as college towns. Both are the capitals of their respective states and are good size cities in their own right. Columbia, though not as large, is also the state capital. I've been there three times for football games and I've yet to even find the campus. Knoxville, also, is a bit large for me to think of it as a college town; however, it probably qualifies. Take out UT and you'd probably realize that it is indeed, primarily a college town.
On the other hand, Athens, Auburn, Fayetteville, Gainesville, Lexington, Oxford, Starkville, and Tuscaloosa are definitely "college towns."
Of those I consider to be "college towns," Of course, I'd pick Tuscaloosa, followed by Athens, Lexington, Oxford, Gainesville. Of all SEC cities, I'd still choose Tuscaloosa first, followed by Nashville, Baton Rouge, Knoxville, Columbia. Regardless Auburn and Starkville would be at the bottom of either list.