The first college fraternity, The Flat Hat Society, was formed in the Raleigh Tavern at the College of William and Mary in Williamsburg, Virginia.

Source