When the olympics were held in Atlanta, they used facilities at UGA, Georgia Tech, and some other smaller colleges in the area, along with some pro sports buildings.
They built the main Track and Field / ceremony stadium so that it could be renovated into a new stadium for the Braves. The Braves, this year, are now moving on to a new stadium and that stadium was bought by Georgia State (Sunbelt I think) to turned into their new football stadium.
In fact, the only new construction that has been demolished since those olympics 20 years ago, is the tennis complex as far as I know.
Bottom line, the richest country in the world has used and reused their olympic facility investments for over 20 years. Brazil has already thrown that entire investment away.
Part of me says, just permanently put the summer olympics in greece. The other part contrasts Atlanta to Rio and says these issues aren't caused by the olympics, but instead are just a side effect of the governments that host them.