The man fell from the top of the 200ft-tall Scott Monument as the neighbouring Princes Street Gardens was packed with day-trippers visiting the Christmas market.
Skaters on a nearby ice rink and staff at the attractions witnessed the man fall at around 1.30pm on Friday, landing at the back of a staff hut on top of a ladder.
Stalls at the busy market remained open, but parts of the area were closed off as police probed the incident, which is believed to be a suicide.
Eyewitnesses saw policing examining a note at the scene and others spoke of their shock at the death.
A staff member at one of the nearby attractions said: “He fell down the back of the staff hut and crashed on top of a ladder.
“Everyone on the ice rink got a perfect view, luckily no kids seen!”
Police were still in attendance at the scene in the late afternoon and had cordoned off part of the Christmas market as they continued to interview witnesses.
Officers were present not only at the scene and below the monument, but also on the Princes Street side of the attraction.
The children’s carousel below the monument was turned off, along with the lights while market staff on the closed side were confined to their booths.
However, the ice rink and main part of the market were still open. Visitors to the market were diverted by police and security staff to the lower part of the gardens.
The man’s body was removed by a private ambulance at 3.40pm.
One shocked shopper who asked not to be named said: “I can’t believe it. I’m totally shocked, what a poor man. I’m just speechless.
“What a horrible way to go.”
While David Gibb,20, said: “I’ve lived here my whole life and I’ve never heard of anyone falling from there.
“I can’t believe it actually. What a shame.”
Visiting photographer Steve Smith Tweeted: “#Scott monument somebody has fallen might be a suicide police are reading what looks like a note.”
Edinburgh Christmas markets opened last weekend and draw thousands of tourists to the area.
A Police Scotland spokesman said: “Police in Edinburgh responded to Princes Street at around 1.35pm on Friday 29th November following reports that a man fell from the Scott Monument.
“Ambulance service were also in attendance. However, he was pronounced dead at the scene.
“There are no suspicious circumstances surrounding his death and a report will be sent to the Procurator Fiscal.”