The body of the male Staffordshire bull terrier was found by a local resident at an address on Whitson Road in the west of the city on Saturday morning.
The poor pooch weighed only half what he was supposed to and was so starved that his bones were clearly visible.
The staffie was described as being small, of dark brindle colour and around 10-years-old.
The dog had pressure sores and smelled strongly of urine which suggested he had been kept in extremely bad conditions.
Chief Inspector Paul Anderson, of the Scottish SPCA, has appealed for anyone with information to come forward.
He said: “This case is horrendous and among the worst we’ve investigated in recent years.
“We haven’t carried out a post mortem yet but the immediate indications are this poor dog starved to death and hadn’t been dead long.
“The dog weighed only 8.75kgs, less than half the weight he should have been.
“He had very little muscle left on his body and was so emaciated it’s as though skin had been pulled over his bones.
“Sadly, he’ll have been in a great deal of pain as when a body starves to death its organs begin to fail.
“The dog also had pressure sores and smelled strongly of urine, which suggests he’d been kept in squalid conditions.”
The small staffie had no identification tag or microchip.
Chief Inspector Paul Anderson added: “The dog was found in a front garden next to some bushes and as there are no gaps big enough for him to have crawled through it seems likely he was thrown over the fence when he was either dead or dying.
“We can’t be sure how long he’d been there but the information we have suggests he was dumped when it was dark, either on Friday night or in the early hours of Saturday morning.
“We’re asking the public for help identifying who is responsible for causing him such terrible suffering.”
Potential penalties for causing an animal unnecessary suffering include a prison sentence, a substantial fine and a ban on keeping animals.
Anyone with information is asked to call the Scottish SPCA animal helpline on 03000 999 999.