Since 2007, the number of full-time PE teachers with permanent contracts has dropped by 7% to just under 1,500.
Schools have made up the shortfall by appointing scores of extra PE teachers on temporary contracts, most of them part time.
Teaching unions and opposition politicians say they are no substitute for PE staff with secure jobs and accuse the government of cutting corners when it comes to physical education.
They claim that puts at risk the chances of building a lasting legacy from Scots’ sporting success in the London Olympics and the 2014 Commonwealth Games in Glasgow.
Figures released under freedom of information show in 2007, when the SNP came to power, there were 1,572 full-time, permanent PE teachers in Scotland.
The most recent figures show this has dropped to 1,465.
The number of part-time PE teachers on permanent contracts has increased by three to 403, giving an overall fall of 109.
But there has been a significant increases in the number of PE teachers on short-term contracts.
Full-time temporary PE teachers increased from 257 to 332 over the period while part-time PE teachers on short-term contracts went up from 172 to 258.
Overall, the use of short-term contracts has boosted total PE teacher numbers by 52.
Ann Ballinger, General Secretary of the Scottish Secondary Teachers Association, said: “We’re a very unhealthy country, the only way to promote our health is to provide young people with the skills they need.
“That doesn’t come with reducing permanent PE teachers.
“A member of staff with a permanent contract can be expected to put more energy into their job.
“It’s vital young people have continuity as well.”
Many temporary contracts only lasted for a term or two, leaving teachers wondering where their next paycheck will come from, she said.
“It takes time going into a new job to settle in.
“How much longer if you have to learn the names of over 100 young people, their skills and what motivates them?” she asked.
Scottish Conservative education spokeswoman Liz Smith said: “This is a very unsatisfactory state of affairs, particularly in light of establishing a workable and lasting legacy for both the London Olympics and the forthcoming Glasgow Commonwealth Games.
“The SNP failed to deliver on its 2007 promise to deliver two hours PE per week per pupil, and it is now presiding over a declining number of qualified PE teachers in our schools.
“This is not good enough.”
She continued: “Many of our Olympic medal winners spoke about how they were inspired at an early age by their PE teachers so no-one should doubt the importance they have in every school.
“Their specialist skills are invaluable and we need more of them not fewer.
“The SNP needs to take immediate action to provide far more sporting chances for our young people.”
A permanent full-time PE teacher can expect to earn up to £34,000 a year.
The SNP has previously faced criticism it missed its target of each pupil having two hours of PE per week.
Figures released in June this year showed many schools were still falling well short of the target.
Only four council met the target across all of their schools: East Ayrshire, East Renfrewshire, South Ayrshire and West Dunbartonshire.
Sport minister Shona Robinson said at the time: “”We are committed to provision of at least two hours per week of PE in primary schools and at least two periods of PE in secondary schools for pupils in S1-S4 by 2014.
“While we still have some way to go, these statistics demonstrate good progress.”
The SNP’s 2007 manifesto stated they wanted to: “ensure that every pupil has two hours of quality PE each week, delivered by specialist PE teachers”.
The data does not include headteachers as well as teachers who teach PE as another subject.
In March this year the Scottish Government unveiled a £6million funding boost to help meet the target on PE lessons.
Larry Flanagan, EIS (Educational Institute Scotland) General Secretary said: “It is ironic that these figures have come to light so soon after the Olympics and the political hype of figures such as the Prime Minister, David Cameron, around inspiring young people to engage in sport.
“In Scotland, as a result of budget cuts, many local authorities are failing to deliver the target of a minimum of two hours PE per week for pupils.
“This reduction in the number of full time PE teachers simply underlines the gap between rhetoric and reality.
“The truth is that tighter staffing levels lead to reduced opportunities for young people.”
A Scottish Government spokeswoman said: “The overall number of PE teachers employed, whether on temporary or permanent contracts, has remained consistent.
“What is more, Scottish schools are making good progress towards delivering our 2014 PE target.”