At least 185 households across Sydney may have been exposed to asbestos by living on or near former James Hardie disposal sites the government has known about for at least a decade.

Some residents have lived with the health hazard unknowingly but Fairfax Media is aware of one council looking to make them pay for the clean up.

Parramatta City Council and the Environment Protection Authority last week door-knocked 185 "high priority" households, some of which have been told not to mow their lawns or dig in the garden.

Floods and construction work on the WestConnex project have unexpectedly revealed the presence of friable asbestos, the kind that more easily crumbles and becomes airborne, the EPA said.

The government's handling of the sites has been sharply criticised by the acting NSW Ombudsman, John McMillan, who said lives were being put at risk.

"This situation is unacceptable and must be dealt with as an urgent priority," Mr McMillan said in a report into the issue.

"Individuals who live on sites where they have found asbestos is present could understandably be very critical of government and say government has failed in their responsibility to provide proper health protections for them and others against deadly health risks," he said in a later interview.

NSW Minister for the Environment and Local Government Gabrielle Upton declined to be interviewed but said that the Heads of Asbestos Co-ordination Authorities would reassess the Parramatta sites.

Ms Upton is considering the acting Ombudsman's recommendation that the state government should pay for sites with significant asbestos contamination to be remediated or even buy them from owners.

Parramatta Council, which has jurisdiction for many of the sites, claims it has not put human health at risk.

Following questions by Fairfax Media, officials from the council and the EPA conducted 80 interviews from the door-knocks.

Among those affected by the problem are the Chaouks, a Granville family who have lived on one of the disposal sites for the past 28 years. Ali Chaouk said he was never told about the contamination until Fairfax Media contacted him, having seen his address on a government report.

Mr Chaouk said he had allowed his 16-month-old daughter to build castles out of dirt in his backyard.

"If you're late on your council rates you know about it, but if you're living on a property that's killing you or potentially killing you, they don't notify you," he said.

"Obviously someone is not doing their job, or doing it too well."

Ali's elderly parents still tend the vegetable patch which they call their "little sanctuary".

Photographs show the area has suffered significant recent flooding which Mr Choauk said had churned up the soil. His sister Manaya said she was "outraged" at the situation.

"I'd like an investigation ... It's something that should have been dealt with, and it hasn't been dealt with, and that's the reality."

The EPA's chairman and chief executive Barry Buffier said he had "complete sympathy" for the households affected.

"That's why we have been out there talking to them - talking to them one on one - explaining what the situation is," he said.

The NSW government will now arrange for the sites to be retested.

"Last year, with issues around WestConnex and flooding and other issues, these came back on to the radar, and testing by Paramatta Council indicated there was some friable asbestos on a number of sites," Mr Buffier said.

"That was not expected, that friable asbestos."

No one knows for sure how much James Hardie asbestos was left in the ground in suburbs such as Parramatta, Granville, Clyde and Camellia.

When council inspectors first began visiting the sites in 2007 they found fragments sitting in plain sight on parks, sporting ovals, car parks, a bowling club and a golf course.

The EPA ordered James Hardie to hand over details of the former disposal sites but found some locations could not be identified due to inaccuracies in the information provided by the company's compensation fund. The fund was set up in exchange for reduced legal liability in a 2006 agreement between the company and the NSW government.

Visible asbestos was removed and since 2007 some sites have since been capped, redeveloped, or incorporated into landfills.

But Fairfax Media spoke to residents of Prince Street, Granville, identified as an affected area in the council reports, and most of them had no idea of the asbestos problem.

Clyde Mooney's property backs onto A'Becketts Creek, where piles of asbestos sheeting and piping lie in cracked heaps along an embankment.

"It's just dragged on far too long," he said.

He bought his home in 2010 but says he knew nothing of the asbestos-contamination in the backyard until two years ago when council told him to clean up the weeds and asbestos fragments.

Having worked with asbestos before, he picked up fragments that filled an icecream container.

Parramatta City Council, which takes in the most former disposal sites, said it had not asked any residents to pick up asbestos and had not put health at risk.

"NSW Health has advised council that there is a low health risk for residents when asbestos material remains undisturbed," a spokesman said.

However, resident Sam Mitry has photographs of piles of asbestos sheeting jutting out on an overgrown embankment metres from his home, next to the M4 motorway.

Mr Mitry said he has been writing to the council and other authorities such as Sydney Water, the Roads and Maritime Service and the EPA since the mid 2000s about the problem.

"No one can wait this time, no one, believe me, no one," he said.

The council said complementary lawn-mowing services had been offered to residents at five properties and taken up by three where asbestos was found.

A council spokesman said the council would consider paying upfront for remediation works but would then charge residents, recouping the money when the homes are sold.

Mr Chaouk was considering legal action over his property.

"It's my castle," he said.

"You're supposed to be safe at home not worry about whether you're going to breathe in a deadly substance or be hurt."