Clean-up efforts at 28 Defence Department sites have cost taxpayers $1 billion so far, according to the government.