It is generally considered best to use clean ones because you can never be certain what sorts of chemicals may have been used on the plants before.
But I have put used pots into my tank and have had no problems. Best thing to do would be to boil the pot at least several times and then leave it to soak for some days. Replace the water you are using to soak the pots several times and I'd say that most anything harmful would be rinsed away.
They have a clay pot in the fish tank in my little sisters class room and the fish are as happy as can be. I thought it was a good idea my self. I'd say boil it, that seems to kill most of anything harmful.