Latex usually contain phthalates (a compount used to keep the latex soft) which is now believed to be carcinogenic (cancer causing). Latex sex toys and condoms can be very hazardous to your health. Aside from the potential for developing cancer from the phthalates, allergic reactions or sensivity can develop over time and not be obvious with first use.
Symptoms range from minor burning to frequent urinary tract and bladder infections to anaphylactic shock resulting in death.
The answer is to use 100% silicone toys and non-latex condoms. In the United States, as long as sex toys contain at least 10% silicone, they can be labeled as 'silicone' on the package, but the remaining 90% can be anything- including latex. Make SURE that you are getting 100% silicone.
The safest material available is VixSkin. Toys made from VixSkin are available from Good Vibrations. You can also find latex free condoms and safe lubes at their website.