Well, I would not really worry too much from the sounds of it. I would simply change your password to something that someone cannot guess and NEVER tell your password to anyone. The only time you will give out your password is never. Not even if Yahoo calls you on the phone and asks for it. They won't do that by the way.