The changes to your breasts could cause anxiety. However, not all changes to your breasts are caused by breast cancer.