More Republicans are realizing that the ex-president and his poor judgment are a hindrance to the party.