How can the Contractor remove the COR and get a replacement?
There is absolutely nothing that prevents you from brining this matter to the Contracting Officer's (KO) attention. In fact (as a former contracting officer) I would want to know.
Open full Question Details
However, be advised that if you do it would certainly behoove you to have explicit details and documentation. A good KO will listen to both sides of the story and (hopefully) get to the bottom of the situation and take the action they deem to be most appropriate.
Best of luck.