You wouldn't be the first.
I had another forum member very frustrated because he couldn't plug his new harness into that connector, it just wouldn't fit! Turned out, after some back and forth, that he was essentially trying to push the female connector of the harness onto the back of the existing female connector that was still connected to the male connector. He had thought pulling out the plugs would just allow him to plug in the new female connector.
Once he realized his error, with some pictures and assistance from me, he separated the two factory connectors and his new harness snapped right into place.