Essentially, you have lost control over your collaborater's part of the song in that situation barring an agreement to the contrary.
Sounds bad, but consider yourself lucky. Could have been way worse.
In the absence of a carefully thought out agreement, your collaborator can insist that there is no OTHER direction you can go with a song than the one he and you went.
Which means that if you don't like the end result, and he does, you can't stop him from putting it out the way it is. . . (which is - from his position - what has happened.) He really can't stop you from using his input any more than you could stop him (or her if that is the case) from using it.
BUT. . . again, consider yourself lucky. Your co-conspirator can insist that you cannot take your half, and do something else with it (without an agreement to the contrary.)
He can say (as half owner of the song - which is what he/she is unless otherwise agreed) that you cannot modify the song he/she is half owner of.
It's unlikely (for any of us) that losing control of a song is going to keep us from "breaking through." Far bigger problem to have our name irrevocably attached to something we/you don't like, and don't want to be associated with. (Again, this is what your collaborator is feeling about this song.)
Point is, have a COMPREHENSIVE agreement in place if you care deeply about these things. There is no safe way to collaborate without it.
Not that I follow that advice myself. . . but I have no illusions that any one of my songs is so important that losing control of it would be a tragedy.
I hope that helps.