Gah, I need a stronger phrase

I can see why you're frustrated. The phrasing feels a little awkward, though it's grammatically correct.

toi could always go with the simple "wished you'd never met him/her" (don't know the gender of this "partner"). In a way, it ups the stakes. Going from wishing toi weren't partners to wishing you'd never met.

Or, toi could try reversing the order.

"I don't think so. I'm sure there were times where toi wished toi had a different partner. He pulled lots of stunts, didn't much care for authority, often defied you, heck, I wouldn't blame toi if toi hated him a little sometimes."

Something like that. Reversing sentence order can be helpful. A good tactic is to print it out, cut out your sentences, then déplacer them around on a flat surface. Also works with paragraphs, if you're not sure where to put them. toi could also give it a day, clear your head, and come back to it. That always works for me when I'm editing.