This has been a really interesting thread. Most of these fish I have always avoided due to their reputation, or I've had them and really, really enjoyed them (severums and paradise fish!).
I would have to say, I do regret buying
Anomolachromis thomasi. Pretty much all of the literature I read said they are pretty peaceful (similar to kribs). These fish didn't read it obviously! I had two. Had to separate them because of beatings. Each one proceeded to pillage their respective tanks, harassing adult angels and gouramis and barbs. The smaller one eventually died, but I still have the larger one... in a mbuna tank!

Nary a nip on him btw.
And the lampeye killies. Awesome little fish. Not so awesome in my water. One by one they faded away or had fungal infections.
Purple Spotted/Rainbow Gudgeons- they were cool until they went carpet surfing. No one warned me that they jump!
The rest of the fish I only regret because once they passed I could never find anymore. I adored my pair of checkerboard cichlids and have been trying to replace them for almost four years now. Likewise, I had a pair of thicklipped gouramis, and can't seem to find any healthy ones.