For some reason everything on my blacklist shows up now
First I noticed this when I tried to blacklist a tag, but now posts with other blacklisted tags are popping up in my search resoults

I had this problem once. Try making sure you press the save button after you edit your blacklist

Nope, they still show up