First person narrators

I’m not really a fan of stories told by a first-person narrator.  I know that, to some people, they make the story seem much more personal, but I just can’t suspend disbelief enough to accept a narrator who can somehow remember the ins and outs of every single conversation he/she has ever had.  There are a few books where the story is so compelling that I’m willing to overlook the first-person narration: N. K. Jemisin’s Inheritance Trilogy, Robert Graves’ I, Claudius, and Patrick Rothfuss’ Kingkiller Chronicles are the ones that spring immediately to mind.

Regular readers of my blog know that I’m not really a fan of H. P. Lovecraft.  But I don’t see his frequent use of the first person as one of his problems.  In fact, he’s usually able to pull it off a lot better than other writers because he doesn’t use a lot of dialogue, so his narrators’ reminiscences seem more believable.

What do you think of first-person narrators?  Am I alone in my distaste?