I think the arousal is based ENTIRELY in the psychological...
Like anal sex can feel great the same way taking a shit does, but a tongue won't give you the same physical feeling as something long and hard trying to turn your asshole inside out.
Rather, the sheer fact that somebody is licking the hole you shit from is, in and of itself, simultaneously empowering in a slightly sadistic sort of way, and a symbol of love/trust that somebody's willing to taste your shithole.
I'm all for it, both ways. Whatever you do to her or him, you should be willing to let them do TO you.