When truth isn't enough

True beliefs are pretty darn useful, so epistemic rationality contributes pretty strongly to instrumental rationality (i.e. the one that matters by definition).

Interestingly, as I've become more aware of what I want out of life and see more long term challenges, epistemic rationality has begun to seem less terminally important - yet still very instrumentally important. It's just that the purpose of epistemic rationality is much more gut level clear to me now - the reason to have a really good map is to get places, not just to hang up it up on your wall and admire for its accuracy.

Additionally, I've become much less hung up on "literal truth" (that is, truth as straightforwardly explicitly/denotationally interpreted) since I realized that that's not really how people work - so obsessing on that level is a bit silly. Once you really grok the idea that true beliefs are for navigation, then you can use all sorts of funky encodings - as long as you keep track of them. If you take a bunch of magic mushrooms and the world starts bending, it's not a simple map, but the information is still mostly there - and you don't run into problems until you forget to walk the same bendy that the sidewalk bends. Try to walk straight and you're in for trouble. And phrases like the "most amazing show ever!" are totally acceptable too - as long as you realize that it really means "I'm experiencing awe and very much enjoyed the show".

Putting these together can explain a lot of "irrationalities" people have where they say stuff that is just blatantly not true and don't seem to care when you point it out.

Say you're trying to convince someone that their religion is false. It's not going to be easy, even if the evidence is just slam dunk. You're looking at it as if the beliefs are there to be explicitly true - and hell, maybe they look at it that way too. But the purpose of religious beliefs isn't to actually have the correct metaphysics - that's not where they get the value and those aren't the considerations that condition for or against it. The purpose is to help them fit in with their social circle, and to keep them from having to acknowledge their fear of death (among other things).

So of course your logical proof of the nonexistence of god wasn't persuading many people - your arguments are just irrelevant from the perspective of how they should navigate. It's like someone's talking about "the most amazing show ever!" and you try to condescendingly explain how that is a priori very unlikely... and then he looks at you like you're socially retarded - that's just not what it's about. Steel man it. Forget the explicit meaning and ask what purpose the belief could serve in context of their mind.

Think about it - if they lost their belief in god, the religious person might lose their social circle, feelings of comfort that everything is going to be okay, grounding for their morality... That's a world of hurt to be thrown into without having a solid back up worldview. It would be a bad idea to just wholeheartedly accept new beliefs without first knowing how the hell they're going to navigate things with it.

If you just take someone's religion from them and give them a half-assed pretense of a solution, they'll be uncomfortable with it for a month or two, then they'll find a reason to be religious again. I've done that experiment. If you take away someone's fear of being honest with themselves about religion, then they'll tell you it's about social pressure and their fear of death - I've done that experiment as well. The conclusion is clear - religion is not simply a bi-stable belief. It's not the case that they're religious by simple circular reasoning and as soon as you break them out, they're free. There's a reason people are religious - even if the particular religion does prove to be extraneous and it does not regrow if surgically removed.

And just like people will refuse to accept true beliefs when it'd mess them up, people will accept very obviously false beliefs if they feel helpful. When an atheist becomes religious, it is most likely because things got tough. My best friend's brother has "always" been religious - ever since he had a near death experience. I personally found myself praying to a "probably non-existent simulator god" (and laughing about it) when things got terrible for me. I felt the pull, but didn't fall in. Not because I have more magic "rationalist virtue dust" that makes me The Chosen One who cares about Truth, unlike others. I didn't fall in because somewhere or other I got the idea ingrained that will-full self deception is hideous in the long run - and so I flinched away even harder. Just like anyone with that alief would.

I've largely (but not completely :P) moved past being irritated by people's unwillingness to accept hurtfully true beliefs and come to recognize that it as a good thing. I mean, sure, it'd be amazing if everyone really grokked that everything self deception can do for them can be better replicated by having the same object level maps and keeping precise track of how they relate to each other - and then piecing things together on their own. It'd be really awesome... but they generally don't. And given that, they shouldn't just accept these hurtful truths - not until they have a way to better navigate the world with their new beliefs. The problem is that they lack foresight and the ability to rework painful truths into nonpainful and useful truths.

So if you want to help people become more able to achieve their values, then don't give them some complicated map that they aren't going to understand well enough to use more effectively just because it's "more true". Instead, give them a more useful map that is locally inviting - and they'll actually take it! If you're worried about limiting this person's growth by getting them stuck in an oversimplified map, then make sure that they are aware of the simplifications. Just stop defending yourself with "but its TRUE, so they should believe me and be thankful!" - or you know.. don't. Keep doing it until you get what I'm saying here and how you can do better without resorting to "shoulding" people.

That's why certain beliefs are "immune to reason". No amount of logic will persuade someone if you're arguing about the wrong thing. Figure out what it's really about, and although the real meaning may all be between the lines, all of a sudden people will reason again.