Monday, October 31, 2011


I intend to use this blog to get my thoughts out in a way that helps me solidify my own understanding of the principles of cognitive engineering and gives others a starting place to learn what I've learned so far. I plan on covering topics like the theory of how it works, what looks possible, and what good general strategies can be used to achieve these things.

I intend to cover a lot of concepts quickly in a very information dense way with a lot of links for further explanation. Depending on your background, I can't promise that this blog will be easy to follow, though I will do my best to be clear. You'll probably have to follow the links, put some real thought into it, and generally "do your homework" to get the most out of what I have to say.  I'd appreciate questions and feedback, since it's not at all easy to know if I'm doing a good job bridging the inferential distance.

I'm going to try to build from the basics to explain hypnosis as something that naturally falls out from other things we know about the brain, not as some mystical state where weird things happen. We will be asking "how far can we take this?" instead of "what is this 'hypnosis' thing?".

I'm going to be linking to LessWrong a lot, not because reading the posts there always gives "Aha! I didn't know that!" feelings (sometimes it will) but because it just makes sense at every step and then you find yourself not making mistakes that you didn't realize you were making, and seeing things in different ways.

While there are obviously many applications for learning how to use your own brain and powerfully influence others, I'm just going to mention a few goals I'd like to work towards that I have not seen done by anyone else yet.

  • LessWrong has done a great job explaining the ways we should think and the common failure modes, but actually implementing these changes has largely been left as an exercise for the reader. I'd like to attack that part of the problem here so we can effectively reach more people. Effectively teaching rationality (and teaching people how to teach themselves rationality) requires knowing how to actually implement new thinking patterns at the 'automatic' level. Hopefully, this can be done effectively enough that you can have a conversation-length interaction with someone and install the minimal amount of necessary cognitive habits for them to stay stably on the path of learning the rest (aka "rationality seed").
  • Another intriguing use is in testing rationality. One could give PHS for disgust, followed by amnesia for that PHS, and then test for undistorted morality. Or perhaps induce anosognosia and see how aspiring rationalists respond.
  • We could also play games like actually doing the sleeping beauty thought experiment for fun/intuition pump. And maybe some of those other whacky decision theory thought experiments too.
  • Conscious locus of control appears to be almost completely unrelated to brain architecture, and instead just relies on how well you know how to use your own brain. Increasing this locus of control is valuable and doable.
  • With increased locus of control, self modification becomes easier and more improvement is possible. Of course, this comes with increased risk of the the valley of bad rationality!
  • Standard therapy stuff has been done with hypnosis since forever, but there's room for improvement. I'd like a method for systematically inducing deep trance and giving specific direct suggestions. Deep trance with direct suggestions can be a much more powerful force for change than indirect light trance stuff, but are typically utilized in a naive manner that doesn't address stability. Understanding the structure of typical problems and the types of suggestions needed for stability is critical for good therapy. I'd like to be able to solve it like engineers have power supply design solved. Solved to the point where you could write an engineering textbook on it. Solved to the point where you can teach a smart grad student to implement your algorithm without having to actually solve any problems himself. 
  •  Breaking people out of tightly held beliefs. People can get sucked deep into some nasty attractors that aren't just attractors because they're true or reasonable. There are plenty of attractors that set up camp and defensively fight off attempts to remove them, even damaging the host if necessary.  I think it'll be a fun but doable challenge to learn how to break them out.


  1. >I'd appreciate questions and feedback, since it's not at all easy to know if I'm doing a good job bridging the inferential distance.

    You're not doing a good job of bridging inferential distances!

    1. Thanks for the feedback! Can you be more specific so I can try to fix it?

    2. Well broadly speaking the posts try to cover too much detail in too little time (space?). Try to include more specific examples where you break it down step by step and explain how the theory corresponds to the anecdote.

      Like with this, say

      I've read and mostly understood around 70% of the Yudkowsky Canon, and I can't make head nor tail of what you're trying to say here. What do the "top" and "bottom" layers actually correspond to? Why do priors come from above and data from below.

      Brains are optimised for understanding narratives from which we can derive general rules (see fairytales -> morals). So give me a story and annotate the theory :)

    3. I *am* aiming for high concept density. My goal was to explain enough to show you where to look, not to walk you through everything in full detail. If I'm not getting this far, I really need to add more. Perhaps it's worth more complete explanations anyway.

      Things like "top/bottom" are the kind of abstraction that I worry a bit about. If this kind of thing is a repeat offender, I'll try to fix that. In this case the "bottom" is down near sensory perception and "top" is after more layers of abstraction. For example, the following list is sorted top to bottom: "I am feeling pain">feeling pain>sensory neuron signals.

      I had hoped the wiki link to top down processing was enough

      I agree about the more stories bit. It *was* intentional, but misguided.
      Thanks again :)

    4. Thanks for the explanation. Though you might want to reconsider just how highly concept-dense you want to make the posts. Regardless of how efficient it might be, people aren't going to bother reading through unless they're extremely interested.

      There's the signalling aspect as well. Anyone who comes into here thinking that this is just another crank talking about hypnosis is going to have that belief strengthened, especially when you link to LW every other sentence.