The Jung-Freud problem (which you mentioned before in another podcast) is very interesting! The trouble is, not only can you not trust the LLM, but you can't even trust the pre-training data, at least not with the way that pre-training is conventionally done today. A similar issue could arise in fictional, hypothetical, and historical scenarios. For example, it's good for ChatGPT to think the world is round, but in a fictional setting with a flat world, that'd be undesirable. So this is a very generally applicable problem.

extremely interesting. one note on the ending: it is possible that these LLM derived minds will not be conscious. eg David Chalmers guesses that there is a 10% probability that current models are conscious and a 20% probability that future models will be conscious. Ofc no one really knows, but the prospect of getting replaced by completely unconscious beings seems unappealing to say the least...

Pretty interesting and provocative transhumanist perspective towards the end

That ending for me is very pessimistic, but I’m guessing Steve would describe it as optimistic. Regardless, it seems more or less inevitable

You said it has implications at the level of the universe that so soon after our ape brain evolving we can build AGI. What did you mean? I presume you mean that biological advanced intelligence is probably rare and transient and that the universe should be filled instead with AI. It so where are they?

👍👍👍

Well that was pessimistic