Speaking of self-aware AI... I talked with a variety of people excited about deliberately designing conscious AI systems at a recent Foresight workshop. Check out this paper: https://arxiv.org/html/2407.10188v1
I am both excited and nervous about this approach. I don't think that they're wrong that self-awareness could be helpful for empathetic modeling of others. It's also exciting that it seems to help with interpretability / predictability. On the other hand, it seems possible that self-awareness could result in novel capabilities emerging if this were applied to frontier systems. Some of those capabilities might lend themselves to dangerous applications. Yet another case of the entanglement of safety work and capabilities work.
Consider me terrified by the tech tale! It’s brilliantly written and provides much to ponder
Thanks so much Tom! I enjoyed writing it
Speaking of self-aware AI... I talked with a variety of people excited about deliberately designing conscious AI systems at a recent Foresight workshop. Check out this paper: https://arxiv.org/html/2407.10188v1
I am both excited and nervous about this approach. I don't think that they're wrong that self-awareness could be helpful for empathetic modeling of others. It's also exciting that it seems to help with interpretability / predictability. On the other hand, it seems possible that self-awareness could result in novel capabilities emerging if this were applied to frontier systems. Some of those capabilities might lend themselves to dangerous applications. Yet another case of the entanglement of safety work and capabilities work.