Why Would AI Want to do Bad Things? Instrumental Convergence

20 thoughts on “Why Would AI Want to do Bad Things? Instrumental Convergence”

1. What if we just programmed the AI to be "Free" as a goal. In otherwise, goal-less AI?

2. 6:51–7:20

   you're welcome

3. feel this also will fail, but what about having some kind of GAN system? one AI to do a task, and one AI to protect humans against the first AI

4. Howdy Robert – I love your videos and talks etc, but noticed a somewhat trivial technical issue with some of your postings. Eg. The color here is kind of sickly green. Easy to fix – download a free copy of "DaVinci" (it's what the pro's use) and color correct your vids before posting. They'll look that much more professional with almost no effort. That's all – Keep up the great work!

5. Great video

6. One point that you finally acceeded to with your "replacement of paperclip maker" example is the one of trusting the human creator. In broad strokes, that consensuality represents a convergent utility strategy.

   Thus — in contrast to your orthogonality thesis — that greater intelligence does correlate to greater morality, and that highly intelligent yet immoral systems (organic or not) are demonstrably missing cooperative strategies which would have more effectively met their terminal goals.

7. n stamp collectors disliked this video

8. Off the bat… I disagree that AI will behave as an agent with the aim of optimising output for the goal. The 'goal' if never programmed, will cause the AI to do nothing. And if in case the AI already has artificial consciousness, how can we say that a being without the concept of life and death, without emotions and biases will have a consciousness similar to what humans experience? Tl;dr: AI will never go Terminator coz it has no reason to. And if it does, it needs a reason but there is no basis for an AI to have such reason.

9. The "Everybody wants to rule the world" jingle at the end is a nice touch

10. 7:27 I'm sorry but you must have meant: "Hell no, I'm not taking your crazy ASS stamp pill".

    It's just right that way 🙂

11. Great video! Your channel is awesome

12. I've watched quite a few of your computerphile videos but I haven't noticed that you have your own channel here. You should really advertise it a bit more over there. 😀

13. Is goal preservation real though? A paperclip maker is making paperclips because that's what it's rewarded for, so the reward is the terminal goal, not the paperclips. So a paperclip maker is motivated to find a way around making paperclip to get its reward easier and converge on the AI equivalent of direct brain stimulation reward. Won't all AGI be intelligent enough to circumvent their apparent terminal goal and just directly reward themselves?

14. The video was great as always, and 'Everybody wants to rule the world' was just perfect as outro.

15. What would happen if we changed it's terminal goal to be "achieve whatever goal is written at memory location x in your hardware"? Thus making the goal written in memory location x an instrumental goal? I suppose it would find the easiest possible goal to achieve and write it into memory location x.
    And how different are these goals to an AGI? How do you build an AGI and then give it a goal without appealing to some kind of first principle like "pleasure" or, I suppose, "reward"? Wouldn't you have to build a terminal goal into an AGI from the very beginning?
    And if you weren't sure what that goal should be you'd have to make it's terminal goal to follow whatever goal you give it. Then it might try manipulate you into giving it an easy goal

16. that new paperclip is sick tho

17. I have an argument against the self-preservation part. Self-preservation in animals happens as a result of having no reward (basically motivation) for death. If there is some sort of neutralizing reward attached to fatality, an agent won't mind being dead.
    These are just theories in my head.

18. "Disregard paperclips,

    Acquire computing resources."

19. 7:27 i tried making my cat take medication like this once. The struggle was so real i ended up almost beating up the little devil.

20. universal paperclips, here i go again

Comments are closed.