[content] => 
    [params] => Array
            [0] => /forum/index.php?threads/safeties-must-be-built-into-ai-ml-is-anyone-doing-it.17707/

    [addOns] => Array
            [DL6/MLTP] => 13
            [Hampel/TimeZoneDebug] => 1000070
            [SV/ChangePostDate] => 2010200
            [SemiWiki/Newsletter] => 1000010
            [SemiWiki/WPMenu] => 1000010
            [SemiWiki/XPressExtend] => 1000010
            [ThemeHouse/XLink] => 1000970
            [ThemeHouse/XPress] => 1010570
            [XF] => 2021370
            [XFI] => 1050270

    [wordpress] => /var/www/html

Safeties Must be built into AI/ML, Is Anyone doing it?

Arthur Hanson

Well-known member

Safeties that are already built into many of the things from transport of all types to tools, must be built into AI/ML from the ground up or it could easily present dangers in ways we can't even imagine. This will have to become part of the foundation of chip design so it can't be erased or altered. This should be started ASAP for with the speed of progress in this area it could be deadly for all in ways we can't even imagine. Any thoughts about the best way to implement safeties and standards to protect from the dangers AI/ML could present should be put forward now and I feel SemiWiki is one of the best places to start this discussion. I am not a chip designer, but feel safety must be built in and a design parameter from the start. This may even become a specialty in itself. Any thoughts, comments or observations on this would be appreciated. Also are any companies now taking steps to implement or even write safety protocols for AI/ML?
The dangers of generative AI have little to do with physical chip designs, because except for the case of FPGAs the functioning of a chip can't be physically altered. But, if you'd like to keep yourself awake at night, it might be possible for some super-AI intelligence to reprogram FPGAs surreptitiously, and since FPGAs are widely used in aircraft avionics perhaps the super AI could cause tens of thousands of planes in the air in the world to crash at the exact same instant.

Worse, even state machine processor-based systems often include microcode or firmware loaded from EEPROMs to alter instruction processing, like Intel CPUs:

Because it's documented on the internet for anyone to see, some super future version of an LLM-based intelligence could read those pages (and numerous others, especially if it got behind Intel's firewall) and figure how to reprogram the instruction processing of the billions of Intel CPUs on the planet to malfunction in a subtle way that would cause a widespread disaster.

See how convincingly horrible all this sounds? I just made all that up in about five minutes based on public information, and it has the same alarming tone of so much of the other click-bait on the internet that sounds entirely plausible, and is mostly nonsense. If it was that easy to alter Intel CPU microcode on existing systems some group of humans would probably have already done it, demonstrated it once, and then demanded a monumental ransom (in Bitcoin, naturally) from every government not to do it on a wider scale.

And you wouldn't even need an AI to implement a nefarious plan, because you could just have a criminally-minded human asking an AI how to compromise a computer system's security features. I suppose to subvert that possibility you could try to mandate an extrapolated AI equivalent of Issac Asimov's Three Laws, which would not allow an AI to be knowingly used for criminal purposes. (Interesting, isn't it? Asimov was thinking about similar problems for robots about 80 years ago.) But what about an AI that decided to create another AI that ignored the Laws? This discussion is easily recursively stupid.

To answer your last question more specifically, the safety protocols would be useless in AI software, because someone could just ignore them. The security mechanisms would have to be built into every device used for operational purposes, the way that Intel and AMD CPUs have trusted execution assists to protect the integrity of executing software. (Amusingly, I've read AMD uses an Arm core to run the firmware in its Platform Security Processor.) However, since no human-designed systems are ever perfectly secure, a super-AI could probably invent ways around the security mechanisms.

I think that the reason all of this super AI worry has become such a frenzy lately is because lots of people have discovered ChatGPT can write poetry and music, pass college entrance exams, and even write convincing love letters in the style of Shakespeare. And now lots of people are questioning the meaning of life as humans, and wondering if the end of the world is near because we think it might be possible to create entities a lot smarter than we are. I'm less impressed. Human thinking is entirely influenced by motivation, which is driven a lot by our biological needs and frailties. What motivations would AIs have?
Mr. Blue, why do you digital guys make everything so complicated. My brain hurts.
Because of most us digital guys aren't smart enough to build things that obey the laws of physics like you do. So we placate our wounded egos by designing over-complicated solutions based on legacy architectures that only have to follow the laws of CPUs, which are much simpler than physics and can be perfectly understood.
Last edited: