Bronco Webinar 800x100 1

16nm HBM Implementation Presentation Highlights CoWoS During TSMC’s OIP

16nm HBM Implementation Presentation Highlights CoWoS During TSMC’s OIP
by Tom Simon on 09-29-2016 at 12:00 pm

Once a year, during the TSMC’s Open Innovation Platform (OIP) Forum you can expect to see cutting edge technical achievements by TSMC and their partners. This year was no exception, with Open-Silicon presenting its accomplishments in implementing an HBM reference design in 16nm. It’s well understood that HBM offers huge benefits in terms of bandwidth and lower power consumption over alternatives such as DDR. With the advent of the JEDEC HBM Gen-2 specification, both density and data rates have gone up significantly. In the 2, 4 or 8 stack configuration HBM Gen-2 supports up to the 8 Gb per stack. In addition, data rates are going up to 1.6 Gb per second or even up to 2 Gb per second per pin.

According to open silicon 16 nm FinFET is the key to unlocking the full benefits of HBM. 16 nm FinFET processes can potentially reduce power by 50% and boost performance by the same amount relative to 28 nm. However, to implement these HBM designs a complete ecosystem is required which includes the die, interposer, assembly and packaging. Open-Silicon paired SK-Hynix’s HBM die stack with a TSMC 16nm/2.5D/CoWoSTM ASIC implementation. CoWoSTM is TSMC’s 2.5D interposer technology. In fact, TSMC has been making a big deal out of all of its advanced packaging options.

TSMC has been innovating their packaging options and is seeing the results in their business. It’s widely understood that TSMC scored a design win with the Apple A10 that is used in the iPhone 7. So clearly packaging technology is becoming a significant differentiator for foundries. We can still expect to see much more creative and diversified offerings in the already exploding packaging market.

But now back to Open-Silicon and their HBM implementation at 16nm. HBM is a good choice for products where there is threefold pressure on form factor, bandwidth and power. These applications include a data centers, networking, radar, virtual reality, gaming and cloud computing. In the target design Open-Silicon was able to replace 24 DDR3 1600 (x16) with 1 HBM stack, the power consumption went from 1.0 mW per gigabit to 0.33 mW. At the same time the data rate climbed from 4 GB/s up to 256 GB/s.

According to Open-Silicon’s Bupesh Dasila, Engineering Manager for Silicon Engineering, some of the major challenges for implementing a 2.5 D SIP using HBM are: having a scalable PHY architecture, designing the 2.5D interposer, managing the custom die-2-die IO’s, and testing the completed system. There were 1840 routes on the interposer that were up to 5 mm in length connecting the HBM to the SOC. To effectively shield the signal lines from cross talk, ground wires of 0.5um were placed 2.1um to the side of each signal wire. This left 2.1 um for each signal line. The signal wires were 0.85 um thick.

After his presentation Bupesh told me that they did extensive modeling to verify that electrical characteristics of the signals on the Interposer. Below is an example of some of their Interposer SPICE simulations. In addition to the PHY design, Bupesh and his team designed the IO for the 16nm die that communicated to the HBM memory module.

Open-Silicon’s roadmap includes heading to 7nm with this approach, but they also are going to be validating the Gen2 HBM on a 28nm design as well. The results of the 16nm chip were impressive, with data rates of 2 Gb/s per pin using their custom IO’s and PHY. They were diligent in adding testability features as well. They added probe pads and included loopback to help if needed with issue isolation among system components.

Open-Silicon emphasized that they are ready to deliver solutions offering the potentially game changing benefits of HBM today. Admittedly this is new technology that requires more up-front costs, nevertheless, the area savings at volume are significant. Also the cooling and power improvements will change the equation for cost of ownership of the finished products they are used in. More information on Open-Silicon’s HBM expertise is available on their website, here.


Power Exploration at RTL Design with Mentor PowerPro

Power Exploration at RTL Design with Mentor PowerPro
by Bernard Murphy on 09-29-2016 at 7:00 am

There was a comment recently that design for low power is not an event, it’s a process; that comment is absolutely correct. Power is affected by everything in the electronic ecosystem, from application software all the way down to layout and process choices. Yet power as a metric is much more challenging to model and control than metrics like timing and area since it depends on factors across that range, particularly activity, which in turn is heavily dependent on use-cases.

Still, a practical design methodology can’t iterate over so wide a span, so each stage aims to optimize – using realistic use-case data – within what can be reasonably controlled (or at least determined as constraints to feed forward) at that stage. One important observation helps: it has been amply demonstrated that power-optimizations made at higher levels in the system have a bigger impact than those made at lower levels. So when the architecture has been fixed, you are going to get the biggest bang for your buck through RTL optimizations. Which is not to say that you won’t polish all the way down to layout if you’re going after picowatt savings, but the Pareto principle suggests you should put most of your effort into the RTL. Unless of course you are unable to change the RTL, which can happen if you want to avoid re-qual.

What are the options at RTL? Architecture and target process are already fixed. Your choices are to reduce leakage in areas that are not very performance sensitive by controlling the Vt mix, to power-down islands of logic during periods those functions are not needed, to reduce redundant/useless activity by gating clocks and related signals, and to scale down V[SUP]2[/SUP]f power (again in islands) where feasible through dynamic voltage and frequency scaling (DVFS). Depending on where you are starting, this bag of tricks together can give you 30% or more reduction in power, or as little as a handful of percent reduction if you’ve already significantly optimized the design. Energy (integrated power over time, which is important for battery life) is mostly controlled by how long you can keep most of the logic in a low/zero power state and how much power is consumed in turning it back on.

Of course, doing all of this stuff comes with costs. Even clock gating adds at least one cycle of latency in turn-on time. Power-domains can be significantly slower to reactivate because they have complex power-up sequences (turn power on, reset/start clocks, restore state from retention registers). And then there’s the issue of what happens when something is switching on or off and something else wants to talk to it. This requires work to prove either that such a problem can never happen or that there is handshake logic in place to ensure that these cases will be handled gracefully. And all of this added circuitry consumes space, may create new timing problems and adds more complexity to verification. All of which means that while you may find lots of ways you could reduce power, they’re not all going to be equally desirable when balanced against other consequences of making those changes. PowerPro’s state of the art solution provides a way to start this analysis by considering all options, automated and guided for power-saving and interactive exploration of these options with feedback on power reduction and cost metrics.


Mentor makes the point that all of this optimization could be handled more effectively if regular RTL designers were to get more involved in optimization for low power. Today this objective is generally handed off to power experts who, while skilled in that domain, necessarily have limited understanding of total design objectives, leaving you wondering what gets left on the table. However in high-pressure design schedules it’s sometimes difficult to see how design teams can significantly rework assignments. Perhaps instead PowerPro can enable a more comprehensive discussion between block, subsystem and top-level designers, the power experts and the verification engineers in debating which power-saving options are most worthy of consideration in the design. Doing this can start with the power expert filtering through the a range of possible directions to boil down to a limited set of most promising scenarios.

At that point being able to interactively flip through scenarios (enabled by nearly real-time performance in PowerPro option what-ifs) would enable optimal choices made by the collective product team, each bringing their own area of expertise to consider a scenario from bandwidth, latency, area, power, performance/criticality and verification complexity perspectives.

You can read more detail on PowerPro in the link at the end of this blog. A couple of interesting questions came up after the Webinar. One touched on how accurate dynamic power estimation is without a SPEF for the design, the other concerned vectorless estimation. Mentor answered both questions well in my view. First, RTL power estimation is good for relative comparisons, which is exactly what you need it for (is this option better than that). Absolute correlation with silicon is not the goal, nor is it likely possible before the design is fully implemented. Second, RTL block designers usually want to know about vectorless estimation because they don’t have much in the way of vectors. Vectorless can give you ballpark estimates but I wouldn’t invest a lot of time in power-saving tweaks based on this analysis – the error-bars on this kind of analysis can easily swamp potential power-savings.

The Mentor Webinar can be found HERE.

More articles by Bernard…


It’s a heterogeneous world and cache rules it now

It’s a heterogeneous world and cache rules it now
by Don Dingee on 09-28-2016 at 4:00 pm

Cache evolved when the world was all about homogeneous processing and slow and expensive shared memory. Now, compute is just part of the problem – devices need to handle display, connectivity, storage, and other tasks, all at the same time. Different, heterogeneous cores handle different workflows in the modern SoC, and the burden of cache coherency is shifting. Continue reading “It’s a heterogeneous world and cache rules it now”


Intel Foundry Rounds Out IP Lineup With ARM at IDF 2016

Intel Foundry Rounds Out IP Lineup With ARM at IDF 2016
by Patrick Moorhead on 09-28-2016 at 12:00 pm

There are always debates on who does what best in the semiconductor industry, but most agree that Intel is the best in process and transistor technology. This leadership has served the company extremely well over the last few decades and allowed them to reach a position of dominance in the PC and server semiconductor markets. In order to build such a leadership position, Intel has repeatedly invested billions in true R&D (with a focus on “R”) to invent the next big transistor and to reach the next big process node. In addition to those investments, Intel has to spend billions of dollars building new fabs to accommodate these new process nodes and technology. This investment not only serves their own designs but since 2010 for foundry customers.

At the Intel Developer Forum, Intel announced new customers, customer updates and new ARM Holdings-based IP for their custom foundry which I believe could really open up a lot of opportunities for their foundry business.

Intel Custom Foundry a natural extension of the fab
The fab business always has been a business of scale, so it comes as no surprise that Intel would logically want to take on the fabrication of others’ chips to increase scale. Intel has competition from Samsung, TSMC and GlobalFoundries in the foundry space, but until 2010 Intel only produced their own chips from their own designs. That changed when Intel took on Altera, which they eventually acquired, but now the company is expand their foundry capabilities and customer base even more. To be more competitive with the competing 10nm processes from Samsung and TSMC, Intel is beginning to take a bigger role in the foundry business with their Custom Foundry which was outlined today.

New foundry customers and details on others

Today at the Intel Developer Forum, Intel Custom Foundry announced new customers and details on previously announced customers ranging from 22nm all the way down to 10nm, including Achronix, LG Electronics, Netronome and Spreadtrum. Their products range from networking accelerators and FPGAs to mobile SoCs.

This is part of a journey that Intel started in 2010 to enable multiple fab customers, which involved making a lot of changes to their standard tools and integrating other companies’ IP. This meant building chips with the help of IP from leaders like Synopsis, Cadence and Mentor Graphics that led to Intel building out their ecosystem of design tools and supported IP. This was a huge change from prior decades where it was Intel designs, Intel tools in intel IP in Intel fabs.

New ARM-based IP
The lessons learned from building chips for others has led Intel to reach another major milestone today with their capabilities as a fab. That milestone announced today at IDF is the ability to now fab processors for ARM Holdings customers using Artisan IP. There’s an ARM processor on the Altera FPGA they fab, too.

The ability to fab ARM Holdings processors in the traditionally x86 Intel fab comes from a partnership between Intel and ARM that includes ARM’s Artisan Physical IP platform. This means that Intel has access to ARM’s high performance and high density logic libraries as well as their memory compilers and POP IP. While Intel has had the ability to fab an ARM-based part, the addition of the Artisan IP makes it easier.

This means that ARM Holdings customers have another choice when it comes to considering a foundry when it comes to a leading process node like 10nm. Currently, there are no 10nm chips so all foundries are currently talking about 10nm as a future node, but the reality is that many foundries will have 10nm chips shipping in 2017.

What it all means
The partnership between ARM and Intel to deliver ARM Artisan Physical IP within Intel’s Custom Foundry further legitimizes Intel’s efforts to open their Custom Foundry to more than what some considered “the random customer”. The addition of ARM IP to Intel’s fab also legitimizes ARM’s own Artisan Physical IP as the industry standard for physical IP with all major fabs now supporting it.

Intel has come a long way since making only their own chips with their own designs and tools and now they are beginning to present themselves as a serious foundry competitor to Samsung, TSMC, and GlobalFoundries, all three who heavily rely on mobile foundry volumes for profitability.

Intel may not be making chips for Qualcomm any time soon and the perceptual competitive challenges will need to be overcome, but there is a lot of potential that Intel could continue to grow their fab business as a way to improve the scale of their fabs and increase profitability. If Intel keeps driving their technology like they have for the last decade, some may have to fab there to remain competitive.

Net-net, this is a win for both Intel and ARM Holdings, but the biggest winner are the customers who see even greater competition in the foundry business.

Also read: The 2016 Leading Edge Semiconductor Landscape


What Will Kill ROP Cyberattacks?

What Will Kill ROP Cyberattacks?
by Matthew Rosenquist on 09-28-2016 at 10:00 am

IBM recently announced a software-oriented solution to help eradicate Return Oriented Programming (ROP) malware attacks. ROP is a significant and growing problem in the industry. Crafty hackers will use snippets of code from other trusted programs and stitch it together to create their attacks. It has become a very popular and effective technique for top malware.

Almost 90 percent of exploit-based software attacks use the hostile ROP technique in the chain of attack.

The Security Intelligence article referenced a blog I wrote in June about how Intel and Microsoft have developed a hardware based solution. Thought leading companies are looking to prevent these types of attacks.

First, let’s recognize that the problem is real, it is an issue now, and will likely be a favorite method of attackers because of its effectiveness and stealth properties. Because it is using parts of trusted code, it is very difficult to detect and stop. Software solutions have tried in the past to stem the problem, but have largely been unsuccessful. Software fighting software is just to even of a fight and the attackers only need to find one way around preventative solutions to win. I hope the IBM solution has a positive effect, but am concerned about the long term viability.

In the end, I believe the future of ROP security will be based on features embedded beneath the software, operating systems, virtual machines, and even beneath the firmware, and located in the hardware processor itself. Hardware tends to be outside the maneuvering zone of software hackers, therefore can give a definitive advantage to securing the system from ROP based attacks. The architecture itself can be designed to give advantages to secure computing practices, help OS’s be more secure, and compensate for vulnerable software.

Regardless of where it happens, it is very important for innovative minds to continue to work on taking the fangs out of ROP attacks.

References:


The Privacy Delusion

The Privacy Delusion
by Roger C. Lanctot on 09-28-2016 at 7:00 am

Why do we think we have privacy in our cars? Why does the government believe there is an interest in preserving privacy in cars? Can we just get over it? One of the least private places known to mankind – outside of the Internet – is the car!

But our transportation regulators in the U.S. and their counterparts at the European Commission cling to the fantasy of automotive privacy. It makes no sense. You sit in a device weighing more than a ton surrounded by glass with a powerful engine that, more often than not, loudly announces its presence.
We drive through camera-equipped intersections and tolling stations and past speed cameras peppered throughout the landscape deluding ourselves that we are somehow below the radar or out of sight. It’s nonsense.

But regulators and the car companies themselves foster this fallacy by promising to protect our privacy at all cost. The latest nonsense effort in this direction is one of the 15 “guidelines” from the U.S. Department of Transportation governing the development of cars capable of automated driving.

The USDOT says car owners should have a clear understanding of what kind of data is being collected by the vehicles. They should also be able to reject any collection of personal information such as biometrics or driver behavior.

This is more naïve nonsense. Any engineer working on self-driving cars today will tell you that these systems must integrate driver monitoring systems. Following the fatal crash of a Tesla Motors Model S in Florida and the latest autopilot software updates from Tesla, this is no longer negotiable. If you take your hands off of the steering wheel in an autopilot-equipped Model S for too long the car will direct you to retake the wheel and if you fail to do so will pull over.

This is not unlike a feature contemplated and being tested by GM, according to the GM Authority newsletter, as part of its Supercruise Level 2 enhanced cruise control – which will monitor driver attention and intervene and slow and stop the car (after an OnStar agent intervention) if the driver fails to respond or is incapacitated. Not only is privacy irrelevant when driving a car, it is dangerous.

Car makers like Tesla – in fact most auto makers – disclose their data collection activities but generally do not provide an opt out capability. Some European auto maker RFQs have included a valet parking function as a form of privacy, but the reality is that the implementation and proliferation of active safety systems will increasingly remove privacy from the equation.

Drivers in autopiloted cars must be monitored. Period.

But there are broader implications to the safety imperative. We are increasingly looking toward an IoT-style driving environment where all vehicles will inform all other vehicles of their presence and heading for the purposes of collision avoidance. No privacy there.

Further, car companies will increasingly be held responsible for being aware of vehicle flaws and failures in real time. Privacy will unquestionably be an impediment to broad communication and collection of diagnostic data in real-time.

So let’s please get over the privacy obsession. When we get in our cars we have unlocked a liberating experience, but we should never be deceived into believing that this experience comes with any privacy privileges or rights.

Also read: The Virus of Car Ownership


Synopsys Hosting Formal Methods in CAD Conference Next Week

Synopsys Hosting Formal Methods in CAD Conference Next Week
by Bernard Murphy on 09-27-2016 at 8:00 pm

SynopsysB

FMCAD (Formal Methods in Computer Aided Design) is a technical conference with a 20-year pedigree. This is a conference for serious formal methods teams. Key notes are from Berkeley and UCLA, committee members are all formal heavyweights and best I can tell, there is no exhibitors area.


Continue reading “Synopsys Hosting Formal Methods in CAD Conference Next Week”


Making photonic design more straightforward

Making photonic design more straightforward
by Don Dingee on 09-27-2016 at 4:00 pm

The arrival of optical computing has been predicted every year for the last fifteen years. As with any other technology backed by prolific research, lofty goals get dialed back as problems are identified. What emerges first is a set of use cases where the technology fits with practical, realizable implementations.

When it comes to photonics, the obvious use case is high bandwidth I/O channels. The ability to multiplex separate data channels in different wavelengths onto a single glass fiber has been the cornerstone of fiber optic communications for decades. These days, it’s not difficult at all to get that done with an SFP transceiver, a physical transition at the edge of a board from electrical to light domain. The idea in play now is to move that transition closer to the processing – and that has significant challenges.

Photons just work differently than electrons, and those differences drive layout and verification of photonic ICs. The photonic approach requires a combination of methods from traditional mixed signal design, RF design, high speed digital design, an understanding of the fab technology needed, plus light-domain expertise. That sounds scary, right? As the saying goes: if it were easy, everyone would do it. Fortunately, borrowing technology from those other design methodologies and adding knowledge of photonics is making some very cool things possible.

I’m struck by this quote in a new white paper co-written by a team at Luceda Photonics (a company started as a spin-off of imec) and the Tanner EDA experts at Mentor Graphics.

There is a large gap between what the silicon photonic technology can accomplish and the functionality that designers can actually design and simulate.

Several of the problems they point out are related to the waveguide nature of photonic design, forcing layouts into a single layer with very specific design and interconnect rules. Another area of big concern is the highly customized process design kits (PDKs) for photonic design, and the heavy burden placed on simulation due to the light spectrum frequencies and electromagnetic, electro-optical, and thermal parameters.


The goal is a design flow that moves from intense academic photonic knowledge to a production-ready environment that looks more familiar to designers coming from mixed signal backgrounds. Luceda Photonics has stepped in with their IPKISS.eda design framework built on Tanner L-Edit. This integration was made possible by use of the OpenAccess database standard and API, including heavy use of Python macros extending L-Edit functionality.

The white paper goes into detail with design of a 2×2 optical crossbar switch based on a Mach-Zehnder Interferometer thermo-optic switch. I’m not a photonic expert nor do I play one on TV, but I’ve done enough RF layout to appreciate how the Luceda environment handles waveguide generation. Note how the Luceda extensions appear as a new set of pull-downs in the familiar Tanner L-Edit tool:


Using Tanner Calibre One nmDRC, design rule checks can be run quickly using the rule deck from the photonic foundry, and highlighted in the results browser against the layout. Functional verification is performed with Luceda’s CAPHE optical circuit simulator. With the photonic environment fully integrated in IPKISS.eda, designers get help from automation with a large degree of control.

The complete white paper is available from Mentor Graphics (registration required):

Luceda Photonics Delivers a Silicon Photonics IC Solution in Tanner L-Edit

I remember the days of brute force RF and MEMS layout and funky tools that didn’t integrate with much of anything except a printer and a file the fab hopefully understood. This white paper is a good read on two fronts: how Luceda used open extensibility features to add functionality to L-Edit, and how challenging photonic chip design still is. I’m encouraged by the progress here. Certainly the philosophy Luceda is pursuing is correct – make photonic design as easy as advanced mixed-signal design. Leveraging Tanner’s capability not only got them there faster, it helps reduce EDA tool fragmentation and reduces the learning curve for users.

Making photonic design more straightforward should increase the number of photonic designs people will be willing to attempt, and that in turn should speed up overall innovation in the field.


Why is Low Frequency Noise Measurement for ICs Such a Big Deal?

Why is Low Frequency Noise Measurement for ICs Such a Big Deal?
by Daniel Payne on 09-27-2016 at 12:00 pm

Even digital designers need to be aware of how noise impacts their circuits because most clocked designs today use a Phase Locked Loop (PLL) block which contains a circuit called a Voltage Controlled Oscillator (VCO) that is quite sensitive in operation to the effects of noise and process variation. As process node scaling continues the effects of low frequency noise increase. There are even new devices coming out of R&D like nano-wires, Silicon Carbide (SiC) and photonic devices where it’s necessary to measure ultra-low current levels. During process development the engineers will need to accurately measure and characterize low frequency noise in devices so that designers of SRAMs, MEMs and sensors have the most accurate statistical models.


Planar MOS device is sensitive to 1/f noise

One of the newer vendors in the noise characterization space is Platform Design Automationwhich has created a fast and accurate system called the NC300.


NC300

Related blog – A Brief History of Platform Design Automation

An ideal noise characterization system would have these characteristics:

  • Fast speed, seconds instead of minutes
  • Lowest current measurements, pA to nA
  • IC testing, SRAM testing, sensor noise testing

How well does the NC300 compare to the ideals listed above?

  • About 8s to 10s per bias, up to 10X faster
  • Low noise floor of under 10[SUP]-29[/SUP]A[SUP]2[/SUP]/Hz
  • Current noise in the pA range
  • Integrated with Source Monitor Units (SMU) and Dynamic Signal Analyzer (DSA)
  • Supports both multi-die and multi-wafer measurements


NC300 – system noise floor

Other noise measurement systems can take weeks to measure 1/f noise with large samples to form corner or statistical models, but with the NC300 you’re getting results 10X quicker than that. This is a big deal because time is money in the semiconductor business.

With this type of instrumentation and software you can perform all four steps from noise measurement to circuit characterization for design engineers:

The design of the NC300 includes the DSA and Low Noise Amplifier (LNA), saving you equipment space and the performance enables high-volume noise measurements, suitable for both on-wafer and packaged parts. With this kind of test and modeling setup you can characterize:

  • Random Telegraph Noise (RTN) in RRAM
  • Circuit noise test
  • MEMs sensor test
  • Mercury Cadmium Telluride (MCT) infrared device noise test
  • Use MeQLab software for data extraction and modeling

Related blog – Are Your Transistor Models Good Enough?

Noise Testing Examples
Let’s take a quick look at some specific noise testing cases starting with a type of low frequency noise called RTN where electrons or holes are getting trapped between the gate and well of devices which in turn causes vibrations in V[SUB]th[/SUB]values:


RTN with electrons


RTN with holes


RTN causing multi-states

Circuit noise from an Op Amp can be measured and a statistical noise model automatically generated:

The IoT market is enabled by sensors, so here’s a pressure sensor tested for both a low frequency spectrum and derived quality factor:


Pressure sensor – Low Frequency Spectrum


Derived quality factor

GUI
A modern GUI makes learning and using the NC300 easy and intuitive:

Customers
The NC300 has been adopted by leading foundries, design companies and research labs. The 5 leading foundries have benefited from NC300 with its fast speed and ease of use. Researchers are now able to do the noise measurement of super-low currents such as the dark current of photo diodes, which was never achieved with other systems.

Summary
There are multiple vendors offering noise characterization systems in the world, and the NC300 from PDA looks to be a strong, new player worth taking a look at.


Good AI

Good AI
by Bernard Murphy on 09-27-2016 at 7:00 am

A hot debate recently, promoted notably by Elon Musk and Stephen Hawking, has explored whether we should fear AI. A key question centers around the ethics of AI – how we can instill ethical values in intelligent systems we will build and how, I hope, we can ensure we use those systems ethically. This is not an academic question – autonomous cars have already been involved in crashes so it is reasonable to expect they will face challenges which require ethical decisions to be made. More concretely, Alphabet (Super-Google), Facebook, Microsoft, Amazon and IBM have been meeting periodically to discuss how to build ethics into AI.

However, this is a different class of problem from other applications of AI. When you think about image or speech recognition, for example, the class of images or words you want to recognize is well-defined and bounded. But ethical problem spaces are not so easily characterized. Philosophers and religious leaders have struggled for at least 2500 years with the difficult question of how to define what is “good”, and what guidance they provide is generally based more on beliefs than an evidence-rooted chain of reasoning. Which might be a workable starting point for automated ethics if we all shared the same beliefs, but unfortunately we don’t seem to be able to make that claim, even within a single cultural group.

Deep reasoning might be one way to approach the problem (within a common belief-system) on the grounds that perhaps you don’t have to understand the issues, you just have to train with sufficient examples. But how can you construct a representative set of training examples if you don’t understand the problem? Verification engineers should relate to this – you can’t develop a reasonable test plan if you don’t understand what you are testing; the same principle should apply to training.

Perhaps we could develop a taxonomy of ethical principles and build ethics systems to learn human behavior for each principle within a given cultural group. It seems that these are easiest to define by domain; one example I found develops a taxonomy for ethical behavior in crowdsourcing. In some contexts, a domain-specific set of ethical principles might be sufficient. But you could imagine that other contexts may require ethical choices to be more challenging. A commonly cited example here is in choosing between multiple options in collision-avoidance – hitting another car (potentially killing you and the other driver), swerving into a wall (killing just you) or swerving onto a sidewalk (killing pedestrians but saving you). A purely rational choice here, no matter what that choice might be, is unlikely to be acceptable from all points of view.

Another viewpoint considers not the basics of recognizing ethical behavior but instead the mechanics of policing such behavior. It starts from the assumption that it is possible to capture ethical guidelines in some manner and provides layers of oversight, similar to societal standards, to an AI system which must be monitored. This approach provides monitors to ensure AI behavior stays within legal bounds (eg obeying a speed limit), super-ethics decision-makers which look not just at the narrow legality of a situation but also larger human ethics (saving a life, minimizing risk) and enforcers (police) outside the control of the local system which can report violations of standards. Perhaps this discussion doesn’t take the society analogy far enough – if we’re going to have laws and police, shouldn’t we also have lawmakers and courts? And how do we manage conflicts between AI laws and human laws?

On which note, Stanford has a 100-year study on AI in which they look at many factors, including implications for public policy. One discussion is on implications for civil and criminal liability and questions of agency (can AI enter into a legally binding contract?) and how those should be used in prosecuting the consequences of unethical behavior. What is interesting here is that damage to another is not limited to bodily harm – it could be financial or other kinds of harm. So definitions of ethical behavior in some contexts can be quite broad (imagine an AI bot ruining your online reputation). And the agency question is very important. In the event that liable behavior is found, who or what should be liable? This area cannot be resolved purely with technology – our laws must advance also, which requires that our societal beliefs must advance.


I found another piece of research quite interesting in this context – a piece of AI designed to engage in debate. Outside of fundamentalist beliefs, many (perhaps most) ethical problems don’t resolve to black and white choices especially when not in full possession of all relevant data. In these cases, the process of arguing a case towards an outcome is at least as important as a bald set of facts in support of the outcome, especially where there is no clear best choice and yet a choice must be made quickly. For me this line of thinking may be at least as important as any taxonomy and deep-reasoning based work.

As you can see, this is a domain rife with questions and problems and much more sparsely populated with answers. This is not a comfortable place for engineers and technologists to operate – we like clean-cut, yes/no choices. But we’re going to have to learn to operate in this fuzzier, messier world if we want AI to grow. The big tech collaboration is discussed HERE, the societal/oversight article can be found HERE. The Stanford study is HERE and the debate video is HERE.

More articles by Bernard…