PCs and AI chips?

[Arthur Hanson]

AI Is Remaking the PC. These 2 Stocks Could Be Big Winners.

AI’s big opportunity goes beyond the cloud. Why Dell and HP Inc. could be big beneficiaries of the AI revolution.

www.barrons.com

It looks like the PC makers are looking to take AI to the PC. Is this possible on a cost/performance basis or is it better to for PCs to be an access tool for AI data centers? Any thoughts or comments on this appreciated. To me it looks like the data center will win with high power processors and a substantial sets of specialized AI/ML software for a range of applications. Any thoughts, insights and observations wanted and appreciated. While the PC might become more a specialized tool for accessing data center resources that would be to expensive on a personal or small business scale. Will the PC market become a two tier market depending on needs?

 

[count]

There is a lot of AI in your phone already, I think this is bringing some of those same kinds of AI accelerators used in a lot of mobile applications to PC. Models would probably be trained in the cloud but deployed to be run client side with low latency.

 

[MKWVentures]

@count @Arthur Hanson

Dumb questions from Me:

What are 2-3 examples of AI tasks in a Laptop PC that requires AI on Chip?

So if I have a Alder Lake Notebook, will I not be able to do these tasks? How much longer with they take.

when I ask Chat GPT, It says a bot will help me use my PC or it will give me search results based on what It thinks I want to see. I hate both.

thanks in advance for the help

 

[VCT]

The only edge AI application I can image is to have an smaller edge version ChatGPT or edge version Microsoft Copilot in you PC.

 

[blueone]

The only edge AI application I can image is to have an smaller edge version ChatGPT or edge version Microsoft Copilot in you PC.

This is a well-written article from Apple, which has been deploying Neural Engines in mobile devices and Macs for years now. Transformers are not the only type of neural networks which benefit from acceleration in clients, but since they're used by LLMs and NLP they're an interesting example.

Deploying Transformers on the Apple Neural Engine

An increasing number of the machine learning (ML) models we build at Apple each year are either partly or fully adopting the Transformer…

machinelearning.apple.com

 

[KevinK]

Good article @blueone - gives good examples of client / edge (PC and smartphone) usage of AI. Several advanced forms of image adjustment, image captioning and real-time language translation. I’m guessing that video editing and compositing can also greatly benefit from AI. Apple probably has a better line of sight to which client side apps can greatly benefit from AI and associated specialized processors.

 

[MKWVentures]

Good article @blueone - gives good examples of client / edge (PC and smartphone) usage of AI. Several advanced forms of image adjustment, image captioning and real-time language translation. I’m guessing that video editing and compositing can also greatly benefit from AI. Apple probably has a better line of sight to which client side apps can greatly benefit from AI and associated specialized processors.

So Image adjustment, Image captioning, Real time language translation, video editing. Can I do these today on my Raptor lake system with a video card? How much faster will this be with AI processors? Is this workstation stuff or notebook computer stuff.

 

[blueone]

So Image adjustment, Image captioning, Real time language translation, video editing. Can I do these today on my Raptor lake system with a video card? How much faster will this be with AI processors? Is this workstation stuff or notebook computer stuff.

On Apple products, video editing is important to many users. Not me, but I see a lot of young people really into it. Then there's natural language processing. I don't use Siri or other software like it, but among people I know I appear to be in the minority. Ditto with Alexa. (I don't like my computers constantly listening to me.) For the functions most of the users do I'm observing I think a video card is a waste of expense and power; the specialized cores in the Apple CPUs are probably good enough, IMO. It's only gaming and high-end video editing that seems to need a full-scale GPU.

 

[blueone]

Good article @blueone - gives good examples of client / edge (PC and smartphone) usage of AI. Several advanced forms of image adjustment, image captioning and real-time language translation. I’m guessing that video editing and compositing can also greatly benefit from AI. Apple probably has a better line of sight to which client side apps can greatly benefit from AI and associated specialized processors.

I especially like many of the technical articles on the Apple, Intel, Nvidia, and Google web sites. Microsoft is a more mixed bag, but sometimes a good one pops through.

I think Apple has done an awesome job targeting its silicon to its customer base. I just acquired an iPad Air with the M1 CPU. An outstanding tablet. Unfortunately for Apple, they won't see me buying another for years. (And, as an aside, Apple finally figured out how to design and build an excellent keyboard folio for the iPad too. A bit heavy, but with an excellent keyboard and a touch pad (albeit a small pad), and unfortunately it costs as much as some cheap Android tablets.)

 

[KevinK]

So Image adjustment, Image captioning, Real time language translation, video editing. Can I do these today on my Raptor lake system with a video card? How much faster will this be with AI processors? Is this workstation stuff or notebook computer stuff.

My view is that most of this client-side AI is really going to be used at the smart phone level - one button cleanup of photos in second. Complex grabs and deletes of distracting background items in photos and videos. Video upscaling and frame rate enhancement (see NVIDIA DLSS and DLAA). And realtime translation of spoken sentences without the need for massive cellular data usage.

 

[MKWVentures]

My view is that most of this client-side AI is really going to be used at the smart phone level - one button cleanup of photos in second. Complex grabs and deletes of distracting background items in photos and videos. Video upscaling and frame rate enhancement (see NVIDIA DLSS and DLAA). And realtime translation of spoken sentences without the need for massive cellular data usage.

agreed. just trying to get input to separate hype from reality ... when most people can't even define what this new AI is (other than Chatbots). I am not convinced people will be able to pick out the 12th or 13th Gen core 9 from the 14th gen core 9 based on typical usage in a Notebook.

 

[blueone]

agreed. just trying to get input to separate hype from reality ... when most people can't even define what this new AI is (other than Chatbots). I am not convinced people will be able to pick out the 12th or 13th Gen core 9 from the 14th gen core 9 based on typical usage in a Notebook.

For Apple, a lot of people think the initial winners from AI apps and acceleration will be Siri, followed by spelling correction that isn't annoying, grammar correction that actually works, and improvement to Mail and iMessage. I don't think the question will be if non-technical people can tell the difference between x86 CPU generations, the question will be how wide the gulf gets between Apple device and software capabilities and x86/Arm/RISC-V system capabilities. I think Apple is ahead, but Intel's extend-the-instruction-set strategy makes software development a lot easier for applications developers, including Microsoft and Google. Personally, I like Apple's strategy better, but if I was paid to I think I could argue for either side.

 

[MKWVentures]

Simplistic: Does AI hardware add significant value to a Notebook PC. To me, based on the feedback here, It is not clear that it does. But we will see if Apple or Intel 14th gen Provide any improvement over my 12th Gen system. As always, it will depend on what app is written for that system that takes advantage of it.

Just so we are clear, Many technical people who work at Intel cannot tell difference on a notebook PC performance between 12th Gen and 13th Gen without running benchmarks.

thanks for the input everyone

 

[delong.height]

@count @Arthur Hanson

Dumb questions from Me:

What are 2-3 examples of AI tasks in a Laptop PC that requires AI on Chip?

So if I have a Alder Lake Notebook, will I not be able to do these tasks? How much longer with they take.

when I ask Chat GPT, It says a bot will help me use my PC or it will give me search results based on what It thinks I want to see. I hate both.

thanks in advance for the help

I think the point isn't about considering what AI workloads that PC needs right now, but what workload can be offloaded from GPU and CPU and see a huge performance + efficiency uplift, making the whole SoC more efficient overall. Thereby, it reduces the criticality of high-end CPU/GPU on the board. Now imagine, if I'm a video editor or a gamer, and I need a 4090 GPU. If the NPU is powerful enough, it can potentially take over some workload that were previously ran on CPU or GPU and run them in NPU. So, if the software is well-baked, I may not need a 4090 GPU, I may be able to get a 4070-level and have the same amount of performance at lower power. I think that's pretty attractive to company like Intel/AMD.

 

[delong.height]

I think the point isn't about considering what AI workloads that PC needs right now, but what workload can be offloaded from GPU and CPU and see a huge performance + efficiency uplift, making the whole SoC more efficient overall. Thereby, it reduces the criticality of high-end CPU/GPU on the board. Now imagine, if I'm a video editor or a gamer, and I need a 4090 GPU. If the NPU is powerful enough, it can potentially take over some workload that were previously ran on CPU or GPU and run them in NPU. So, if the software is well-baked, I may not need a 4090 GPU, I may be able to get a 4070-level and have the same amount of performance at lower power. I think that's pretty attractive to company like Intel/AMD.

There are 100 or so ISVs that's currently working with Intel

Simplistic: Does AI hardware add significant value to a Notebook PC. To me, based on the feedback here, It is not clear that it does. But we will see if Apple or Intel 14th gen Provide any improvement over my 12th Gen system. As always, it will depend on what app is written for that system that takes advantage of it.

there's some info on recent AI PC Acceleration program, 8x more power efficient vs previous generations. That's a combination of new architecture, new process technology, new packaging, and perhaps software optimization as well.

 

[blueone]

Simplistic: Does AI hardware add significant value to a Notebook PC. To me, based on the feedback here, It is not clear that it does. But we will see if Apple or Intel 14th gen Provide any improvement over my 12th Gen system. As always, it will depend on what app is written for that system that takes advantage of it.

Just so we are clear, Many technical people who work at Intel cannot tell difference on a notebook PC performance between 12th Gen and 13th Gen without running benchmarks.

thanks for the input everyone

I think you made up your mind before asking. As for Intel technical people, without knowing which applications they're running it's impossible to judge the efficacy of what they're saying. FWIW, if your idea of PC applications are just MS Office, Outlook, and web browsing, I think it would be difficult to tell the difference between the 8th generation iPad I just reset to sell and the latest i9 PC with 64GB of DRAM. When people ask me what to buy in a PC for those applications I listed, I tell them with confidence that any Intel i5 and up (or AMD equivalent) CPU will do, and get the biggest DRAM store they can afford (32GB minimum, 64GB is better), and a PCIe Gen4 SSD, like a Samsung 980.

 

[count]

Things like image and audio processing are already examples. Like Windows Hello, where you can unlock your computer using facial recognition. Photo and video editing are also very clear use cases.

Microsoft is starting to integrate ChatGPT/LLMs into all kinds of applications, and I can imagine they would like to add functions to do things like write emails, edit word documents, build excel spreadsheets automatically, ect.