3 trends for 2024: AI drives more edge intelligence, RISC-V, & chiplets

Find the Latest SoC Solutions
for... Automotive... IoT... Security... Audio... Video

With the rise of AI, once "simple" devices are becoming increasingly intelligent, leading to more computing devices than ever before.

www.embedded.com, Jan. 02, 2024 – 

With CES 2024 set to open its doors in Las Vegas just a week from now, it's clear that this year is all about evolving consumer electronics products that rely on ever more connected, embedded edge intelligence.

This is nothing new, and we've been talking about it for a few years, but after the industry 'hype' of 2023 around generative AI, consumers will begin to understand more of what it means for them in their everyday lives. Almost every industry vertical will see more connected embedded devices with even more smartness or intelligence at the edge.

In his keynote at CES 2024 on Tuesday 9th January 2024, Pat Gelsinger, CEO of Intel, will explore how silicon, amplified by innovative and open software, is enabling AI capabilities for consumers and business alike. And on the following day, Qualcomm president and CEO Cristiano Amon will highlight how more devices will be seamlessly integrated into our lives; he'll explain that AI running pervasively and continually on devices will transform user experiences, making them more natural, intuitive, relevant, and personal, with the need for increased immediacy, privacy, and security.

That means more machine learning (ML) in more and more constrained devices, in the sensors, whether it is for the internet of things (IoT), for industrial automation, for autonomous mobility and software-defined vehicles (SDVs), or for health and wearable devices.

In the embedded world here are what I see as trends enabling some of this:

1. Edge intelligence gets better

If it's any indication of the direction of travel, then the research paper just released by Apple on deploying large language models (LLMs) on resource constrained devices with limited memory is certainly a pointer. It's paper, entitled "LLM in a flash: Efficient Large Language Model Inference with Limited Memory", tackles the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters on flash memory but bringing them on demand to DRAM.

In the paper, the Apple team said their methos involves constructing an inference cost model that harmonizes with the flash memory behavior, enabling optimization in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. To do this, they have introduced two techniques – one called "windowing" to strategically reduce data transfer by reusing previously activated neurons, and the second being "row-column bundling" to increase the size of data chunks read from flash memory.

The papers states, "These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively. Our integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory."

What this points to is that the general direction of travel in the industry is the deployment of more machine learning and inference at the edge.

2. RISC-V adoption becomes more visible

In August 2023, several companies had announced the formation of a new unnamed company that would help accelerate commercialization of RISC-V hardware globally. Well in the last week, that company was named as Quintauris, with Alexander Kocher appointed as CEO. Headquartered in Munich, Germany, the company's investors are Bosch, Infineon, Nordic Semiconductor, NXP Semiconductors, and Qualcomm Technologies, Inc. While the web site is minimal at the moment, it states:

"The company will be a single source to enable compatible RISC-V based products, provide reference architectures, and help establish solutions widely used in the industry. Initial application focus will be automotive, but with an eventual expansion to include mobile and IoT."

It would be interesting to see how Quintauris sits alongside RISC-V International, its neighbor headquartered in Zurich, Switzerland. The industry association no doubt looks after maintaining the instruction set architecture's (ISA's) specifications, while Quintauris is likely to be a resource for developers needing ready-made reference boards and systems for their own development.

When many commentators talk about RISC-V, the point that is often missed is that developers are designing systems based on heterogenous architectures – so multiple ISAs are likely to be part of the chip with RISC-V being deployed for various functions.

Consulting firm SHD Group presented a report at the November 2023 RISC-V Summit in the U.S. to highlight its own RISC-V market analysis, which it expects to release as a report this year. In a briefing with embedded.com in December 2023, SHD Group's principal analyst Richard Wawrzyniak told us, "We're in a heterogeneous world. We're not saying that RISC-V is taking over the world, but the ecosystem is building out well with all the elements a designer needs to create their own silicon."

click here to read more...

 Back