AI/ML is the Path to Advanced, Energy-Efficient Edge IoT

­Artificial intelligence (AI) and the internet of things (IoT), two of the broadest trends defining technology today, are distinct, but with significant intersections. Where they overlap at the network edge, AI has become integral to a growing number of increasingly sophisticated edge IoT devices.

AI simultaneously serves edge IoT products in two distinct ways.As well as improving performance and/or adding new capabilities and features, it also enables mechanisms that allow edge devices to judiciously moderate and even reduce energy usage.

In the past, technology innovations generally progressed at an expected rate. If, for example, engineers needed a certain amount of processing power at a certain price point to achieve some envisioned product, they could consult the curve plotted by Moore’s Law and determine – sometimes down to the calendar quarter – when their ideas would likely become feasible.

With the speed of AI and edge IoT technology development, however, that script has flipped. Edge IoT device developers can – today – apply AI and economically accomplish far more than many of them realize. Powerful tools and techniques for designing advanced, AI-powered edge IoT devices are readily available market opportunities for IoT application developers abound.

Edge and IoT overlap

The edge of the network may be notional, but its conceptual location has moved and moved again several times in just the last decade or so. Only a few short years ago, a data center owned by a telco was considered to be part of the edge, because technically it wasn’t part of the cloud. Not long after that, local server racks and routers installed by small and medium-sized businesses (SMBs) were considered the edge.

Now the edge is defined by far smaller connected devices that gather and process data in places that are as physically remote from the biggest data centers as can be. These devices might be anything from security cameras to connected appliances to wearables to individual sensors – in short, the types of devices that define the IoT. As the IoT has expanded, the edge has shifted to meet it, be that smart cities, smart homes, agricultural fields or even maritime oil rigs.

This broad electronic environment is, however, characterised by twin challenges. Firstly, extremely constrained resources are at the forefront of design considerations. Weight, size, prices, and the cost of operation (which includes energy consumption) must all be minimized. Furthermore, the IoT edge comprises a set of profoundly competitive market spaces where smaller, cheaper, more economical alternatives are always around the corner.

AI and Edge IoT Combine

AI has become intrinsic at the edge for two reasons. Using AI is one of the best ways to get smaller, cheaper, and more economical. At the same time, business logic is moving toward the source of the data, for economic and security reasons, and using AI furthers both goals.

Adding AI to edge IoT devices might be the difference between using a large battery that makes the product too cumbersome to be truly portable versus using a small, unobtrusive button battery that makes the device comfortably wearable. Conversely, it could be the variable that allows a battery to last for days as opposed to the same battery being functional for weeks or longer.

Meanwhile, performance improvements and new features that AI can enable a product that was occasionally useful into something that is genuinely vital. For example, AI combined with advancements in sensor technology are allowing device manufacturers to evolve fitness trackers that measure a few vital signs, indicating general health, into wearables that provide reliable data that medical professionals can legally use for diagnostics and treatments.

AI-Enabled Edge IoT’s Machine Learning Evolution

AI with machine learning (ML) is a powerful combination. AIs are trained on models, but ML is a technology that allows AIs to continue refining the models they were trained on. ML essentially does what it says, it allows AIs to learn.

Models for edge AI can be relatively simple, especially when compared to the types of complex large language models (LLMs) upon which chat tools are based. That’s one way to reduce the processing burden of AI for edge IoT applications. There are others that involve the creation of clever algorithms, but the point here is that AI can be readily scaled down to run on very modestly powered processors that are appropriately sized and priced for edge computing. These tend to be microcontrollers (MCUs) running at lower frequencies, accompanied by mere megabytes of memory.

Click image to enlarge

In much the same way that models can be right-sized for edge AI in general, ML can be right-sized too. There is an industry effort to make ML run on the same small processors. Originally called TinyML, it is now commonly referred to as EdgeAI - another indication that AI and edge IoT are not only intersecting but are merging.

AI/ML is particularly useful at the edge because the vast majority of data collected at the edge is not useful anywhere else, and so there is no reason to transmit it further and several reasons to actively avoid this:

l  Transmission of the raw data expends energy

l  Use of any network typically has a monetary cost

l  Bandwidth is always limited, and transmission of the raw data consumes the bandwidth that could have been dedicated to more important traffic

l  Data processing in data centers is performed by processing systems that consume far more energy than edge IoT processors

l  Return transmission of the processed results consumes more energy and bandwidth

l  The round-trip transmission process incurs network latency; it introduces a time lag between input and result

This means that processing data collected at the edge is a matter of business logic, and business logic is moving to the edge because AI/ML processing at the edge produces the necessary results while consuming the fewest resources (bandwidth, energy, money, time), while also being more efficient.

If someone asks Siri to turn on the lights, the response should be immediate; network latency is intolerable. Response times can be hastened by avoiding network latency, but network outages do occur; localized processing means an answer will still be forthcoming.

A final consideration is that local processing is safer inasmuch as it reduces the chances the data will be intercepted, certainly when compared to sending it through multiple transfers to and from a data center. Governments in several major markets have imposed legal obligations to protect personal data, making keeping local data local a tenet in data privacy and protection.

AI Edge IoT On, Off, and Waiting

Looking back to energy efficiency, reducing the amount of power drawn has important ramifications for batteries and battery life, and AI/ML has a role to play in this.

There are several ways to reduce power consumption. Edge IoT devices tend to be connected by wireless means, communicating using a variety of protocols that include Wi-Fi, but more commonly Bluetooth Low Energy (BLE). But even with power-saving options such as BLE, every transmission still costs energy – quite a lot by the standards of a battery-operated device.

Take the example of security devices that listen for glass breaking. Originally, and up until a few years ago, these devices had to be always on. Furthermore, every sound it detected that might have been glass breaking would be transmitted to the network for processing. That tended to include any other sharp sounds such as hand claps. Meaning needlessly expanded energy alerting users of irrelevant sounds.

AI/ML eliminates this issue. Instead of being always-on, the device can be put into a low-power sleep mode. With TinyML (EdgeAI), the device does local processing and learns to detect the difference between a clap and glass breaking. Only when it determines that a sound is glass breaking does it initiate the process of turning on the rest of the device, including the relatively power-hungry wireless radio, to trigger an alarm. The device remains in an energy-saving sleep state until needed, and local processing using AI/ML assures that energy-consuming transmissions of irrelevant data are eliminated.

This is eminently possible even without AI/ML if the device incorporates a sophisticated enough processor – but that level of processing power is likely to cost far too much for an edge IoT device. With AI/ML, however, this capability is possible on a processor that is an appropriate size for an edge IoT product.

As these devices are expected to run anywhere from three to five years on a single battery, these energy saving processes are transformative.

Only The Start

AI/ML makes it possible to do extraordinary things on edge IoT devices today, in a completely economical manner, using tools that make the design process far easier than might be expected.

Furthermore, AI/ML in edge IoT is still evolving. Ongoing device integration means even the modest processors used at this level will get more powerful.

Optimized algorithms and increasing edge compute power are certain to lead to significant advancements in user experience on local devices. TinyML (EdgeAI) is on a path to bring conversational AI capabilities down to edge devices.

Silicon Labs