Edge AI means running artificial intelligence models, usually for AI inference, on or near the devices and sites where data is produced, such as cameras, sensors, vehicles, machines, store servers or a local edge data center, rather than sending every request to a distant cloud. The model makes its prediction or decision locally, which can cut response time, reduce the data sent over the network and keep sensitive data on site. Models are commonly trained and managed centrally, then deployed out to the edge.
At a glance
- Runs AI inference on devices, gateways or local servers close to where data is created.
- Can reduce latency, bandwidth costs and dependence on a constant internet connection.
- Training and model management usually stay central, often in the cloud.
- Hardware ranges from small AI chips in devices to GPU servers at a site, depending on the model.
- The main burden is operating many distributed devices: updates, monitoring, security and replacements.
What problem it solves
Many AI uses depend on data that is created far from a data center. A camera checking for safety-gear compliance, a scanner reading labels on a conveyor or a sensor watching a motor for signs of failure all produce data continuously. Sending all of it to the cloud for analysis can be slow, expensive and fragile: latency may be too high for real-time decisions, video uploads consume large amounts of bandwidth, and the process stops if the site’s connection drops.
Running the model locally can ease those issues. Processing on site can cut the round trip to a distant cloud and reduce the data sent upstream, and a well-designed system can keep working during an outage. How much latency actually drops and how much data is still transmitted depend on the workload, the hardware and how the system is built. It can also help where data, such as video of employees or patients, should stay on the premises.
How it works
Train centrally. A model is trained or fine-tuned on collected data, usually in the cloud or a data center, using machine learning (ML) tools. Many organizations start from a vendor’s pre-trained model.
Optimize for the hardware. Models are often made smaller and faster, for example by compression or by using lower-precision numbers, so they fit the memory and power budget of the target device. This can reduce accuracy slightly, so results should be tested.
Deploy to the edge. The model is installed on devices, gateways or local servers. Options include AI-capable cameras, industrial PCs, small servers at a branch, or racks in an edge data center. This is a specific use of broader edge computing.
Run locally. Data from sensors or cameras is processed on site. The device acts on the result, such as raising an alert or stopping a line, and often sends summaries or exceptions to central systems; some designs also upload raw data for review or retraining. Video analytics and Industrial Internet of Things (IIoT) monitoring are common examples.
Manage the fleet. A central console pushes model updates, monitors device health and accuracy, and collects data for retraining. Without it, models drift out of date and devices become hard to secure.
When it matters for buyers
- When decisions must be fast. Safety, quality inspection and machine control often can’t wait for a round trip to the cloud.
- When data volumes are large. Video and high-frequency sensor data can be far cheaper to analyze locally than to upload.
- When sites have unreliable connectivity. Local inference keeps working through outages, if designed to.
- When data should stay on site. Processing locally can support privacy goals or customer requirements, though it doesn’t remove them.
- When you are budgeting for operations. Distributed hardware needs lifecycle management, remote support and physical security.
Our edge compute overview covers infrastructure for running workloads at sites, and our artificial intelligence overview covers model and platform choices.
Questions to ask vendors
- What hardware does your solution run on, and what are its power, cooling and environmental requirements?
- Which models are included, can we bring our own, and how is accuracy measured on our data?
- How are model and software updates deployed to many sites, and can we roll back?
- What happens when a device loses its connection to the central console?
- What data leaves the site, and where is it stored?
- How are devices secured, monitored and replaced if they fail?
- How is the solution priced: per device, per camera, per model or by usage?
How it differs from edge computing
Edge computing is the broad practice of running workloads, such as applications, data processing or caching, close to users and devices. Edge AI is one kind of edge workload: specifically, running AI models there. It inherits the same trade-offs, such as managing distributed hardware, but adds AI-specific concerns: model size and hardware accelerators, accuracy in real-world conditions, and keeping models updated. It also overlaps with private AI, since both keep data and inference under your control, but private AI can just as well run in your own data center.