Photography tools

Unlocking AI at the Edge with Deci’s New Tools

Couldn’t attend Transform 2022? Discover all the summit sessions now in our on-demand library! Look here.

Peripheral devices must be able to process the data provided quickly and in real time. Additionally, cutting-edge AI applications are only effective and scalable when they can make highly accurate imagery predictions.

Take on the complex and critical task of autonomous driving: all relevant objects in the driving scene must be taken into account, whether they are pedestrians, lanes, sidewalks, other vehicles or signs signs and lights.

“For example, an autonomous vehicle driving through a crowded city must maintain high accuracy while operating in real time with very low latency; otherwise, the lives of drivers and pedestrians may be at risk,” said Yonatan Geifman, CEO and co-founder of deep learning company Deci.

The key to this is semantic segmentation, or image segmentation. But there is a dilemma: semantic segmentation models are complex, often slowing down their performance.


MetaBeat 2022

MetaBeat will bring together thought leaders to advise on how metaverse technology will transform the way all industries communicate and do business on October 4 in San Francisco, California.

register here

“There is often a trade-off between accuracy, speed and size in these models,” said Geifman, whose company this week released a set of semantic segmentation models, DeciSeg, to help solve this complex problem.

“This can be a barrier to real-time edge applications,” Geifman said. “Creating accurate and computationally efficient models is a real challenge for deep learning engineers, who go to great lengths to achieve both the accuracy and speed that will satisfy the task at hand.”

The power of the edge

According to Allied Market Research, the global advanced AI (artificial intelligence) market size will reach nearly $39 billion by 2030, growing at a compound annual growth rate (CAGR) of nearly 19% over 10 years . Meanwhile, Astute Analytica reports that the global market for advanced AI software will reach over $8 billion by 2027, growing at a CAGR of almost 30% from 2021.

“Edge computing with AI is a powerful combination that can bring promising applications to consumers and businesses,” Geifman said.

For end users, that translates to more speed, improved reliability and a better overall experience, he said. Not to mention better data privacy, as the data used for processing stays on the local device (mobile phones, laptops, tablets) and does not need to be uploaded to third-party cloud services. For enterprises with consumer applications, this means a significant reduction in cloud computing costs, Geifman said.

Another reason cutting-edge AI is so important: communication bottlenecks. Many machine vision edge devices require intensive analysis of high-resolution video streams. But, if the communication requirements are too large compared to the network capacity, some users will not get the required analysis. “Therefore, moving computation to the edge, even partially, will enable large-scale operation,” Geifman said.

No critical compromise

Semantic segmentation is the key to advanced artificial intelligence and is one of the most widely used computer vision tasks in many industries: automotive, healthcare, agriculture, media and entertainment, consumer applications, smart cities and other image-intensive implementations.

Many of these applications “are critical in the sense that getting the segmentation prediction correct and in real time can be a matter of life and death,” Geifman said.

Autonomous vehicles, for one; another is cardiac semantic segmentation. For this critical task in MRI analysis, images are divided into several anatomically significant segments that are used to estimate criticalities such as myocardial mass and wall thickness, Geifman explained.

There are, of course, examples beyond critical situations, he said, such as video conferencing virtual background features or smart photography.

Unlike image classification models — which are designed to determine and label an object in a given image — semantic segmentation models assign a label to each pixel in an image, Geifman explained. They are generally designed using an encoder/decoder architecture structure. The encoder progressively downsamples the input while increasing the number of feature maps, thereby building informative spatial features. The decoder receives these features and progressively upsamples them into a full-resolution segmentation map.

And, while it is often required for many edge AI applications, there are significant hurdles to running semantic segmentation models directly on edge devices. These include high latency and the inability to deploy models due to their size.

Highly accurate segmentation models are not only much larger than classification models, Geifman explained, they are also often applied on larger input images, which “quadratically increases” their computational complexity. This results in slower inference performance.

Case in point: defect inspection systems operating on manufacturing lines that must maintain high accuracy to reduce false alarms, but cannot sacrifice speed in the process, Geifman said.

Lower latency, higher accuracy

The DeciSeg models were automatically generated by Deci’s Automated Neural Architecture Construction (AutoNAC) technology. The Tel Aviv-based company says these models “significantly outperform” existing publicly available models, including Apple’s MobileViT and Google’s DeepLab.

As Geifman explained, the AutoNAC engine takes into account a large search space of neural architectures. When searching in this space, it considers parameters such as baseline accuracy, performance goals, inference hardware, compilers, and quantization. AutoNAC attempts to solve a constrained optimization problem while achieving multiple goals at once, i.e., preserving baseline accuracy with a model that has some memory footprint.

The models offer more than 2x lower latency and 3-7% higher accuracy, Geifman said. This enables companies to develop new use cases and applications on edge AI devices, reduce inference costs (as AI practitioners will no longer need to perform tasks in expensive cloud environments), open up new markets and shorten development times, Geifman said. AI teams can solve deployment problems while achieving desired accuracy, speed, and model size.

“DeciSeg models enable semantic segmentation tasks that previously could not be performed on cutting-edge applications because they were too resource-intensive,” Geifman said. The new set of models “has the potential to transform industries as a whole”.

VentureBeat’s mission is to be a digital public square for technical decision makers to learn about transformative enterprise technology and conduct transactions. Discover our Briefings.