Deep Learning on Mobile Devices:  What's New in 2023?

08:00 Pacific Time   ┈   Andrey Ignatov   ┈   AI Benchmark Project Lead, ETH Zurich

Biography: Andrey Ignatov is the project lead of the AI Benchmark initiative targeted at performance evaluation of mobile, IoT and desktop hardware at ETH Zurich, Switzerland. His PhD research was aimed at designing efficient image processing models for smartphone NPUs / DSPs and developing the next-generation deep learning based mobile camera ISP solution. He is the lecturer on Deep Learning for Smartphones course at ETH Zurich and the main author of the AI Benchmark papers describing the current state of deep learning and AI hardware acceleration on mobile devices. He is a co-founder of the AI Witchlabs and co-organizer of the NTIRE and AIM events. His main line of research is focused on image restoration and automatic image quality enhancement, adaptation of AI applications for mobile devices and benchmarking machine learning hardware.

Abstract: In this tutorial, we will review the recent Android AI software stack updates, and will talk about the performance of the latest mobile chipsets from Qualcomm, MediaTek, Google, Samsung and Unisoc released during the past year. We will also discuss the power efficiency of mobile chipsets and their NPUs, and will analyze their energy consumption for a number of typical AI workloads.

Mobile AI Evolution - the Trend and Challenge

08:40 Pacific Time   ┈   Hsien-Kai Kuo   ┈   Senior Department Manager, MediaTek Inc.

09:00 Pacific Time   ┈   Yu-Syuan Xu   ┈   Senior AI Engineer, MediaTek Inc.

Biography: Hsien-Kai Kuo is a Senior Department Manager at MediaTek Inc. He received the PhD degree in electrical engineering from National Chiao Tung University, Taiwan, in 2014. He serves as a reviewer/author of several papers in CVPR’23/20, TC’16, CAL’15-16, TODAES’14, DAC’14, ASP-DAC’12-14. His research interests include deep learning, computer vision, computer architecture, electronic design automation, parallel and heterogeneous computing, multicore and GPGPU. He is also a member of IEEE. You can learn more about his work on his Google Scholar profile at https://scholar.google.com/citations?user=EeVsWaUAAAAJ.

Yu-Syuan Xu is an Senior AI engineer at MediaTek. His research mainly focuses on deep learning and its applications, specifically on developing and deploying the latest and trendiest AI models on edge devices equipped with MediaTek APU. He is responsible for co-designing AI accelerators from an algorithmic point of view. He received both his B.S. degree and M.S. degree in computer science from National Tsing Hua University in 2017 and 2018, respectively, under the guidance of Prof. Chun-Yi Lee. You can learn more about his work on his website at https://xusean0118.github.io/ and on his Google Scholar profile at https://scholar.google.com/citations?user=QA3VSaYAAAAJ.

Abstract: Mobile AI is evolving from short-burst applications like photos to sustained ones such as videos, games, and VR/MR. Recently, Transformer, designed for natural language processing, has become the foundation network of generative AI, and has shown superior performance in vision and voice applications for mobile devices. In this talk, we will introduce mobile AI applications, with a particular focus on the advantages and challenges of utilizing transformers. However, running vision transformer models with higher-resolution data under a limited power budget is the most critical challenge for mobile AI applications. We will set up a vision transformer based video super resolution challenge to promote the efficient deployment of transformer.

Make the World a Clearer Place

09:30 Pacific Time   ┈   Lan Fu   ┈   Senior Research Engineer, OPPO US R&D

Biography:

Abstract: Although deep learning based video super-resolution methods have achieved promising performance, they are so computation intensive that it is impractical to deploy them on edge devices demanding higher efficiency. Moreover, various degradations in real-world applications pose a great challenge to their generalization abilities. In this talk, we present a video super-resolution solution on mobile devices, which can be real-time and deal with a wide range of degradations as well. Extensive experiments demonstrate its state-of-the-art performance on a public video super-resolution dataset. We have successfully delivered it to OPPO smartphones.

CiaoSR:  Arbitrary-Scale Image Super-Resolution

09:55 Pacific Time   ┈   Jiezhang Cao   ┈   Computer Vision Researcher, ETH Zurich

Biography:

Abstract: Learning continuous image representations is recently gaining popularity for image super-resolution because of its ability to reconstruct high-resolution images with arbitrary scales from low-resolution inputs. Existing methods mostly ensemble nearby features to predict the new pixel at any queried coordinate in the SR image. Such a local ensemble suffers from some limitations: i) it has no learnable parameters and it neglects the similarity of the visual features; ii) it has a limited receptive field and cannot ensemble relevant features in a large field which are important in an image. To address these issues, we propose a continuous implicit attention-in-attention network, called CiaoSR. We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features. Furthermore, we embed a scale-aware attention in this implicit attention network to exploit additional non-local information. CiaoSR achieves the state-of-the-art performance on the arbitrary-scale SR task, the effectiveness of the method is also demonstrated on the real-world SR setting.