Organization
School of Computing Associate Professor
Title
Associate Professor
Homepage
External link
SUN HEMING
Degree
-
Doctor of Engineering ( 2017.3 Waseda University )
Research Interests
-
Deep learning
-
Large Language Model
-
Computer Vision
-
Video Processing
-
Very Large Scale Integration Circuits
Research Areas
-
Informatics / Computer system
-
Manufacturing Technology (Mechanical Engineering, Electrical and Electronic Engineering, Chemical Engineering) / Electron device and electronic equipment
-
Informatics / Perceptual information processing
Education
-
Waseda University Graduate School of Information, Production and Systems Doctor of Engineering
2014.4 - 2017.3
-
Shanghai Jiao Tong University Graduate School of Electronics, Information and Electrical Engineering (SEIEE) Master of Engineering
2012.9 - 2014.3
-
Waseda University Graduate School of Information, Production and Systems Master of Engineering
2010.9 - 2012.9
-
Shanghai Jiao Tong University School of Electronics, Information and Electrical Engineering (SEIEE) Bachelor of Engineering
2007.9 - 2011.7
Research History
-
Institute of Science Tokyo Associate Professor
2026.4
-
Yokohama National University Associate Professor
2023.4 - 2026.3
-
Japan Science and Technology Agency Adjunct Researcher
2019.10 - 2023.3
-
Waseda University Assistant Professor
2018.9 - 2023.3
-
NEC Corporation Central Research Laboratories Researcher
2017.4 - 2018.9
-
The University of Tokyo
2016.7 - 2016.9
-
University of California, Davis
2015.8 - 2015.9
Professional Memberships
-
IEEE
-
IEICE
Committee Memberships
-
IEEE Journal on Emerging and Selected Topics in Circuits and Systems Guest Editor
2024.6
Committee type：Academic society
-
IEEE Transactions on Circuits and Systems for Video Technology Associate Editor
2023.1
Committee type：Academic society
-
Picture Coding Symposium Special Session Chair
2022.12
-
IEEE VCIP Area Chair
2021.12
-
IEEE ISCAS Session Chair
2021.5
-
Picture Coding Symposium Special Session Chair
2019.11
Committee type：Academic society
Papers
-
Real-time Video Prediction With Fast Video Interpolation Model and Prediction Training.
Shota Hirose, Kazuki Kotoyori, Kasidis Arunruangsirilert, Fangzheng Lin, Heming Sun, Jiro Katto
CoRR abs/2503.23185 2025.3
-
Hardware-Efficient Multipliers With FPGA-Based Approximation for Error-Resilient Applications.
Yi Guo, Qilin Zhou, Xiu Chen, Heming Sun
IEEE Transactions on Circuits and Systems I: Regular Papers 71 ( 12 ) 5919 - 5930 2024.12
-
Guest Editorial Advances in Generative Visual Signal Coding and Processing.
Zhibo Chen 0001, Heming Sun, Li Zhang 0006, Fan Zhang 0017
IEEE Journal of Emerging and Selected Topics in Circuits and Systems 14 ( 2 ) 145 - 148 2024.6
-
Fast Transform Kernel Selection Based on Frequency Matching and Probability Model for AV1.
Zhijian Hao, Heming Sun, Guohao Xu, Jiaming Liu, Xiankui Xiong, Xuanpeng Zhu, Xiaoyang Zeng, Yibo Fan
IEEE Transactions on Broadcasting 70 ( 2 ) 693 - 707 2024.6
-
FPGA Codec System of Learned Image Compression With Algorithm-Architecture Co-Optimization.
Heming Sun, Qingyang Yi, Masahiro Fujita
IEEE Journal of Emerging and Selected Topics in Circuits and Systems 14 ( 2 ) 334 - 347 2024.6
-
Zhibo Chen 0001, Heming Sun, Li Zhang 0006, Fan Zhang 0017
IEEE Journal of Emerging and Selected Topics in Circuits and Systems 14 ( 2 ) 149 - 171 2024.6
-
SCP: Spherical-Coordinate-Based Learned Point Cloud Compression
Ao Luo, Linxin Song, Keisuke Nonaka, Kyohei Unno, Heming Sun, Masayuki Goto, Jiro Katto
Proceedings of the AAAI Conference on Artificial Intelligence 38 ( 4 ) 3954 - 3962 2024.3
-
Hang Liu, Yuying Li, Haoyuan Shen, Bo Wu, Yu Jin, Duli Yu, Heming Sun
IEEE Transactions on Circuits and Systems II: Express Briefs 71 ( 3 ) 1047 - 1051 2024.3
-
Ran Wang 0015, Wen Jiang, Heming Sun, Jiro Katto
VCIP 1 - 5 2024
-
High-Efficiency FPGA - Based Approximate Multipliers with LUT Sharing and Carry Switching.
Yi Guo, Qilin Zhou, Xiu Chen, Heming Sun
Design(DATE) 1 - 2 2024
-
LMM-driven Semantic Image-Text Coding for Ultra Low-bitrate Learned Image Compression.
Shimon Murai, Heming Sun, Jiro Katto
VCIP 1 - 5 2024
-
Tell Codec What Worth Compressing: Semantically Disentangled Image Coding for Machine with LMMs.
Jinming Liu, Yuntao Wei, Junyan Lin, Shengyang Zhao, Heming Sun, Zhibo Chen 0001, Wenjun Zeng 0001, Xin Jin 0014
VCIP 1 - 5 2024
-
Lightweight Stochastic Video Prediction via Hybrid Warping.
Kazuki Kotoyori, Shota Hirose, Heming Sun, Jiro Katto
VCIP 1 - 5 2024
-
Accelerating Learnt Video Codecs with Gradient Decay and Layer-Wise Distillation.
Tianhao Peng 0004, Ge Gao, Heming Sun, Fan Zhang 0017, David Bull 0001
Picture Coding Symposium(PCS) 1 - 5 2024
-
Power-Efficient and Small-Area Approximate Multiplier Design with FPGA-Based Compressors.
Yi Guo 0010, Xiu Chen, Qilin Zhou, Heming Sun
IEEE International Symposium on Circuits and Systems(ISCAS) 1 - 5 2024
-
Privacy-preserving with Flexible Autoencoder for Video Coding for Machines.
Aorui Gou, Heming Sun, Xiaoyang Zeng, Yibo Fan
IEEE International Symposium on Circuits and Systems(ISCAS) 1 - 5 2024
-
Real-Time Video Prediction With Fast Video Interpolation Model and Prediction Training.
Shota Hirose, Kazuki Kotoyori, Kasidis Arunruangsirilert, Fangzheng Lin, Heming Sun, Jiro Katto
ICIP 2015 - 2021 2024
-
Recoil: Parallel rANS Decoding with Decoder-Adaptive Scalability
Fangzheng Lin, Kasidis Arunruangsirilert, Heming Sun, Jiro Katto
ACM International Conference Proceeding Series 31 - 40 2023.8
-
A Reconfigurable Multiple Transform Selection Architecture for VVC Reviewed
Zhijian Hao, Heming Sun, Guoqing Xiang, Peng Zhang, Xiaoyang Zeng, Yibo Fan
IEEE Transactions on Very Large Scale Integration (VLSI) Systems 2023.3
-
PTS-LIC: Pruning Threshold Searching for Lightweight Learned Image Compression
Ao Luo, Heming Sun, Jinming Liu, Fangzheng Lin, Jiro Katto
2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023 2023
-
Learned Lossless Image Compression With Combined Channel-Conditioning Models and Autoregressive Modules
Ran Wang, Jinming Liu, Heming Sun, Jiro Katto
IEEE Access 11 73462 - 73469 2023
-
Multistage Spatial Context Models for Learned Image Compression
Fangzheng Lin, Heming Sun, Jinming Liu, Jiro Katto
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings 2023-June 2023
-
Learned Image Compression with Mixed Transformer-CNN Architectures
Jinming Liu, Heming Sun, Jiro Katto
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2023-June 14388 - 14397 2023
-
iPhone 240fps Video Dataset for Various Model Training Tasks
Joi Shimizu, Shion Komatsu, Satsuki Kobayashi, Heming Sun, Jiro Katto
Digest of Technical Papers - IEEE International Conference on Consumer Electronics 2023-January 2023
-
ABCAS: Adaptive Bound Control of spectral norm as Automatic Stabilizer
Shota Hirose, Shiori Maki, Naoki Wada, Jiro Katto, Heming Sun
Digest of Technical Papers - IEEE International Conference on Consumer Electronics 2023-January 2023
-
A Highly Pipelined and Highly Parallel VLSI Architecture of CABAC Encoder for UHDTV Applications.
Chen Fu, Heming Sun, Zhiqiang Zhang, Jinjia Zhou
Sensors 23 ( 9 ) 4293 - 4293 2023
-
Q-LIC: Quantizing Learned Image Compression with Channel Splitting Reviewed
Heming Sun, Lu Yu, Jiro Katto
IEEE Transactions on Circuits and Systems for Video Technology 2022.12
-
Deep Image Compression Based on Multi-scale Deformable Convolution Reviewed
Daowen Li, Yingming Li, Heming Sun, Lu Yu
Journal of Visual Communication and Image Representation 2022.8
-
QA-Filter: A QP-Adaptive Convolutional Neural Network Filter for Video Coding Reviewed
Chao Liu, Heming Sun, Jiro Katto, Xiaoyang Zeng, Yibo Fan
IEEE Transactions on Image Processing 31 3032 - 3045 2022.4
-
F-LIC: FPGA-based Learned Image Compression with a Fine-grained Pipeline
Heming Sun, Qingyang Yi, Fangzheng Lin, Lu Yu, Jiro Katto, Masahiro Fujita
2022 IEEE Asian Solid-State Circuits Conference, A-SSCC 2022 - Proceedings 2022
-
Semantic Segmentation in Learned Compressed Domain
Jinming Liu, Heming Sun, Jiro Katto
2022 Picture Coding Symposium, PCS 2022 - Proceedings 181 - 185 2022
-
An Efficient Low-Complexity Convolutional Neural Network Filter Reviewed
Chao Liu, Heming Sun, Jiro Katto, Xiaoyang Zeng, Yibo Fan
IEEE Multimedia 29 ( 2 ) 83 - 93 2022
-
MEMORY-EFFICIENT LEARNED IMAGE COMPRESSION WITH PRUNED HYPERPRIOR MODULE
Ao Luo, Heming Sun, Jinming Liu, Jiro Katto
Proceedings - International Conference on Image Processing, ICIP 3061 - 3065 2022
-
STREAMING-CAPABLE HIGH-PERFORMANCE ARCHITECTURE OF LEARNED IMAGE COMPRESSION CODECS
Fangzheng Lin, Heming Sun, Jiro Katto
Proceedings - International Conference on Image Processing, ICIP 286 - 290 2022
-
Improving Latent Quantization of Learned Image Compression with Gradient Scaling
Heming Sun, Lu Yu, Jiro Katto
2022 IEEE International Conference on Visual Communications and Image Processing, VCIP 2022 2022
-
Real-time Learned Image Codec on FPGA
Heming Sun, Qingyang Yi, Fangzheng Lin, Lu Yu, Jiro Katto, Masahiro Fujita
2022 IEEE International Conference on Visual Communications and Image Processing, VCIP 2022 2022
-
Fast Intra Mode Decision for VVC Based on Histogram of Oriented Gradient
Aorui Gou, Heming Sun, Jiro Katto, Tingting Li, Xiaoyang Zeng, Yibo Fan
Proceedings - IEEE International Symposium on Circuits and Systems 2022-May 3028 - 3032 2022
-
Improving Multiple Machine Vision Tasks in the Compressed Domain
Jinming Liu, Heming Sun, Jiro Katto
Proceedings - International Conference on Pattern Recognition 2022-August 331 - 337 2022
-
Enhanced Intra Prediction for Video Coding by Using Multiple Neural Networks Reviewed
Heming Sun, Zhengxue Cheng, Masaru Takeuchi, Jiro Katto
IEEE Transactions on Multimedia 22 ( 11 ) 2764 - 2779 2020.11
-
A Pipelined 2D Transform Architecture Supporting Mixed Block Sizes for the VVC Standard Reviewed
Yibo Fan, Yixuan Zeng, Heming Sun, Jiro Katto, Xiaoyang Zeng
IEEE Transactions on Circuits and Systems for Video Technology 30 ( 9 ) 3289 - 3295 2020.9
-
Energy Compaction-Based Image Compression Using Convolutional AutoEncoder Reviewed
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
IEEE Transactions on Multimedia 22 ( 4 ) 860 - 873 2020.4
-
Small-Area and Low-Power FPGA-Based Multipliers using Approximate Elementary Modules
Yi Guo, Heming Sun, Shinji Kimura
Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC 2020- 599 - 604 2020.1
-
Approximate FPGA-Based Multipliers Using Carry-Inexact Elementary Modules.
Yi Guo, Heming Sun, Ping Lei, Shinji Kimura
IEICE Trans. Fundam. Electron. Commun. Comput. Sci. 103-A ( 9 ) 1054 - 1062 2020
-
A Fast QTMT Partition Decision Strategy for VVC Intra Prediction Reviewed
Yibo Fan, Jun'An Chen, Heming Sun, Jiro Katto, Ming'E Jing
IEEE Access 8 107900 - 107911 2020
-
Energy-Efficient and High-Speed Approximate Signed Multipliers with Sign-Focused Compressors
Yi Guo, Heming Sun, Shinji Kimura
International System on Chip Conference 2019- 330 - 335 2019.9
-
Design of Power and Area Efficient Lower-Part-OR Approximate Multiplier
Yi Guo, Heming Sun, Shinji Kimura
IEEE Region 10 Annual International Conference, Proceedings/TENCON 2018- 2110 - 2115 2019.2
-
Low-Cost Approximate Multiplier Design using Probability-Driven Inexact Compressors
Yi Guo, Heming Sun, Li Guo, Shinji Kimura
2018 IEEE Asia Pacific Conference on Circuits and Systems, APCCAS 2018 291 - 294 2019.1
-
Approximate DCT Design for Video Encoding Based on Novel Truncation Scheme. Reviewed
Heming Sun, Zhengxue Cheng, Amir Masoud Gharehbaghi, Shinji Kimura, Masahiro Fujita
IEEE Trans. Circuits Syst. I Regul. Pap. 66-I ( 4 ) 1517 - 1530 2019
-
Deep Residual Learning for Image Compression.
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019
-
Design of Low-Cost Approximate Multipliers Based on Probability-Driven Inexact Compressors. Reviewed
Yi Guo, Heming Sun, Ping Lei, Shinji Kimura
IEICE Trans. Fundam. Electron. Commun. Comput. Sci. 102-A ( 12 ) 1781 - 1791 2019
-
Gamut-Extension Methods Considering Color Information Restoration Reviewed
Masaru Takeuchi, Yusuke Sakamoto, Ryota Yokoyama, Heming Sun, Yasutaka Matsuo, Jiro Katto
IEEE Access 7 80146 - 80158 2019
-
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
2018 IEEE Conference on Computer Vision and Pattern Recognition Workshops 2613 - 2616 2018
-
Fast Algorithm and VLSI Architecture of Rate Distortion Optimization in H.265/HEVC Reviewed
Heming Sun, Dajiang Zhou, Landan Hu, Shinji Kimura, Satoshi Goto
IEEE TRANSACTIONS ON MULTIMEDIA 19 ( 11 ) 2375 - 2390 2017.11
-
Accelerating HEVC Inter Prediction with Improved Merge Mode Handling Reviewed
Zhengxue Cheng, Heming Sun, Dajiang Zhou, Shinji Kimura
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES E100A ( 2 ) 546 - 554 2017.2
-
An 8K H.265/HEVC Video Decoder Chip With a New System Pipeline Design Reviewed
Dajiang Zhou, Shihao Wang, Heming Sun, Jianbin Zhou, Jiayi Zhu, Yijin Zhao, Jinjia Zhou, Shuping Zhang, Shinji Kimura, Takeshi Yoshimura, Satoshi Goto
IEEE JOURNAL OF SOLID-STATE CIRCUITS 52 ( 1 ) 113 - 126 2017.1
-
A Low-Power VLSI Architecture for HEVC De-Quantization and Inverse Transform Reviewed
Heming Sun, Dajiang Zhou, Shuping Zhang, Shinji Kimura
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES E99A ( 12 ) 2375 - 2387 2016.12
-
Human Detection Method Based on Non-Redundant Gradient Semantic Local Binary Patterns Reviewed
Jiu Xu, Ning Jiang, Wenxin Yu, Heming Sun, Satoshi Goto
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES E98A ( 8 ) 1735 - 1742 2015.8
-
A Low-Cost VLSI Architecture of Multiple-Size IDCT for H.265/HEVC Reviewed
Heming Sun, Dajiang Zhou, Peilin Liu, Satoshi Goto
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES E97A ( 12 ) 2467 - 2476 2014.12
-
Fast Prediction Unit Selection and Mode Selection for HEVC Infra Prediction Reviewed
Heming Sun, Dajiang Zhou, Peilin Liu, Satoshi Goto
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES E97A ( 2 ) 510 - 519 2014.2
-
Real-Time Human Detection Based on Multi-Scale Bidirectional Local Template Patterns Reviewed
Jiu Xu, Ning Jiang, Heming Sun, Axel Beaugendre, Satoshi Goto
IIEEJ Transactions on Electronics and Visual Computing 1 ( 1 ) 28 - 37 2013.12
MISC
-
Lightweight model architecture towards Real-time Video Prediction
廣瀬翔太, 琴寄和樹, SUN Heming, 甲藤二郎
画像符号化シンポジウム・映像メディア処理シンポジウム(Web) 39th-29th 2024
-
End-to-End Learned Image and Video Compression: Design, Implementation, and Computer Vision Applications
Wen-Hsiao Peng, Heming Sun
IEEE/CVF International Conference on Computer Vision, tutorial 2023.10
-
Advances in Design and Implementation of End-to-End Learned Image and Video Compression
Wen-Hsiao Peng, Heming Sun
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), tutorial 2023.1
-
On the Channel Slicing Method of the Context Model in Learned Image Compression
村井史門, LIN Fangzheng, SUN Heming, 甲藤二郎
画像符号化シンポジウム・映像メディア処理シンポジウム(Web) 38th-28th 2023
-
A Frame Extrapolation Method using Forward and Backward Warpings
琴寄和樹, SUN Heming, 甲藤二郎
画像符号化シンポジウム・映像メディア処理シンポジウム(Web) 38th-28th 2023
-
An Approach towards Video Prediction Training Using Frame Interpolation Model
廣瀬翔太, SUN Heming, 甲藤二郎
画像符号化シンポジウム・映像メディア処理シンポジウム(Web) 38th-28th 2023
-
Zero Latency Interaction of Video and Somatic Integrated Services over B5G Networks
甲藤二郎, 金井謙治, SUN Heming, WEI Bo, 勝山裕, ZHENG Wen, 中村裕一, 近藤一晃, 下西慶, 小野浩司, 根波健一, 青木智資, 片野淳一, 吉岡修一, 作中剛, 小林康雄, 小沢基一, 秋田純一
電子情報通信学会大会講演論文集(CD-ROM) 2022 2022
-
深層学習を利用した超解像のINT8実装に関する調査と考察
廣瀬翔太, 和田直己, SUN Heming, 甲藤二郎, 甲藤二郎
電子情報通信学会技術研究報告(Web) 121 ( 346(IE2021 27-33) ) 2022
-
Development of Zero Latency Video and Somatic Integration Network with Low Latency and Interaction-Future prediction and integration technology of video and Somatic information-
勝山裕, ZHENG Wen, 金井謙治, 金井謙治, SUN Heming, WEI Bo, 甲藤二郎, 甲藤二郎, 甲藤二郎
電子情報通信学会大会講演論文集(CD-ROM) 2022 2022
-
Advances in Design and Implementation of End-to-End Learned Image and Video Compression
Wen-Hsiao Peng, Heming Sun
IEEE ISCAS, tutorial 2021.5
-
Deep Learning Method for Image Compression
Heming Sun
Information Processing Society of Japan 2021.2
-
Research and examination on implementation of super-resolution models using deep learning with INT8 accuracy
廣瀬翔太, 和田直己, SUN Heming, SUN Heming, 甲藤二郎
画像符号化シンポジウム・映像メディア処理シンポジウム(Web) 36th-26th 2021
-
VVC Video Coding with Deep Learning Based Frame Interpolation
清水盛偉, SUN Heming, SUN Heming, 甲藤二郎
電子情報通信学会技術研究報告(Web) 120 ( 389(IMQ2020 10-35) ) 2021
-
A Study on Compression Artifacts Reduction by a Super-Resolution Network
小松蒔遠, 清水盛偉, SUN Heming, SUN Heming, 甲藤二郎
電子情報通信学会技術研究報告(Web) 121 ( 54(SIP2021 1-10) ) 2021
-
Application of Deep Learning Based Frame Interpolation to HEVC Video Coding
清水盛偉, CHENG Zhengxue, SUN Heming, SUN Heming, 竹内健, 甲藤二郎
情報処理学会研究報告(Web) 2020 ( AVM-111 ) 2020
-
Video Coding Using Optimal Infra Prediction Mode Estimation by CNN
横山怜汰, 田原雅彦, SUN Heming, SUN Heming, 竹内健, 松尾康孝, 甲藤二郎
電子情報通信学会技術研究報告 119 ( 421(ITS2019 30-56) ) 2020
-
CNNを用いた動画像符号化における最適Intra予測モードの推定
横山怜汰, 田原雅彦, SUN Heming, 竹内健, 松尾康孝, 甲藤二郎
画像符号化シンポジウム・映像メディア処理シンポジウム 34th-24th 2019
Presentations
-
Improving Latent Quantization of Learned Image Compression with Gradient Scaling
Heming Sun, Lu Yu, Jiro Katto
IEEE International Conference on Visual Communications and Image Processing 2022.12
-
F-LIC: FPGA-based Learned Image Compression with a Fine-grained Pipeline
Heming Sun, Qingyang Yi, Fangzheng Lin, Lu Yu, Jiro Katto, Masahiro Fujita
IEEE Asian Solid-State Circuits Conference 2022.11
-
Learned Video Compression with Residual Prediction and Feature-aided Loop Filter
Chao Liu, Heming Sun, Xiaoyang Zeng, Yibo Fan
IEEE International Conference on Image Processing 2022.10
-
Streaming-capable High-performance Architecture of Learned Image Compression Codecs
Fangzheng Lin, Heming Sun, Jiro Katto
IEEE International Conference on Image Processing 2022.10
-
Memory-Efficient Learned Image Compression with Pruned Hyperprior Module
Ao Luo, Heming Sun, Jinming Liu, Jiro Katto
IEEE International Conference on Image Processing 2022.10
-
Improving Multiple Machine Vision Tasks in the Compressed Domain
Jinming Liu, Heming Sun, Jiro Katto
International Conference on Pattern Recognition 2022.8
-
A QP-adaptive Mechanism for CNN-based Filter in Video Coding
Chao Liu, Heming Sun, Jiro Katto, Xiaoyang Zeng, Yibo Fan
IEEE International Symposium on Circuits and Systems 2022.5
-
An Area-efficient Unified Transform Architecture for VVC
Zhijian Hao, Qi Zheng, Yibo Fan, Guoqing Xiang, Peng Zhang, Heming Sun
IEEE International Symposium on Circuits and Systems 2022.5
-
Fast Intra Mode Decision for VVC Based on Histogram of Oriented Gradient
Aorui Gou, Heming Sun, Jiro Katto, Tingting Li, Xiaoyang Zeng, Yibo Fan
IEEE International Symposium on Circuits and Systems 2022.5
-
Research and examination on implementation of super-resolution models using deep learning with INT8 precision
Shota Hirose, Naoki Wada, Jiro Katto, Heming Sun
4th International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2022 - Proceedings 2022
-
Forward and Backward Warping for Optical Flow-Based Frame Interpolation
Joi Shimizu, Heming Sun, Jiro Katto
4th International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2022 - Proceedings 2022
-
ViT-GAN: Using Vision Transformer as Discriminator with Adaptive Data Augmentation
Shota Hirose, Naoki Wada, Jiro Katto, Heming Sun
2021 3rd International Conference on Computer Communication and the Internet, ICCCI 2021 2021.6
-
Learned Image Compression with Fixed-point Arithmetic
Heming Sun, Lu Yu, Jiro Katto
Picture Coding Symposium 2021.6
-
A Hardware Architecture for Adaptive Loop Filter in VVC Decoder
Xin Wang, Heming Sun, Jiro Katto, Yibo Fan
Proceedings of International Conference on ASIC 2021
-
Approximated reconfigurable transform architecture for VVC
Yixuan Zeng, Heming Sun, Jiro Katto, Yibo Fan
Proceedings - IEEE International Symposium on Circuits and Systems 2021
-
Accelerating convolutional neural network inference based on a reconfigurable sliced systolic array
Yixuan Zeng, Heming Sun, Jiro Katto, Yibo Fan
Proceedings - IEEE International Symposium on Circuits and Systems 2021
-
Deep Pedestrian Density Estimation For Smart City Monitoring.
Kazuki Murayama, Kenji Kanai, Masaru Takeuchi, Heming Sun, Jiro Katto
ICIP 2021
-
Learning in Compressed Domain for Faster Machine Vision Tasks
Jinming Liu, Heming Sun, Jiro Katto
2021 International Conference on Visual Communications and Image Processing, VCIP 2021 - Proceedings 2021
-
Fast Object Detection in HEVC Intra Compressed Domain
Liuhong Chen, Heming Sun, Jiro Katto, Xiaoyang Zeng, Yibo Fan
European Signal Processing Conference 2021
-
Fully Neural Network Mode Based Intra Prediction of Variable Block Size
Heming Sun, Lu Yu, Jiro Katto
IEEE International Conference on Visual Communications and Image Processing (VCIP) 2020.12
-
HEVC video coding with deep learning based frame interpolation
Joi Shimizu, Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
2020 IEEE 9th Global Conference on Consumer Electronics, GCCE 2020 2020.10
-
Scalable Learned Image Compression With A Recurrent Neural Networks-Based Hyperprior
Rige Su, Zhengxue Cheng, Heming Sun, Jiro Katto
2020 IEEE International Conference on Image Processing (ICIP) 2020.10 IEEE
-
End-To-End Learned Image Compression With Fixed Point Weight Quantization
Heming Sun, Zhengxue Cheng, Masaru Takeuchi, Jiro Katto
2020 IEEE International Conference on Image Processing (ICIP) 2020.10 IEEE
-
A Learning-Based Low Complexity in-Loop Filter for Video Coding
Chao Liu, Heming Sun, Jiro Katto, Xiaoyang Zeng, Yibo Fan
2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) 2020.7 IEEE
-
Low Bitrate Image Compression with Discretized Gaussian Mixture Likelihoods
Zhengxue Cheng, Heming Sun, Jiro Katto
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2020.6 IEEE
-
An Image Compression Framework with Learning-based Filter
Heming Sun, Chao Liu, Jiro Katto, Yibo Fan
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2020.6 IEEE
-
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020.6 IEEE
-
Learned Lossless Image Compression with A Hyperprior and Discretized Gaussian Mixture Likelihoods
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2020.5 IEEE
-
CNN Based Optimal Intra Prediction Mode Estimation in Video Coding
Ryota Yokoyama, Masahiko Tahara, Masaru Takeuchi, Heming Sun, Yasutaka Matsuo, Jiro Katto
IEEE International Conference on Consumer Electronics (ICCE) 2020.1
-
Small-Area and Low-Power FPGA-Based Multipliers using Approximate Elementary Modules.
Yi Guo, Heming Sun, Shinji Kimura
Asia and South Pacific Design Automation Conference (ASP-DAC) 2020 IEEE
-
Fast Variance- and Gradient-based QTMT Partition Decision Algorithm in VVC Intra Coding
Jun’an Chen, Heming Sun, Jiro Katto, Xiaoyang Zeng, Yibo Fan
IEEE International Conference on Visual Communications and Image Processing (VCIP) 2019.12
-
Dual Learning-based Video Coding with Inception Dense Blocks
Chao Liu, Heming Sun, Jun’an Chen, Zhengxue Cheng, Masaru Takeuchi, Jiro Katto, Xiaoyang Zeng, Yibo Fan
Picture Coding Symposium (PCS) 2019.11
-
Road Infrastructure Monitoring System using E-Bikes and Its Extensions for Smart Community
Jiro Katto, Masaru Takeuchi, Kenji Kanai, Heming Sun
Proceedings of the Annual International Conference on Mobile Computing and Networking, MOBICOM 2019.10
-
A Gamut Extension Method considering Color Information Restoration using Convolutional Neural Networks
Masaru Takeuchi, Yusuke Sakamoto, Ryota Yokoyama, Heming Sun, Yasutaka Matsuo, Jiro Katto
IEEE International Conference on Image Processing (ICIP) 2019.9
-
Perceptual Quality Study on Deep Learning based Image Compression
Zhengxue Cheng, Pinar Akyazi, Heming Sun, Jiro Katto, Touradj Ebrahimi
IEEE International Conference on Image Processing (ICIP) 2019.9
-
Deep Residual Learning for Image Compression
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2019.6
-
A MinimalAdder-oriented 1D DST-VII/DCT-VIII Hardware Implementation for VVC Standard
Yixuan Zeng, Heming Sun, Jiro Katto, Xiaoyang Zeng, Yibo Fan
IEEE International System-on-chip Conference (ISOCC) 2019.6
-
Learning Image and Video Compression through Spatial-Temporal Energy Compaction
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
Conference on Computer Vision and Pattern Recognition (CVPR) 2019.6
-
Energy-Efficient and High-Speed Approximate Signed Multipliers with Sign-Focused Compressors.
Yi Guo, Heming Sun, Shinji Kimura
IEEE International System-on-chip Conference (ISOCC) 2019 IEEE
-
Deep Convolutional AutoEncoder-based Lossy Image Compression
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
2018 Picture Coding Symposium, PCS 2018 - Proceedings 2018.9
-
Lossy Image Compression using Deep Convolutional AutoEncoder
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
IEICE technical report 2018.6
-
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2018.6
-
Deep Convolutional AutoEncoder-based Lossy Image Compression
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto
Picture Coding Symposium 2018.6
-
Low-Cost Approximate Multiplier Design using Probability-Driven Inexact Compressors.
Yi Guo, Heming Sun, Li Guo 0006, Shinji Kimura
IEEE Asia Pacific Conference on Circuits and Systems 2018 IEEE
-
Design of Power and Area Efficient Lower-Part-OR Approximate Multiplier.
Yi Guo, Heming Sun, Shinji Kimura
IEEE Region 10 Conference (TENCON) 2018 IEEE
-
Canran Jin, Heming Sun, Shinji Kimura
Asia and South Pacific Design Automation Conference (ASP-DAC) 2018 IEEE
-
Time-efficient and TSV-aware 3D gated clock tree synthesis based on self-tuning spectral clustering
Fan Yang, Minghao Lin, Heming Sun, Shinji Kimura
Midwest Symposium on Circuits and Systems 2017.9 Institute of Electrical and Electronics Engineers Inc.
-
A low-cost approximate 32-point transform architecture
Heming Sun, Zhengxue Cheng, Amir Masoud Gharehbaghi, Shinji Kimura, Masahiro Fujita
Proceedings - IEEE International Symposium on Circuits and Systems 2017.9 Institute of Electrical and Electronics Engineers Inc.
-
High Accuracy 8×8 Approximate Multiplier based on OR Operation
Yi Guo, Heming Sun, Canran Jin, Shinji Kimura
IEICE technical report 2017.3
-
A 4Gpixel/s 8/10b H.265/HEVC Video Decoder Chip for 8K Ultra HD Applications
Dajiang Zhou, Shihao Wang, Heming Sun, Jianbin Zhou, Jiayi Zhu, Yijin Zhao, Jinjia Zhou, Shuping Zhang, Shinji Kimura, Takeshi Yoshimura, Satoshi Goto
2016 IEEE INTERNATIONAL SOLID-STATE CIRCUITS CONFERENCE (ISSCC) 2016 IEEE
-
Power-Efficient and Slew-Aware Three Dimensional Gated Clock Tree Synthesis
Minghao Lin, Heming Sun, Shinji Kimura
2016 IFIP/IEEE INTERNATIONAL CONFERENCE ON VERY LARGE SCALE INTEGRATION (VLSI-SOC) 2016 IEEE
-
A fast level filtering algorithm for inter prediction in HEVC encoder
Zhengxue Cheng, Heming Sun, Landan Hu, Shinji Kimura
International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC) 2015.6
-
HARDWARE-ORIENTED RATE-DISTORTION OPTIMIZATION ALGORITHM FOR HEVC INTRA-FRAME ENCODER
Landan Hu, Heming Sun, Dajiang Zhou, Shinji Kimura
2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) 2015 IEEE
-
Merge Mode Based Fast Inter Prediction for HEVC
Zhengxue Cheng, Heming Sun, Dajiang Zhou, Shinji Kimura
2015 VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP) 2015 IEEE
-
A fast mode selection algorithm for HEVC intra prediction
Heming Sun, Satoshi Goto
International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC) 2014.7
-
Low-complexity rate-distortion optimization algorithms for HEVC intra prediction
Zhe Sheng, Dajiang Zhou, Heming Sun, Satoshi Goto
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2014
-
AN AREA-EFFICIENT 4/8/16/32-POINT INVERSE DCT ARCHITECTURE FOR UHDTV HEVC DECODER
Heming Sun, Dajiang Zhou, Jiayi Zhu, Shinji Kimura, Satoshi Goto
2014 IEEE VISUAL COMMUNICATIONS AND IMAGE PROCESSING CONFERENCE 2014 IEEE
-
VLSI ARCHITECTURE OF HEVC INTRA PREDICTION FOR 8K UHDTV APPLICATIONS
Jianbin Zhou, Dajiang Zhou, Heming Sun, Satoshi Goto
2014 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP) 2014 IEEE
-
Multi-scale Bidirectional Local Template Patterns for Real-time Human Detection
Jiu Xu, Ning Jiang, Xinwei Xue, Heming Sun, Wenxin Yu, Satoshi Goto
2013 IEEE 15TH INTERNATIONAL WORKSHOP ON MULTIMEDIA SIGNAL PROCESSING (MMSP) 2013 IEEE
-
A low-complexity HEVC intra prediction algorithm based on level and mode filtering
Heming Sun, Dajiang Zhou, Satoshi Goto
Proceedings - IEEE International Conference on Multimedia and Expo 2012
-
Real-time Learned Image Codec on FPGA
Heming Sun, Qingyang Yi, Fangzheng Lin, Lu Yu, Jiro Katto, Masahiro Fujita
IEEE International Conference on Visual Communications and Image Processing (VCIP) 2022.12
-
Learned Image Compression with Mixed Transformer-CNN Architectures
Jinming Liu, Heming Sun, Jiro Katto
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023.6
-
Multistage Spatial Context Models for Learned Image Compression
Fangzheng Lin, Heming Sun, Jinming Liu, Jiro Katto
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2023.6
-
ABCAS: Adaptive Bound Control of spectral norm as Automatic Stabilizer
Shota Hirose, Shiori Maki, Naoki Wada, Jiro Katto, Heming Sun
IEEE International Conference on Consumer Electronics (ICCE) 2023.1
-
iPhone 240fps Video Dataset for Various Model Training Tasks
Joi Shimizu, Shion Komatsu, Satsuki Kobayashi, Heming Sun, Jiro Katto
IEEE International Conference on Consumer Electronics (ICCE) 2023.1
Awards
-
IPSJ/IEEE Computer Society Young Computer Researcher Award
2024.7
-
The Telecommunications Advancement Foundation Award
2023.3
-
Ando Incentive Prize for the Study of Electronics
2022.6 The Foundation of ANDO Laboratory
-
Picture Coding Symposium Top-10 Paper
2021.6
-
IEEE VCIP Best Paper Award
2020.12
-
Young Researcher Award
2020.1 Kenjiro Takayanagi Foundation
-
Picture Coding Symposium Grand Challenge on Short Video Coding Silver Award
2019.11
-
CVPR Workshop and Challenge on Learned Image Compression MOS Rank 5
2019.6
-
Student Award
2018.3 The Telecommunications Advancement Foundation
-
VDEC Design Award
2017.9 VLSI Design and Education Center, University of Tokyo
-
ISSCC 2016 Takuo Sugano Award for Outstanding Far-East Paper
2016.2
Research Projects
-
学習型静止画像圧縮の実用化に関する研究
Grant number：23K16861 2023.4 - 2026.3
日本学術振興会 科学研究費助成事業 若手研究
孫 鶴鳴
Authorship：Principal investigator
Grant amount：\4680000 （ Direct Cost: \3600000 、 Indirect Cost：\1080000 ）
This year, I mainly worked on two topics. First is the algorithm-architecture co-optimization of learned image compression (LIC) on FPGA. Based on a pipelined architecture, the input and output channel parallelism is restricted to ease the routing phase, so that more DSP can be used. After that, the neural network channel is searched to improve the DSP efficiency. With high DSP utilization and efficiency, compared with the recent work, the throughput can be improved by at most 1.5x. Besides, the compression efficiency will not be affected after the neural network search.
Second is the privacy of LIC for machine. The overall framework includes the client side which captures image and the cloud side which conducts the machine vision. To avoid the privacy leakage in the cloud side, feature of captured image is generated at the client side, coded by an LIC autoencoder and sent to the cloud side. The cloud side then decodes the feature and performs the machine vision based on the decoded feature. By optimizing the layer number of feature, we can not only reach a good trade-off between rate for the feature transmission and accuracy for the machine vision, but also avoid leaking the privacy information in the cloud. -
再構成アクセラレータのための近似最適化手法
Grant number：23K28056 2023.4 - 2026.3
日本学術振興会 科学研究費助成事業 基盤研究(B)
木村 晋二, 戸川 望, 孫 鶴鳴
Grant amount：\18980000 （ Direct Cost: \14600000 、 Indirect Cost：\4380000 ）
今年度は、再構成アクセラレータ向けのデータ表現法、Ad Hoc な近似演算器の設計手法、およびシステマティックな近似回路の合成手法の文献調査を行った。近似演算器の場合は誤差と電力などとのトレードオフの下で設計最適化を行うので、誤差の評価は非常に重要である。Ad Hoc な近似乗算器の設計法に関しては、乗算における部分積の各桁の積算のための圧縮機に着目し、同じ重みで2つに圧縮する圧縮機を用いた乗算回路、新たな部分積の圧縮機の提案と種類の異なる圧縮機を用いた誤差削減を用いた乗算回路、誤差が正負の方向に同じ確率で現れるバイアスのない乗算回路についてパレート最適化と評価を行い、国際会議において発表した。また、各桁に符号をつけた符号付二進数の最適化に基づく8-ポイントの近似 DCT (Discrete Cosign Transformation) 回路の設計と評価を行い、国際会議において発表した。DCT の定数係数との乗算では、符号付二進数を用いることで連続した1からなる数字との乗算を一回の減算に変換できるので、出力への影響に基づいて係数をなるべく簡単な符号付二進数に近似することで演算のハードウェア資源を大きく削減している。近似回路の自動合成に向けては、与えられた論理関数を厳密に最小の素子数で合成する手法の検討を行い、3入力中の2つ以上が1であるときに出力が1となる多数決演算向けの厳密合成手法の提案を行った。厳密合成では、素子数の少ない順に、すべての構造を調べて目的の論理関数が実現できるかをチェックするが、素子数が大きいと、すべての構造を一度にチェックするよりも、クラスタに分けてチェックする方が効率的となるため、各素子の入力のレベルでクラスタ化する手法を提案し、他のクラスタ化法と比較して全体の合成時間を削減できることを示し、論文誌に掲載した。
-
再構成アクセラレータのための近似最適化手法
Grant number：23H03366 2023.4 - 2026.3
日本学術振興会 科学研究費助成事業 基盤研究(B)
木村 晋二, 戸川 望, 孫 鶴鳴
Grant amount：\18980000 （ Direct Cost: \14600000 、 Indirect Cost：\4380000 ）
-
Low-complexity research for next-generation VVC standard and its neural network extension
Grant number：21K17770 2021.4 - 2023.3
Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research Grant-in-Aid for Early-Career Scientists Grant-in-Aid for Early-Career Scientists
Grant amount：\2730000 （ Direct Cost: \2100000 、 Indirect Cost：\630000 ）
-
Real-time Low-power Innovative Learning-based Video Compression System
Grant number：19206134 2019.10 - 2023.3
Teaching Experience
-
Theory and Applications of Neural Networks
2019.9
-
Fundamentals of Programming
2019.4
Academic Activities
-
IEEE Signal Processing Letters
Role(s)： Peer review
-
IEEE Transactions on Pattern Analysis and Machine Intelligence
Role(s)： Peer review
-
IEEE Transactions on Multimedia
Role(s)： Peer review
-
IEEE Transactions on Circuits and Systems I: Regular Papers
Role(s)： Peer review
-
IEEE Transactions on Circuits and Systems—II: Express Briefs
Role(s)： Peer review
-
IEEE Transactions on Circuits and Systems for Video Technology
Role(s)： Peer review
-
International Journal of Computer Vision
Role(s)： Peer review
-
ACM Transactions on Reconfigurable Technology and Systems
Role(s)： Peer review