中国科学院重庆绿色智能技术研究院机构知识库

QQ客服 官方微博 反馈留言

	Enhancing Representation Power of Deep Neural Networks With Negligible Parameter Growth for Industrial Applications
	Chen, Liangming1,2; Jin, Long1,2; Shang, Mingsheng1,3; Wang, Fei-Yue4
	2024-05-01
摘要	In industrial applications where computational resources are finite and data noises are prevalent, the representation power of deep neural networks (DNNs) is crucial. Traditional network structures often require a significant increase in the parameter amount to enhance the representation power, making it difficult to achieve effective representation under parameter amount constraints. In order to alleviate this problem, this work leverages the ordinary differential equation (ODE) interpretation of deep residual networks, elucidating the relationship between the fine-grained connectivity modes of blocks in DNNs and the representation power. We build a bridge from the order of numerical methods and the order of ODEs to the representation power of DNNs. Besides, we show that higher-order ODEs can be approximated by k -step methods incorporating trainable coefficients. Empirically, we validate our theoretical insights by demonstrating the superior representation power of our proposed network structures through enhanced performance on industrial tasks, such as surface defect detection, critical temperature prediction of superconductors, and image classification under noises. The proposed method provides a new approach to the design of network structures for robust and accurate DNNs, enhancing the representation power with a negligible number of additional parameters. The code is publicly available at https://github.com/LongJin-lab/Order-and-Representation-Power.
关键词	Residual neural networks Vectors Noise Biological neural networks Medical services Defect detection Agriculture Deep neural networks (DNNs) industrial applications ordinary differential equation (ODE) representation power
DOI	10.1109/TSMC.2024.3387408
发表期刊	IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
ISSN	2168-2216
页码	12
通讯作者	Jin, Long(jinlongsysu@foxmail.com)
收录类别	SCI
WOS记录号	WOS:001214317700001
语种	英语