论文部分内容阅读
针对神经网络结构设计问题,提出一种基于神经网络复杂度的修剪算法.其实质是在训练过程中,利用网络连接权矩阵的协方差矩阵计算网络的信息熵,获得网络的复杂度;在保证网络信息处理能力的前提下,删除对网络复杂度影响最小的隐节点.该算法不要求训练网络到代价函数的极小点,适合在线修剪网络结构,并且避免了结构调整前的网络权值预处理.通过对典型函数逼近的实验结果表明,该算法在保证网络逼近精度的同时,可有效地简化网络结构.
In order to solve the problem of neural network structure design, a pruning algorithm based on the complexity of neural network is proposed. The essence of this algorithm is to compute the network entropy by using the covariance matrix of the network connection weight matrix during training. Network information processing ability, we remove the hidden nodes which have the least influence on the network complexity.The algorithm does not require training the network to the minimum point of the cost function, is suitable for online trimming the network structure, and avoids the network weight before structural adjustment The experimental results of approximation of typical functions show that this algorithm can effectively simplify the network structure while ensuring the network approximation accuracy.