GENERALIZATION IN 2-LAYER NEURAL NETWORKS

Abstract

Generalization of a fully connected large two-layer neural networks with hidden nodes using the annealed approximation and the Monte Carlo simulation. When the number of output node is much smaller than the number of the hidden nodes, the system trapped be the metastable state and the generalization error converges to a constant value. When the number of ouput nodes are comparable to that of hidden nodes, the metastable state disappears and sysrem shows better generalization curve.