Storage capacities of committee machines with overlapping and non-overlapping receptive fields
SCIE
SCOPUS
- Title
- Storage capacities of committee machines with overlapping and non-overlapping receptive fields
- Authors
- Kwon, C; Oh, JH
- Date Issued
- 1997-09-21
- Publisher
- IOP PUBLISHING LTD
- Abstract
- We present theoretical investigations via the replica theory of the storage capacities of committee machines with a large number M of hidden units and spherical weights. Difficulties arise in the solution of this problem in the limit of large M. In the case of overlapping receptive fields, as the number of patterns increases, both permutation symmetry and replica symmetry are broken, which leads to the appearance of many order parameters and causes additional difficulty. We observe that the relations among these order parameters yield a set of quantities which are small in the limit of large M, making the asymptotic calculation tractable. Using the one-step replica symmetry breaking scheme, we compute the asymptotic value alpha(c) of the storage capacity per input unit in the limit of large M. We find that alpha(c) similar or equal to (8 root 2/(pi-2))M root ln M. The shift to the case of non-overlapping receptive fields can be made easily; we then find alpha c similar or equal to (8 root 2/pi)root ln M. Both values satisfy the bound of Mitchison and Durbin.
- Keywords
- MULTILAYER NEURAL NETWORKS; SPIN-GLASS; INTERNAL REPRESENTATIONS; STATISTICAL-MECHANICS; MODELS; SPACE; EXAMPLES
- URI
- https://oasis.postech.ac.kr/handle/2014.oak/21217
- DOI
- 10.1088/0305-4470/30/18/011
- ISSN
- 0305-4470
- Article Type
- Article
- Citation
- JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, vol. 30, no. 18, page. 6273 - 6285, 1997-09-21
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.