Artificial neural networks in various configurations are showing great potential in a number of applications.

**Multi-Layer Neural Networks**

A multi-layer neural network is built using an information-preserving criterion.The structure provides a potentially powerful substrate for classificationa and filtering.

- .A.N. Palmieri, M. Baldi, A. Buonanno and G. Di Gennaro, "
*Information-Preserving Networks and the Mirrored*," in*Transform**Proc. XXIX IEEE International Workshop on Machine Learning for Signal**Processing*(MLSP2019), Pittsburgh, PA, USA, Oct. 13–16, 2019. DOI: 10.1109/MLSP.2019.8918805

Detailed analyses of the activations in a popular multi-layer network, revel the intrinsec workings of the system in providing progressively high-level embeddings of the information contained in the sensed image.

- F.A.N. Palmieri, M. Baldi, A. Buonanno, G. Di Gennaro and F. Ospedale, "
," in*Probing a Deep Neural Network**Neural Approache**s to Dynamics of Signal Exchanges*(Smart Innovation, Systems and Technologies 151), A. Esposito, M.Faundez-Zanuy, F.C. Morabito and E. Pasero, Eds., Springer, 2019, pp. 201–211. DOI: 10.1007/978-981-13-8950-4_19 - Also Presented at the 28th Italian Workshop on Neural Networks (WIRN), June 13-15, 2018.

A cascade of linear layers, with constrained connectivity, that maximize decorrelation with an energy criterion, is shown to converge to a global Principal component analyzer.

- F. Palmieri, M. Corvino, "
," Proceedings of*Principal Components Via Cascades of Block-Layers**IEEE International Conference on Neural Networks*, Houston, TX, pp. 1035-1040, June 1997.

Efficient algorithms, obtained from a Kalman filter-like formulation, are derived for a multi-layer neural network.

- S. Shah, F. Palmieri and M. Datum, "
," in*Optimal Filtering Algorithms for Fast Learning in Feedforward Neural Networks**Neural Networks*, Vol. 5, pp. 779-787, Sept. 1992. - F. Palmieri, S. Shah, "
," Proceedings of*Fast Training of Multilayer Perceptrons Using Multilinear Parametrization**IEEE-INNS Int. Joint Conference on Neural Networks*, Washington DC, pp. 696-699, January 1990. - S. Shah and F. Palmieri, "
," Proceedings of*MEKA - A Fast Local Algorithm for Training Feedforward Neural Networks**Int. Joint Conference on Neural Networks*, San Diego, CA, pp. III 41-46, July 1990. - F. Palmieri, "
," Published review of the paper "*An Approach to Faster Backpropagation**Speeding Up Backpropagation by Gradient Correlation*," by D. V. Shreibman and E. M. Norris,*Neural Network Review*, Vol. 4, No. 1, 1990, p. 23. - F. Palmieri, S. Shah, "
," Proceedings of the*A New Algorithm for Multilayer Perceptrons**IEEE Int. Conference on Systems Man and Cybernetics*, Boston, MA, pp. 427-428, Nov. 1989.

**Recurrent Nets**

The behavior of a neural network with sigmoidal non linearities with feedbacks, becomes dynamic and may very complicated to analyze. We show how the forward and the lateral weights of the network can be computed to provide a controlled behavior.

- A. Budillon, M. Corrente and F. Palmieri, "
," Proceedings of the*How a Neural Network Can Discover Gaussian Clusters**International ICSC Workshop on Independence and Artificial Neural Networks*, I&ANN'98, University of Laguna, Tenerife, Spain, ISBN: 3-906454-13-4, pp. 59-63, Feb. 1998. - A. Budillon, M. Corrente and F. Palmieri, "
," Proceedings of*EM Algorithm: A Neural Network View**the 9th Italian Workshop on Neural Nets*, Springer Verlag Ed., Vietri s.m., SA, Italy, pp. 285-292, May 1997. - A. Budillon, M. Corrente and F. Palmieri, "
," Internal Technical Report, Dip. di Ing. Elettronica e delle Telecomunicazioni, Univ. di Napoli "Federico II", via Claudio 21, Napoli, Italy, 1998.*A Dynamic Neural Network for Approximating Gaussian Posterior Probabilities*

** **

**Neural Networks for Sound Localization **

Inspired by the capability of the barn owl to locate objects with great precision only from sound, we have analized the binaural sound localization problem using neural networks.

- M. Datum, F. Palmieri and A. Moiseff, ''
," in*An Artificial Neural Network for Sound Localization Using Binaural Cues**The Journal of the Acoustical Society of America*, Vol. 100, N. 1, pp. 372-383, July 1996. - F. Palmieri, A. Shah, A. Moiseff, "
," Proc. of*Neural Coding of Interaural Time Difference**IEEE Int. Joint Conference on Neural Networks*, Baltimore, MD, pp. IV 271-276, June 1992. - F. Palmieri, M. Datum, A. Shah and A. Moiseff, "
," Proceedings of*Sound Localization with a Neural Network Trained with the Multiple extended Kalman Algorithm**International Joint Conference on Neural Networks*, Seattle, pp. I 125-131, July 1991. - A. Moiseff, F. Palmieri, M. Datum and a. Shah, ''
," Proceedings of the*An Artificial Neural Network for Studying Binaural Sound Localization**IEEE 17th Annual Northeast Bioengineering Conference*, Hartford, CT, pp.1-2, April 1991. - F. Palmieri, A. Moiseff, M. Datum and A.Shah, ''
," Proceedings of the*Learning Binaural Sound Localization Though a Neural Network**IEEE 17th Annual Northeast Bioengineering Conference*, Hartford, CT, pp. 13-14, April 1991. - F. Palmieri, M. Datum, A. Shah, A. Moiseff, "
," Proceedings of the*Application of a Neural Network Trained with the Multiple Extended Kalman Algorithm**IEEE Third Biennal Acoustics*, Speech and Signal Processing Mini Conference, Weston, MA, pp. S16.1-2, April 1991.

**Hebbian Learning**

The postulate of Hebb's that says that sysnapses are strenghtened by how much they are able to stimulate their target neurons, has been applied to artificial neural networks in deriving Hebbian rules for learning the network paraments from examples.

- F. Palmieri, J. Zhu, "
," in*Self-Association and Hebbian Learning in Linear Neural Networks**IEEE Trans. on Neural Networks*, Vol. 6, N. 5, pp. 1165-1183, Sept. 1995. - F. Palmieri, "
," Proceeedings of*The Anti-Hebbian Synapse in a Nonlinear Neural Network**WIRN '95, VII Italian Workshop on Neural Nets*, Vietri s.m., SA, Italy, pp. 117-122, May 18-20, 1995. - F. Palmieri, "
," (invited paper) Proceedings of*Hebbian Learning and Self-Association in Nonlinear Neural Networks**IEEE World Congress on Computational Intelligence*, Orlando, Florida, June 26-July 2 1994. - F. Palmieri, "
," Proceedings of*The Analysis of Mixtures of Hebbian and Anti-Hebbian Synapses at a Neural Node**International Conference on Artificial Neural Networks*, Sorrento, Italy, pp. 1071-1074, May 26-29, 1994. - F. Palmieri, J. Zhu and C. Chang, "
," in*Anti-Hebbian Learning in Topologically Constrained Linear Neural Networks: a Tutorial**IEEE Trans. on Neural Networks*, Vol. 4, N. 5, 748-761, Sept. 1993. - F. Palmieri, J. Zhu, "
," Proceedings of*The Behavior of a Single Linear Self-Associative Neuron**World Congress on Neural Networks*, Portland, Oregon, July 1993. - F. Palmieri, J. Zhu, "
," Technical report 5-93, Department of Electrical and Systems Engineering, The University of Connecticut, Storrs, CT, May 1993.*Hebbian Learning in Linear Neural Networks: A Review* - F. Palmieri and J. Zhu, ''
," Proceedings of*A Comparison of Two Eigen-Networks**International Joint Conference on Neural Networks*, Seattle, WA, pp. II 193-199, July 1991. - F. Palmieri and J. Zhu, ''
," Proceedings of*Linear Neural Networks Which Minimize the Output Variance**International Joint Conference on Neural Networks*, Seattle, WA, pp. I 791-797, July 1991. - F. Palmieri and J. Zhu, ''
," Proceedings of the*Unsupervised Learning in Constrained Linear Networks**IEEE 17th Annual Northeast Bioengineering Conference*, Hartford, CT, pp. 9-10, April 1991. - F. Palmieri, J. Zhu, "
," Proceedings of the*Eigenstructure Decomposition in a Cascaded Linear Network**IEEE Third Biennal Acoustics, Speech and Signal Processing Mini Conference*, Weston, MA, pp. F9.1-2, April 1991. - F. Palmieri, J. Zhu and C. Chang, "
," Technical Report 92-3, Department of Electrical and Systems Engineering, The University of Connecticut, Storrs, CT 06269-3157, Nov. 1991.*Self-Organizing Linear Neural Networks with an Energy Criterion*

**Universal Networks**

We refer to a Universal Network as a neural network that can compute progressive embeddings of the input space that can be collected for classification or filtering. We have proposed this paradigm showing how the embeddings can self-organize on the input data and can be used for implementing arbitrarily complex functions.

- F. Palmieri, "
," Proceedings of*A Paradigm for Supervised Learning Without Backpropagation**First Conference on Applications of Artificial Intelligence Techniques in Engineering*, Naples, Italy, pp. 79-88, Oct. 5-7, 1994. - F. Palmieri, "
," Proceedings of*Linear Self-Association for Universal Memory and Approximation**World Congress on Neural Networks*, Portland, Oregon, July 1993. - F. Palmieri, "
," Proc. of*A Self-Organizing Neural Network for Multidimensional Approximation**IEEE International Joint Conference on Neural Networks*, 07-11 Jun 1992, Baltimore, MD 1992 (IJCNN 1992), Vol. 4, pp. 802 - 807, DOI: 10.1109/IJCNN.1992.227219, Print ISBN: 0-7803-0559-0. - F. Palmieri, "
," Proc. of*A Self-Organizing Neural Network for Nonlinear Filtering**IEEE International Symposium on Circuits and Systems*, San Diego, CA 10-13 May 1992 (ISCAS '92), Vol. 6, pp. 2629 - 2632, DOI: 10.1109/ISCAS.1992.230681, Print ISBN: 0-7803-0593-0. - F. Palmieri, "
," Tech. Report 92-13, Overheads from a talk delivered at the Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Oct. 22, 1992. Department of Electrical and Systems Engineering, The University of Connecticut, Storrs, CT, 1992.*Hebbian Learning for Universal Memory and Approximation*

**Spiking Neural Networks**

We show how to go from a spiking neuron to a sigmoid-based model.

- F. Palmieri, A. Luongo, A. Moiseff, "
," Proceedings of the*From Spiking Neurons to Dynamic Perceptrons**11th Italian Workshop on Neural Nets*, WIRN'99, Vietri s.m., SA, Italy, May 1999.

**Support Vector Machines (SVM)**

We have proposed various approaches to simplify the SVM classifiers.

- D. Mattera, F. Palmieri, S. Haykin, "
,'' in*Simple and Robust Methods for Support Vector Expansions**IEEE Trans. on Neural Networks*, vol. 10, n. 5, pp. 1038-1047, sept. 1999. - D. Mattera, F. Palmieri, S. Haykin, "
,'' in*An explicit algorithm for training support vector machines**IEEE Signal Processing Letters*, vol. 6, n. 9, pp. 243-245, sept. 1999. - D. Mattera, F. Palmieri, S. Haykin, "
," Proceedings of*Generalized Support Vector Machines**European Symposium on Artificial Neural Networks*, ESANN'99, Bruge, Belgium, April 1999. - D. Mattera, F. Palmieri, S. Haykin, "
," Proceedings of the*Training Semiparametric Support Vector Machines**11th Italian Workshop on Neural Nets*, WIRN'99, Vietri s.m., SA, Italy, May 1999. - D. Mattera, F. Palmieri, S. Haykin, "
,'' in Signal Processing IX - Theory and applications - Proceedings of*Adaptive nonlinear filtering with the support vector method**Eusipco-98 - Ninth Signal Processing Conference*, Rhodes, Greece, 8-11 September 1998, S. Theodoridis, I. Pitas, A. Stouraitis, N. Kalouptsidis (eds.), Typorama Editions, Patras, Greece, pp.773-776, 1998. - D. Mattera, F. Palmieri, "
," Proceedings of the*Support Vector Machine for Nonparametric Binary Hypothesis Testing**10th Italian Workshop on Neural Nets*, Springer Verlag, Vietri s.m., SA, Italy, pp. 132-137, May 1998.

**Learning Theory**

VC theory predicts the capability of a classifier to genaralize. We have derived bounds and analyses for this paradigm.

- D. Mattera, F. Palmieri, "
,'' Internal Technical Report, Dip. di Ing. Elettronica e delle Telecomunicazioni, Univ. di Napoli ``Federico II", via Claudio 21, 80125 Napoli (Italy), 1998.*New Generalization Bounds with a Non-Null Training Error* - D. Mattera and F. Palmieri, "
,'' Internal Technical Report, Dip. di Ing. Elettronica e delle Telecomunicazioni, Univ. di Napoli ``Federico II", via Claudio 21, 80125 Napoli (Italy), 1998.*Improvement of a Bound in Learning Binary Functions* - D. Mattera, F. Palmieri, "
," Proceedings of*New Bounds for Correct Generalization**IEEE International Conference on Neural Networks*, Houston, TX, pp. 1051-1055, June 1997. - D. Mattera, F. Palmieri, "
," Proceedings of the*A Distribution-Free VC-Dimension-Based Performance Bound**9th Italian Workshop on Neural Nets*, Springer Verlag Ed., Vietri s.m., SA, Italy, pp. 162-168, May 1997. - F. Palmieri, D. Mattera, "
," Proc. of*The Computational Neural Map and its Capacity**WIRN '96, VIII Italian Worshop on Neural Nets*, Vietri s.m., SA, Italy, pp. 137-142, May 23-25, 1996. - F. Palmieri, D. Mattera, "
," Proceedings of*An Approach to Network Capacity in Continuous-Valued Neural Networks**World Congress on Neural Networks*, San Diego, CA, pp. 1003-1006, Sept. 15-18, 1996.