ACN6349/CGS4315 Syllabus 1/7/06

ACN6349/CGS4315: Intelligent Systems Design

Who should be interested in this course?:
This applied mathematics course will be of great interest to students interested in building
mathematical models in the fields of psychology, biology, engineering, social sciences, and
economics. It is often claimed that high-dimensional nonlinear dynamical systems are so
complex that it is impossible to understand them without simulating them on a computer. We
will show how one can use advanced mathematics to characterize the behavior of many
complex simulation models and guide computer simulation research. This course will also be
of great interest to engineers and computer scientists who want to learn how to
mathematically analyze and design artificial intelligence systems based upon artificial neural
network technology. The topics covered in this two course sequence are essential for the
development of the necessary engineering skills for real-world applications of artificial neural
network technology as well as scientists interested in developing formally sound
mathematical models.

Course Objectives:
Upon successful completion of this course, students should be able to:
Ø Compute Gradients and Hessians of objective functions typically encountered in
artificial neural net research (e.g., gradient and Hessian for multi-layer recurrent
artificial neural nets).
Ø Apply Markov Chain Monte Carlo (MCMC) methods to neural net analysis and design.
Ø Formulate ANNs within classical statistical pattern recognition framework. Construct
statistical tests for model evaluation and selection.
Ø Students will obtain expertise in the formal mathematical analysis of artificial neural
networks, vector calculus with Artificial Neural Network applications and vector
stochastic processes. Students will obtain expertise in making rigorous formal
statements about complex systems.

Course Time and Location: Tuesday and Thursday, 5:30pm-6:45pm, GR4.204

Course Instructor:
Dr. Richard M. Golden (www.utdallas.edu/~golden) (972-883-2423) (golden@utdallas.edu).

Course Book:
Mathematical Methods for Neural Network Analysis and Design by Richard M. Golden (MIT
PRESS), 1996 (ISBN=0-262-07174-6). [We will do Chapters 5-8]. See
www.utdallas.edu/~golden/book_abs.html for more information about this book! Student
edition of MATLAB is not required but highly recommended.

Background Reading:
The following books will be helpful for background reading in this course:
Ø Anderson, J. A. An Introduction to Neural Networks. MIT Press.
Ø Hsu, H. Probability, Random Variables, and Stochastic Processes (Schaum’s Outline)
Ø Marlow. Mathematics for Operations Research (Dover Book)
Ø Rosenlicht, M. Introduction to Analysis (Dover Book)

1
ACN6349/CGS4315 Syllabus 1/7/06

Prerequisites:
Linear algebra, multivariable calculus, and a calculus-based probability theory course (STAT
4351/5351) or equivalent coursework are required. See Dr. Golden if you have questions
regarding these prerequisites and alternative methods for satisfying them. Previous
experience involving computer simulation work with artificial neural networks (e.g., a course
in artificial neural network work) is highly desirable but not required. Experience with some
computer programming language is required (MATLAB programming experience is
preferred). ACN6347 (Intelligent Systems Analysis) is highly recommended but not required.
Students who have not taken ACN6347 will be expected to pick up relevant background
material on an as-needed basis.

Homework: Homework will be assigned regularly throughout the course. The homework
problems are intended to be representative of the exam questions. Answers to selected
homework problems are provided in the back of the course textbook. Homework will not be
graded but will be representative of exam questions.

Grading:
ACN6349: In Class Exam 1 (20%), Take Home Exam (20%), In Class Exam 2 (20%), Take
Home Final Exam (40%)
CGS4315: The “worst grade” of the above 4 exams will be dropped and the remaining 3
grades will be appropriately averaged.

Tentative Schedule:
Weeks 1-4: Local and global knowledge representations. Local and global processing
algorithms. Markov Chains and Markov Fields (Excerpts from Luenberger, 1979; Ch. 5,6,
Golden, Handouts, plus supplemental material). In Class Exam 1.

Weeks 5-7: Simulated Annealing and MCMC Sampling for Optimization and Numerical
Integration. Hopfield (1982), Boltzmann Machine (1986), ICM, Gibbs Sampler, Metropolis
Algorithm, Metropolis-Hasting Algorithm. (Ch. 5,6, Golden plus supplemental material).

Take Home Exam (Covers Weeks 1-6).

Weeks 8-10: Expected Risk Classification and Learning Theory. (Ch. 7, Golden).

Weeks 10-12: Confidence Intervals, Hypothesis Testing, and Model Selection (Ch. 8,
Golden). In Class Exam 2.

Weeks 13-15: Advanced Topics in Model Selection (Golden, 2003, Psychometrika). Making
correct statistical inferences in the presence of missing data and model misspecification (to
be submitted). Information Matrix Tests for Goodness-of-Fit Assessment (to be submitted).

Final Exam
Web Site: www.utdallas.edu/~golden/MATHANN
Login: golden