Preface	9
	Acknowledgements	13
	Contents	15
	PART I. INTRODUCTION	23
	1. Adaptive Filtering	25
	1.1 Linear Adaptive Filters	27
	1.1.1 Linear Adaptive Filter Algorithms	29
	1.2 Nonlinear Adaptive Filters	31
	1.2.1 Adaptive Volterra Filters	31
	1.3 Nonclassical Adaptive Systems	32
	1.3.1 Artificial Neural Networks	32
	1.3.2 Fuzzy Logic	33
	1.3.3 Genetic Algorithms	33
	1.4 A Brief History and Overview of Classical Theories	34
	1.4.1 Linear Estimation Theory	34
	1.4.2 Linear Adaptive Filters	35
	1.4.3 Adaptive Signal Processing Applications	36
	1.4.4 Adaptive Control	38
	1.5 A Brief History and Overview of Nonclassical Theories	39
	1.5.1 Artificial Neural Networks	39
	1.5.2 Fuzzy Logic	40
	1.5.3 Genetic Algorithms	40
	1.6 Fundamentals of Adaptive Networks	41
	1.7 Choice of Adaptive Filter Algorithm	45
	2. Linear Systems and Stochastic Processes	47
	2.1 Basic Concepts of Linear Systems	49
	2.2 Discrete-time Signals and Systems	51
	2.3 The Discrete Fourier Transform (DFT)	53
	2.3.1 Discrete Linear Convolution Using the DFT	54
	2.3.2 Digital Sampling Theory	55
	2.4 The Fast Fourier Transform (FFT)	59
	2.5 The z-Transform	62
	2.5.1 Relationship between Laplace Transform and z-Transform	62
	2.5.2 General Properties of the DFT and z-Transform	66
	2.6 Summary of Discrete-time LSI Systems	68
	2.7.1 Phase Response from Frequency Magnitude Response	72
	2.8 Linear Algebra Summary	73
	2.8.1 Vectors	73
	2.8.2 Linear Independence, Vector spaces, and Basis Vectors	74
	2.8.3 Matrices	75
	2.8.4 Linear Equations	77
	2.8.5 Special Matrices	78
	2.8.6 Quadratic and Hermitian Forms	81
	2.8.7 Eigenvalues and Eigenvectors	81
	2.9 Introduction to Stochastic Processes	83
	2.10 Random Signals	85
	2.11 Basic Descriptive Properties of Random Signals	86
	2.11.1 The Mean Square Value and Variance	86
	2.11.2 The Probability Density Function	87
	2.11.3 Jointly Distributed Random Variables	90
	2.11.4 The Expectation Operator	90
	2.11.5 The Autocorrelation and Related Functions	91
	2.11.6 Power Spectral Density Functions	94
	2.11.7 Coherence Function	95
	2.11.8 Discrete Ergodic Random Signal Statistics	96
	2.11.9 Autocovariance and Autocorrelation Matrices	97
	2.11.10 Spectrum of a Random Process	98
	2.11.11 Filtering of Random Processes	100
	2.11.12 Important Examples of Random Processes	102
	2.12 Exercises	104
	2.12.1 Problems	104
	PART II. MODELLING	109
	3. Optimisation and Least Squares Estimation	111
	3.1 Optimisation Theory	111
	3.2 Optimisation Methods in Digital Filter Design	113
	3.3 Least Squares Estimation	117
	3.4 Least Squares Maximum Likelihood Estimator	119
	3.5 Linear Regression - Fitting Data to a Line	120
	3.6 General Linear Least Squares	121
	3.7 A Ship Positioning Example of LSE	122
	3.9 Measure of LSE Precision	130
	3.10 Measure of LSE Reliability	131
	3.11 Limitations of LSE	132
	3.12 Advantages of LSE	132
	3.13 The Singular Value Decomposition	133
	3.13.1 The Pseudoinverse	134
	3.13.2 Computation of the SVD	134
	3.14 Exercises	138
	3.14.1 Problems	138
	4. Parametric Signal and System Modelling	141
	4.1 The Estimation Problem	142
	4.2 Deterministic Signal and SystemModelling	143
	4.2.1 The Least Squares Method	144
	4.2.2 The Padé Approximation Method	146
	4.2.3 Prony’s Method	149
	4.2.4 Autocorrelation and Covariance Methods	155
	4.3 Stochastic Signal Modelling	159
	4.3.1 Autoregressive Moving Average Models	159
	4.3.2 Autoregressive Models	161
	4.3.3 Moving Average Models	162
	4.4 The Levinson-Durbin Recursion and Lattice Filters	163
	4.4.1 The Levinson-Durbin Recursion Development	164
	4.4.2 The Lattice Filter	168
	4.4.3 The Cholesky Decomposition	171
	4.4.4 The Levinson Recursion	173
	4.5 Exercises	176
	4.5.1 Problems	176
	PART III. CLASSICAL FILTERS and SPECTRALANALYSIS	179
	5. OptimumWiener Filter	181
	5.1 Derivation of the Ideal Continuous-time Wiener Filter	182
	5.2 The Ideal Discrete-time FIR Wiener Filter	184
	5.2.1 General Noise FIR Wiener Filtering	186
	5.2.2 FIRWiener Linear Prediction	187
	5.3 Discrete-time Causal IIR Wiener Filter	189
	5.3.1 Causal IIR Wiener Filtering	191
	5.3.2 Wiener Deconvolution	192
	5.4 Exercises	193
	5.4.1 Problems	193
	6. Optimum Kalman Filter	195
	6.1 Background to The Kalman Filter	195
	6.2 The Kalman Filter	196
	6.2.1 Kalman Filter Examples	203
	6.3 Kalman Filter for Ship Motion	207
	6.3.1 Kalman Tracking Filter Proper	208
	6.3.2 Simple Example of a Dynamic Ship Model	211
	6.3.3 Stochastic Models	214
	6.3.4 Alternate Solution Methods	214
	6.3.5 Advantages of Kalman Filtering	215
	6.3.6 Disadvantage of Kalman Filtering	215
	6.4 Extended Kalman Filter	216
	6.5 Exercises	216
	6.5.1 Problems	216
	7. Power Spectral Density Analysis	219
	7.1 Power Spectral Density Estimation Techniques	220
	7.2 Nonparametric Spectral Density Estimation	221
	7.2.1 Periodogram Power Spectral Density Estimation	221
	7.2.2 Modified Periodogram - Data Windowing	225
	7.2.3 Bartlett’s Method - Periodogram Averaging	227
	7.2.4 Welch’s Method	228
	7.2.5 Blackman-Tukey Method	230
	7.2.6 Performance Comparisons of Nonparametric Methods	231
	7.2.7 Minimum Variance Method	231
	7.2.8 Maximum Entropy (All Poles) Method	234
	7.3 Parametric Spectral Density Estimation	237
	7.3.1 Autoregressive Methods	237
	7.3.2 Moving Average Method	240
	7.3.3 Autoregressive Moving Average Method	241
	7.3.4 Harmonic Methods	241
	7.4 Exercises	245
	7.4.1 Problems	245
	PART IV. ADAPTIVE FILTER THEORY	247
	8. Adaptive Finite Impulse Response Filters	249
	8.1 Adaptive Interference Cancelling	250
	8.2 Least Mean Squares Adaptation	252
	8.2.1 Optimum Wiener Solution	253
	8.2.2 The Method of Steepest Gradient Descent Solution	255
	8.2.3 The LMS Algorithm Solution	257
	8.2.4 Stability of the LMS Algorithm	259
	8.2.5 The Normalised LMS Algorithm	261
	8.3 Recursive Least Squares Estimation	261
	8.3.1 The Exponentially Weighted Recursive Least Squares Algorithm	262
	8.3.2 Recursive Least Squares Algorithm Convergence	265
	8.3.3 The RLS Algorithm as a Kalman Filter	266
	8.4 Exercises	267
	8.4.1 Problems	267
	9. Frequency Domain Adaptive Filters	269
	9.1 Frequency Domain Processing	269
	9.1.1 Time Domain Block Adaptive Filtering	270
	9.1.2 Frequency Domain Adaptive Filtering	271
	9.2 Exercises	278
	9.2.1 Problems	278
	10. Adaptive Volterra Filters	279
	10.1 Nonlinear Filters	279
	10.2 The Volterra Series Expansion	281
	10.3 A LMS Adaptive Second-order Volterra Filter	281
	10.4 A LMS Adaptive Quadratic Filter	283
	10.5 A RLS Adaptive Quadratic Filter	284
	10.6 Exercises	286
	10.6.1 Problems	286
	11. Adaptive Control Systems	289
	11.1 Main Theoretical Issues	290
	11.2 Introduction to Model-reference Adaptive Systems	292
	11.2.1 The Gradient Approach	293
	11.2.2 Least Squares Estimation	295
	11.2.3 A General Single-input-single-output MRAS	296
	11.2.4 Lyapunov’s Stability Theory	299
	11.3 Introduction to Self-tuning Regulators	302
	11.3.1 Indirect Self-tuning Regulators	304
	11.3.2 Direct Self-tuning Regulators	305
	11.4 Relations between MRAS and STR	306
	11.5 Applications	307
	PART V. NONCLASSICALADAPTIVE SYSTEMS	309
	12. Introduction to Neural Networks	311
	12.1 Artificial Neural Networks	311
	12.1.1 Definitions	312
	12.1.2 Three Main Types	312
	12.1.3 Specific Artificial Neural Network Paradigms	314
	12.1.4 Artificial Neural Networks as Black Boxes	315
	12.1.5 Implementation of Artificial Neural Networks	316
	12.1.6 When to Use an Artificial Neural Network	317
	12.1.7 How to Use an Artificial Neural Network	317
	12.1.8 Artificial Neural Network General Applications	318
	12.1.9 Simple Application Examples	319
	12.2 A Three-layer Multi-layer Perceptron Model	322
	12.2.1 MLP Backpropagation-of-error Learning	324
	12.2.2 Derivation of Backpropagation-of-error Learning	325
	12.2.3 Notes on Classification and Function Mapping	330
	12.2.4 MLP Application and Training Issues	330
	12.3 Exercises	332
	12.3.1 Problems	332
	13. Introduction to Fuzzy Logic Systems	335
	13.1 Basic Fuzzy Logic	335
	13.1.1 Fuzzy Logic Membership Functions	336
	13.1.2 Fuzzy Logic Operations	337
	13.1.3 Fuzzy Logic Rules	338
	13.1.4 Fuzzy Logic Defuzzification	339
	13.2 Fuzzy Logic Control Design	340
	13.2.1 Fuzzy Logic Controllers	341
	13.3 Fuzzy Artificial Neural Networks	344
	13.4 Fuzzy Applications	345
	14. Introduction to Genetic Algorithms	347
	14.1 A General Genetic Algorithm	348
	14.2 The Common Hypothesis Representation	349
	14.3 Genetic Algorithm Operators	351
	14.4 Fitness Functions	352
	14.5 Hypothesis Searching	352
	14.6 Genetic Programming	353
	14.7 Applications of Genetic Programming	354
	14.7.1 Filter Circuit Design Application of GAs and GP	355
	14.7.2 Tic-tac-to Game Playing Application of GAs	356
	PART VI. ADAPTIVE FILTER APPLICATION	359
	15. Applications of Adaptive Signal Processing	361
	15.1 Adaptive Prediction	362
	15.2 Adaptive Modelling	364
	15.3 Adaptive Telephone Echo Cancelling	365
	15.4 Adaptive Equalisation of Communication Channels	366
	15.5 Adaptive Self-tuning Filters	368
	15.6 Adaptive Noise Cancelling	368
	15.7 Focused Time Delay Estimation for Ranging	370
	15.7.1 Adaptive Array Processing	371
	15.8 Other Adaptive Filter Applications	372
	15.8.1 Adaptive 3-D Sound Systems	372
	15.8.2 Microphone arrays	373
	15.8.3 Network and Acoustic Echo Cancellation	374
	15.8.4 Real-world Adaptive Filtering Applications	375
	16. Generic Adaptive Filter Structures	377
	16.1 Sub-band Adaptive Filters	377
	16.2 Sub-space Adaptive Filters	380
	16.2.1 MPNN Model	382
	16.2.2 Approximately Piecewise Linear Regression Model	384
	16.2.3 The Sub-space Adaptive Filter Model	386
	16.2.4 Example Applications of the SSAF Model	388
	16.3 Discussion and Overview of the SSAF	392
	References	395
	Index	403
	More eBook at www.ciando.com	0