1 Functional Analysis 1

1.1 Concept of Function 1

1.2 Continuity and Limits 3

1.3 Partial Differentiation 6

1.4 Total Differential 8

1.5 Taylor Series 9

1.6 Maxima and Minima of Functions 13

1.7 Extrema of Functions with Conditions 17

1.8 Derivatives and Differentials of Composite Functions 21

1.9 Implicit Function Theorem 23

1.10 Inverse Functions 28

1.11 Integral Calculus and the Definite Integral 30

1.12 Riemann Integral 32

1.13 Improper Integrals 35

1.14 Cauchy Principal Value Integrals 38

1.15 Integrals Involving a Parameter 40

1.16 Limits of Integration Depending on a Parameter 44

1.17 Double Integrals 45

1.18 Properties of Double Integrals 47

1.19 Triple and Multiple Integrals 48

References 49

Problems 49

2 Vector Analysis 55

2.1 Vector Algebra: Geometric Method 55

2.1.1 Multiplication of Vectors 57

2.2 Vector Algebra: Coordinate Representation 60

2.3 Lines and Planes 65

2.4 Vector Differential Calculus 67

2.4.1 Scalar Fields and Vector Fields 67

2.4.2 Vector Differentiation 69

2.5 Gradient Operator 70

2.5.1 Meaning of the Gradient 71

2.5.2 Directional Derivative 72

2.6 Divergence and Curl Operators 73

2.6.1 Meaning of Divergence and the Divergence Theorem 75

2.7 Vector Integral Calculus in Two Dimensions 79

2.7.1 Arc Length and Line Integrals 79

2.7.2 Surface Area and Surface Integrals 83

2.7.3 An Alternate Way to Write Line Integrals 84

2.7.4 Greens Theorem 86

2.7.5 Interpretations of Greens Theorem 88

2.7.6 Extension to Multiply Connected Domains 89

2.8 Curl Operator and Stokess Theorem 92

2.8.1 On the Plane 92

2.8.2 In Space 96

2.8.3 Geometric Interpretation of Curl 99

2.9 Mixed Operations with the Del Operator 99

2.10 Potential Theory 102

2.10.1 Gravitational Field of a Star 105

2.10.2 Work Done by Gravitational Force 106

2.10.3 Path Independence and Exact Differentials 108

2.10.4 Gravity and Conservative Forces 109

2.10.5 Gravitational Potential 111

2.10.6 Gravitational Potential Energy of a System 113

2.10.7 Helmholtz Theorem 115

2.10.8 Applications of the Helmholtz Theorem 116

2.10.9 Examples from Physics 120

References 123

Problems 123

3 Generalized Coordinates and Tensors 133

3.1 Transformations between Cartesian Coordinates 134

3.1.1 Basis Vectors and Direction Cosines 134

3.1.2 Transformation Matrix and Orthogonality 136

3.1.3 Inverse Transformation Matrix 137

3.2 Cartesian Tensors 139

3.2.1 Algebraic Properties of Tensors 141

3.2.2 Kronecker Delta and the Permutation Symbol 145

3.3 Generalized Coordinates 148

3.3.1 Coordinate Curves and Surfaces 148

3.3.2 Why Upper and Lower Indices 152

3.4 General Tensors 153

3.4.1 Einstein Summation Convention 156

3.4.2 Line Element 157

3.4.3 Metric Tensor 157

3.4.4 How to Raise and Lower Indices 158

3.4.5 Metric Tensor and the Basis Vectors 160

3.4.6 Displacement Vector 161

3.4.7 Line Integrals 162

3.4.8 Area Element in Generalized Coordinates 164

3.4.9 Area of a Surface 165

3.4.10 Volume Element in Generalized Coordinates 169

3.4.11 Invariance and Covariance 171

3.5 Differential Operators in Generalized Coordinates 171

3.5.1 Gradient 171

3.5.2 Divergence 172

3.5.3 Curl 174

3.5.4 Laplacian 178

3.6 Orthogonal Generalized Coordinates 178

3.6.1 Cylindrical Coordinates 179

3.6.2 Spherical Coordinates 184

References 189

Problems 189

4 Determinants and Matrices 197

4.1 Basic Definitions 197

4.2 Operations with Matrices 198

4.3 Submatrix and Partitioned Matrices 204

4.4 Systems of Linear Equations 207

4.5 Gausss Method of Elimination 208

4.6 Determinants 211

4.7 Properties of Determinants 214

4.8 Cramers Rule 216

4.9 Inverse of a Matrix 221

4.10 Homogeneous Linear Equations 224

References 225

Problems 225

5 Linear Algebra 233

5.1 Fields and Vector Spaces 233

5.2 Linear Combinations, Generators, and Bases 236

5.3 Components 238

5.4 Linear Transformations 241

5.5 Matrix Representation of Transformations 242

5.6 Algebra of Transformations 244

5.7 Change of Basis 246

5.8 Invariants under Similarity Transformations 247

5.9 Eigenvalues and Eigenvectors 248

5.10 Moment of Inertia Tensor 257

5.11 Inner Product Spaces 262

5.12 The Inner Product 262

5.13 Orthogonality and Completeness 265

5.14 Gram-Schmidt Orthogonalization 267

5.15 Eigenvalue Problem for Real Symmetric Matrices 268

5.16 Presence of Degenerate Eigenvalues 270

5.17 Quadratic Forms 276

5.18 Hermitian Matrices 279

5.19 Matrix Representation of Hermitian Operators 283

5.20 Functions of Matrices 284

5.21 Function Space and Hilbert Space 286

5.22 Diracs Bra and Ket Vectors 287

References 288

Problems 289

6 Practical Linear Algebra 293

6.1 Systems of Linear Equations 294

6.1.1 Matrices and Elementary Row Operations 295

6.1.2 Gauss-Jordan Method 295

6.1.3 Information From the Row-Echelon Form 300

6.1.4 Elementary Matrices 301

6.1.5 Inverse by Gauss-Jordan Row-Reduction 302

6.1.6 Row Space, Column Space, and Null Space 303

6.1.7 Bases for Row, Column, and Null Spaces 307

6.1.8 Vector Spaces Spanned by a Set of Vectors 310

6.1.9 Rank and Nullity 312

6.1.10 Linear Transformations 315

6.2 Numerical Methods of Linear Algebra 317

6.2.1 Gauss-Jordan Row-Reduction and Partial Pivoting 317

6.2.2 LU-Factorization 321

6.2.3 Solutions of Linear Systems by Iteration 325

6.2.4 Interpolation 328

6.2.5 Power Method for Eigenvalues 331

6.2.6 Solution of Equations 333

6.2.7 Numerical Integration 343

References 349

Problems 350

7 Applications of Linear Algebra 355

7.1 Chemistry and Chemical Engineering 355

7.1.1 Independent Reactions and Stoichiometric Matrix 356

7.1.2 Independent Reactions from a Set of Species 359

7.2 Linear Programming 362

7.2.1 The Geometric Method 363

7.2.2 The Simplex Method 367

7.3 Leontief Input-Output Model of Economy 375

7.3.1 Leontief Closed Model 375

7.3.2 Leontief Open Model 378

7.4 Applications to Geometry 381

7.4.1 Orbit Calculations 382

7.5 Elimination Theory 383

7.5.1 Quadratic Equations and the Resultant 384

7.6 Coding Theory 388

7.6.1 Fields and Vector Spaces 388

7.6.2 Hamming (7,4) Code 390

7.6.3 Hamming Algorithm for Error Correction 393

7.7 Cryptography 396

7.7.1 Single-Key Cryptography 396

7.8 Graph Theory 399

7.8.1 Basic Definition 399

7.8.2 Terminology 400

7.8.3 Walks, Trails, Paths and Circuits 402

7.8.4 Trees and Fundamental Circuits 404

7.8.5 Graph Operations 404

7.8.6 Cut Sets and Fundamental Cut Sets 405

7.8.7 Vector Space Associated with a Graph 407

7.8.8 Rank and Nullity 409

7.8.9 Subspaces in WG 410

7.8.10 Dot Product and Orthogonal vectors 411

7.8.11 Matrix Representation of Graphs 413

7.8.12 Dominance Directed Graphs 417

7.8.13 Gray Codes in Coding Theory 419

References 419

Problems 420

8 Sequences and Series 425

8.1 Sequences 426

8.2 Infinite Series 430

8.3 Absolute and Conditional Convergence 431

8.3.1 Comparison Test 431

8.3.2 Limit Comparison Test 431

8.3.3 Integral Test 431

8.3.4 Ratio Test 432

8.3.5 Root Test 432

8.4 Operations with Series 436

8.5 Sequences and Series of Functions 438

8.6 M-Test for Uniform Convergence 441

8.7 Properties of Uniformly Convergent Series 441

8.8 Power Series 443

8.9 Taylor Series and Maclaurin Series 446

8.10 Indeterminate Forms and Series 447

References 448

Problems 448

9 Complex Numbers and Functions 453

9.1 The Algebra of Complex Numbers 454

9.2 Roots of a Complex Number 458

9.3 Infinity and the Extended Complex Plane 460

9.4 Complex Functions 463

9.5 Limits and Continuity 465

9.6 Differentiation in the Complex Plane 467

9.7 Analytic Functions 470

9.8 Harmonic Functions 471

9.9 Basic Differentiation Formulas 474

9.10 Elementary Functions 475

9.10.1 Polynomials 475

9.10.2 Exponential Function 476

9.10.3 Trigonometric Functions 477

9.10.4 Hyperbolic Functions 478

9.10.5 Logarithmic Function 479

9.10.6 Powers of Complex Numbers 481

9.10.7 Inverse Trigonometric Functions 483

References 483

Problems 484

10 Complex Analysis 491

10.1 Contour Integrals 492

10.2 Types of Contours 494

10.3 The Cauchy-Goursat Theorem 497

10.4 Indefinite Integrals 500

10.5 Simply and Multiply Connected Domains 502

10.6 The Cauchy Integral Formula 503

10.7 Derivatives of Analytic Functions 505

10.8 Complex Power Series 506

10.8.1 Taylor Series with the Remainder 506

10.8.2 Laurent Series with the Remainder 510

10.9 Convergence of Power Series 514

10.10 Classification of Singular Points 514

10.11 Residue Theorem 517

References 522

Problems 522

11 Ordinary Differential Equations 527

11.1 Basic Definitions for Ordinary Differential Equations 528

11.2 First-Order Differential Equations 530

11.2.1 Uniqueness of Solution 530

11.2.2 Methods of Solution 532

11.2.3 Dependent Variable is Missing 532

11.2.4 Independent Variable is Missing 532

11.2.5 The Case of Separable f(x, y) 532

11.2.6 Homogeneous f(x, y) of Zeroth Degree 533

11.2.7 Solution When f(x, y) is a Rational Function 533

11.2.8 Linear Equations of First-order 535

11.2.9 Exact Equations 537

11.2.10 Integrating Factors 539

11.2.11 Bernoulli Equation 542

11.2.12 Riccati Equation 543

11.2.13 Equations that Cannot Be Solved for y' 546

11.3 Second-Order Differential Equations 548

11.3.1 The General Case 549

11.3.2 Linear Homogeneous Equations with Constant Coefficients 551

11.3.3 Operator Approach 556

11.3.4 Linear Homogeneous Equations with Variable Coefficients 557

11.3.5 Cauchy-Euler Equation 560

11.3.6 Exact Equations and Integrating Factors 561

11.3.7 Linear Nonhomogeneous Equations 564

11.3.8 Variation of Parameters 564

11.3.9 Method of Undetermined Coefficients 566

11.4 Linear Differential Equations of Higher Order 569

11.4.1 With Constant Coefficients 569

11.4.2 With Variable Coefficients 570

11.4.3 Nonhomogeneous Equations 570

11.5 Initial Value Problem and Uniqueness of the Solution 571

11.6 Series Solutions: Frobenius Method 571

11.6.1 Frobenius Method and First-order Equations 581

References 582

Problems 582

12 Second-Order Differential Equations and Special Functions 589

12.1 Legendre Equation 590

12.1.1 Series Solution 590

12.1.2 Effect of Boundary Conditions 593

12.1.3 Legendre Polynomials 594

12.1.4 Rodriguez Formula 596

12.1.5 Generating Function 597

12.1.6 Special Values 599

12.1.7 Recursion Relations 600

12.1.8 Orthogonality 601

12.1.9 Legendre Series 603

12.2 Hermite Equation 606

12.2.1 Series Solution 606

12.2.2 Hermite Polynomials 610

12.2.3 Contour Integral Representation 611

12.2.4 Rodriguez Formula 612

12.2.5 Generating Function 613

12.2.6 Special Values 614

12.2.7 Recursion Relations 614

12.2.8 Orthogonality 616

12.2.9 Series Expansions in Hermite Polynomials 618

12.3 Laguerre Equation 619

12.3.1 Series Solution 620

12.3.2 Laguerre Polynomials 621

12.3.3 Contour Integral Representation 622

12.3.4 Rodriguez Formula 623

12.3.5 Generating Function 623

12.3.6 Special Values and Recursion Relations 624

12.3.7 Orthogonality 624

12.3.8 Series Expansions in Laguerre Polynomials 625

References 626

Problems 626

13 Bessels Equation and Bessel Functions 629

13.1 Bessels Equation and Its Series Solution 630

13.1.1 Bessel Functions J±m(x), Nm(x), and H(1,2)m (x) 634

13.1.2 Recursion Relations 639

13.1.3 Generating Function 639

13.1.4 Integral Definitions 641

13.1.5 Linear Independence of Bessel Functions 642

13.1.6 Modified Bessel Functions Im(x) and Km(x) 644

13.1.7 Spherical Bessel Functions jl(x), nl(x), and h(1,2)l (x) 645

13.2 Orthogonality and the Roots of Bessel Functions 648

13.2.1 Expansion Theorem 652

13.2.2 Boundary Conditions for the Bessel Functions 652

References 656

Problems 656

14 Partial Differential Equations and Separation of Variables 661

14.1 Separation of Variables in Cartesian Coordinates 662

14.1.1 Wave Equation 665

14.1.2 Laplace Equation 666

14.1.3 Diffusion and Heat Flow Equations 671

14.2 Separation of Variables in Spherical Coordinates 673

14.2.1 Laplace Equation 677

14.2.2 Boundary Conditions for a Spherical Boundary 678

14.2.3 Helmholtz Equation 682

14.2.4 Wave Equation 683

14.2.5 Diffusion and Heat Flow Equations 684

14.2.6 Time-Independent Schrödinger Equation 685

14.2.7 Time-Dependent Schrödinger Equation 685

14.3 Separation of Variables in Cylindrical Coordinates 686

14.3.1 Laplace Equation 688

14.3.2 Helmholtz Equation 689

14.3.3 Wave Equation 690

14.3.4 Diffusion and Heat Flow Equations 691

References 701

Problems 701

15 Fourier Series 705

15.1 Orthogonal Systems of Functions 705

15.2 Fourier Series 711

15.3 Exponential Form of the Fourier Series 712

15.4 Convergence of Fourier Series 713

15.5 Sufficient Conditions for Convergence 715

15.6 The Fundamental Theorem 716

15.7 Uniqueness of Fourier Series 717

15.8 Examples of Fourier Series 717

15.8.1 Square Wave 717

15.8.2 Triangular Wave 719

15.8.3 Periodic Extension 720

15.9 Fourier Sine and Cosine Series 721

15.10 Change of Interval 722

15.11 Integration and Differentiation of Fourier Series 723

References 724

Problems 724

16 Fourier and Laplace Transforms 727

16.1 Types of Signals 727

16.2 Spectral Analysis and Fourier Transforms 730

16.3 Correlation with Cosines and Sines 731

16.4 Correlation Functions and Fourier Transforms 735

16.5 Inverse Fourier Transform 736

16.6 Frequency Spectrums 736

16.7 Dirac-Delta Function 738

16.8 A Case with Two Cosines 739

16.9 General Fourier Transforms and Their Properties 740

16.10 Basic Definition of Laplace Transform 743

16.11 Differential Equations and Laplace Transforms 746

16.12 Transfer Functions and Signal Processors 748

16.13 Connection of Signal Processors 750

References 753

Problems 753

17 Calculus of Variations 757

17.1 A Simple Case 758

17.2 Variational Analysis 759

17.2.1 Case I: The Desired Function is Prescribed at the End Points 761

17.2.2 Case II: Natural Boundary Conditions 762

17.3 Alternate Form of Euler Equation 763

17.4 Variational Notation 765

17.5 A More General Case 767

17.6 Hamiltons Principle 772

17.7 Lagranges Equations of Motion 773

17.8 Definition of Lagrangian 777

17.9 Presence of Constraints in Dynamical Systems 779

17.10 Conservation Laws 783

References 784

Problems 784

18 Probability Theory and Distributions 789

18.1 Introduction to Probability Theory 790

18.1.1 Fundamental Concepts 790

18.1.2 Basic Axioms of Probability 791

18.1.3 Basic Theorems of Probability 791

18.1.4 Statistical Definition of Probability 794

18.1.5 Conditional Probability and Multiplication Theorem 795

18.1.6 Bayes Theorem 796

18.1.7 Geometric Probability and Buffons Needle Problem 798

18.2 Permutations and Combinations 800

18.2.1 The Case of Distinguishable Balls with Replacement 800

18.2.2 The Case of Distinguishable Balls without Replacement 801

18.2.3 The Case of Indistinguishable Balls 802

18.2.4 Binomial and Multinomial Coefficients 803

18.3 Applications to Statistical Mechanics 804

18.3.1 Boltzmann Distribution for Solids 805

18.3.2 Boltzmann Distribution for Gases 807

18.3.3 Bose-Einstein Distribution for Perfect Gases 808

18.3.4 Fermi-Dirac Distribution 810

18.4 Statistical Mechanics and Thermodynamics 811

18.4.1 Probability and Entropy 811

18.4.2 Derivation of ß 812

18.5 Random Variables and Distributions 814

18.6 Distribution Functions and Probability 817

18.7 Examples of Continuous Distributions 819

18.7.1 Uniform Distribution 819

18.7.2 Gaussian or Normal Distribution 820

18.7.3 Gamma Distribution 821

18.8 Discrete Probability Distributions 821

18.8.1 Uniform Distribution 822

18.8.2 Binomial Distribution 822

18.8.3 Poisson Distribution 824

18.9 Fundamental Theorem of Averages 825

18.10 Moments of Distribution Functions 826

18.10.1 Moments of the Gaussian Distribution 827

18.10.2 Moments of the Binomial Distribution 827

18.10.3 Moments of the Poisson Distribution 829

18.11 Chebyshevs Theorem 831

18.12 Law of Large Numbers 832

References 833

Problems 834

19 Information Theory 841

19.1 Elements of Information Processing Mechanisms 844

19.2 Classical Information Theory 846

19.2.1 Prior Uncertainty and Entropy of Information 848

19.2.2 Joint and Conditional Entropies of Information 851

19.2.3 Decision Theory 854

19.2.4 Decision Theory and Game Theory 856

19.2.5 Travelers Dilemma and Nash Equilibrium 862

19.2.6 Classical Bit or Cbit 866

19.2.7 Operations on Cbits 869

19.3 Quantum Information Theory 871

19.3.1 Basic Quantum Theory 872

19.3.2 Single-Particle Systems and Quantum Information 878

19.3.3 Mach-Zehnder Interferometer 880

19.3.4 Mathematics of the Mach-Zehnder Interferometer 882

19.3.5 Quantum Bit or Qbit 886

19.3.6 The No-Cloning Theorem 889

19.3.7 Entanglement and Bell States 890

19.3.8 Quantum Dense Coding 895

19.3.9 Quantum Teleportation 896

References 900

Problems 901

Further Reading 907

Index 915