![cc](pix/cc-attrib-nc.png)
Licensed under the 
[Creative Commons attribution-noncommercial license](http://creativecommons.org/licenses/by-nc/3.0/).
Please share \& remix noncommercially, mentioning its origin.

## What is a "nonlinear" model?

- a model whose objective function (usu. likelihood) *cannot* be expressed as a linear function of predictors ($\beta_0 + \beta_1 x_1 + \beta_2 x_2$ ...)

## What's so good about linear models?

- solution by linear algebra, rather than iteration
- *generalized* linear models - *iterative* linear algebra (IRLS: @kane_glms_2016, @arnold_computational_2019)
- guarantees:
     - performance
	 - convergence
	 - unimodal, *log-quadratic* likelihoods
- no hyperparameter tuning
- no tests for convergence
- no need to pick starting values
- robust to parameter scaling and correlation
- interpretability
- convenient framework for covariates/interactions/etc.

## what does "linear" include?

- additive models (splines for general smooth continuous functions)
- linearization tricks 
     - e.g. generalized Ricker: $y = a^b x e^{cx} \to \log(y) = \log(a) + b \log(x) + c x$
	 - Michaelis-Menten: $y = ax/(b+x) \to 1/y = (b/a) (1/x) + (1/a)$
	 - power-law (allometric) fitting: $y=ax^b \to \log(y) = a + b \log(x)$
- borderline:
    - generalized least squares
    - generalized linear models (link function)
    - (G)L mixed models

## Nonlinear models

- everything else 

- "nonlinear" $\leftrightarrow$ most zoology = the study of non-elephant animals (Ulam)
- many methods use *locally* linear approaches (gradient descent; Newton)
- today's class focuses on less-canned/one-off solutions

## Simple examples

- power-law curves (if additive: $y=ax^b$ can be linearized)
- logistic curves, unknown asymptote ($K$, or prob <1), and generalizations (e.g. Richards equation)
- simple movement models (log-Normal step length/von Mises turning angle)
- saturating curves (functional responses: Holling/Rogers equation)

## Harder examples

- trajectory-matching ODEs
- seed shadow models
- mixture models

## Special-case toolboxes

Many classes of problems have special algorithms that linearize or otherwise solve most of the problem, then solve a *low-dimensional* nonlinear optimization problem

- Kalman filtering
- hidden Markov models (Viterbi algorithm)
- linear mixed models
- generalized linear (mixed) models (IRLS/penalized IRLS+Laplace/quadrature)
- generalized least-squares
- mixture models via expectation-maximization
- capture-mark-recapture data
- machine learning (neural networks, random forests, ...)

## General advice

- simplify problem expression
- use most efficient algorithms
- good starting values (more later)




