Misc-Functions¶
The misc-functions are a small collection of snippets of code that execute small but repeatedly used functions. They are primarily used by the LogLikelihood class, though some functions are called elsewhere.
-
double sigmoid(double x)¶
Implements an expit sigmoid. Some cleverness was used to avoid numerical overflows for exp(large numbers).
- Parameters
x – Value to be expit-ed
- Returns
\(\sigma(x) = \frac{1}{1 + \exp(-x)}\)
-
double elu(double x)¶
A truncated exponential (apparently elu is a mahcine learning term?). For x > elu_transitionPoint, returns exp(-x), otherwise returns the linear function which makes elu(x) both continuous and smooth.
- Parameters
x – Value to be elu-ed
- Returns
\(\text{elu}(x) =\begin{cases} \exp(-x) ~~~& x > \rho \\ (1 + \rho - x) \exp(-\rho) & x \leq \rho \end{cases}\)
-
double elu_grad(double x, double elu_x)¶
Calculate the gradient of elu() evaluated at x.
- Parameters
x – The value at which gradient is to be evaluated at
elu_x – the functional value of elu() at the chosen point. Can avoid having to recalulate exp(x) if possible
- Returns
The requisite gradient
-
double log_add_exp(double a, double b)¶
Given two (potentially very large) numbers a = log(x) ,b = log(y) we often want to know log(x+y). To compute this we need to take exponentials and then add/subtract them: even if the answer should be a finite number, the intermediary values can often lead to numerical overflows. This function calculates it in a much safer way.
- Parameters
a – The logarithm of a large number
b – The logarithm of another large number
- Returns
\(\log\left( e^a + e^b\right)\) , calculated such that the intermediary results do not overflow