### Control Keys

move to next slide (also Enter or Spacebar).
move to previous slide.
d  enable/disable drawing on slides
p  toggles between print and presentation view
CTRL  +  zoom in
CTRL  -  zoom out
CTRL  0  reset zoom

Slides can also be advanced by clicking on the left or right border of the slide.

### Notation

Type Font Examples
Variables (scalars) italics $a, b, x, y$
Functions upright $\mathrm{f}, \mathrm{g}(x), \mathrm{max}(x)$
Vectors bold, elements row-wise $\mathbf{a}, \mathbf{b}= \begin{pmatrix}x\\y\end{pmatrix} = (x, y)^\top,$ $\mathbf{B}=(x, y, z)^\top$
Matrices Typewriter $\mathtt{A}, \mathtt{B}= \begin{bmatrix}a & b\\c & d\end{bmatrix}$
Sets calligraphic $\mathcal{A}, B=\{a, b\}, b \in \mathcal{B}$
Number systems, Coordinate spaces double-struck $\mathbb{N}, \mathbb{Z}, \mathbb{R}^2, \mathbb{R}^3$

### Image-based Lighting

• For image-based lighting, the emitted radiance $L_i$ of the environment is given by a spherical environment map
$s$
$t$
$0.0$
$1.0$
$1.0$
$0.0$
$0.0$
$\pi$
$\pi$
$0.0$
$-\pi$
$\theta$
$\phi$
Source: : HDR environment map from HDRI Haven, CC0

### Environment Map

$y$
$x$
$z$
$\mathbf{p}$
$\phi$
$\theta$
• Relationship between texture coordinates $(s, t)$ and spherical coordinates $(\phi, \theta):$
\begin{align} s &= 0.5 - \frac{\phi}{2\pi} \\ &= 0.5 - \frac{1}{2\pi} \operatorname{arctan2}(y, x)\\ t &= 1.0 - \frac{1}{\pi} \theta \\ &= 1.0 - \frac{1}{\pi} \arccos(z) = \frac{1}{\pi} \arccos(-z) \end{align}
• Example: Envmap
vec2 directionToSphericalEnvmap(vec3 dir) {
float s = 0.5 - atan(dir.y, dir.x) / (2.0 * PI);
float t = 1.0 / (PI) * acos(-dir.z);
return vec2(s, t);
}

### Refresher: Importance Sampling

• Using the theory of importance sampling, we can also approximate any integral by a sum:
$\int\limits_a^b \mathrm{f}(x) \,dx \approx \frac{1}{N} \sum\limits_{n=1}^{N} \frac{\mathrm{f}(x_n)}{\mathrm{p}(x_n)}$
where $p(x)$ is some arbitrary probability density function (PDF) that must fulfill the condition:
$\int\limits_a^b \mathrm{p}(x) \, dx = 1$
• In theory, the best PDF (with the smallest variance) would be
$\mathrm{p}(x) = \frac{\mathrm{f}(x)}{ \int_a^b \mathrm{f}(x)}$
which means the PDF should follow the shape of the function (i.e., the sampling density should be higher if the function values are higher).

### Refresher: Inverse Transform Sampling

• The probability density function (PDF) $\mathrm{p}(x)$ can be chosen arbitrarily
• But how can we generate sampling positions with a specific PDF?
• Many programming languages provide functions to generate uniform distributed samples
• With inverse transform sampling it is possible to transform uniformly generated random numbers to random numbers with other PDFs

### Refresher: Inverse Transform Sampling

1. Compute the cumulative distribution function (CDF) from the probability density function (PDF):
$\mathrm{P}(x) = \int\limits_{-\infty}^x p(\tilde{x}) \,d\tilde{x}$
2. Compute the inverse of the CDF $\,\mathrm{P}^{-1}(u)$
3. Generate uniformly distributed random values $u \in [0, 1]$
4. Compute random numbers $x$ according to $\mathrm{p}(x)$ using the inverse of the CDF
$x = \mathrm{P}^{-1}_x(u)$

### Refresher: Inverse Transform Sampling for a 2D PDF

• The inverse transform sampling can also be used for a two-dimensional PDF $p(x, y)$
• To do this, first calculate the marginal distribution for one dimension, e.g. for $x$:
$p(x) = \int p(x, y) \,dy$
• If $x$ is generated according to this PDF, the conditional probability density function of $y$ is
$p(y | x) = \frac{p(x, y)}{p(x)}$
• Now use the known inverse transform sampling for the two PDFs $p(x)$ and $p(y | x)$
• Obviously, as an alternative, $p(y)$ and $p(x | y)$ can be used as well

### Transformation of Probability Densities

• If a multidimensional random variable $\mathbf{x} = (x_1, x_2, \dots, x_n)^\top$ is mapped with a multidimensional bijective (=biunique) function
$\mathbf{y} = \mathrm{f}(\mathbf{x}) = (\mathrm{f}_1(\mathbf{x}), \mathrm{f}_2(\mathbf{x}), \dots, \mathrm{f}_n(\mathbf{x}))^\top$
it follows for the PDF that
$\mathrm{p}(\mathbf{y}) = \frac{1}{|\mathtt{J}_f(\mathbf{x})|} \mathrm{p}(\mathbf{x})$
where $|\mathtt{J}_f(\mathbf{x})|$ is the determinate of the Jacobian matrix
$\mathtt{J}_f(\mathbf{x}) = \begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \dots & \frac{\partial f_1}{\partial x_n} \\ \vdots & \ddots & \vdots \\ \frac{\partial f_n}{\partial x_1} & \dots & \frac{\partial f_n}{\partial x_n} \end{bmatrix}$

### Importance Sampling of an Environment Map

• The environment image is converted into a gray scale image $\mathrm{g}(s, t)$
• The image is parameterized in texture coordinate $(s, t) \in [0, 1]$
• The image has $S \times T$ pixels and square brackets $\mathrm{g}[x, y]$ denote access to the given pixel
• The integral over the whole gray scale image is:
$I = \int\limits_{s \in [0, 1]} \int\limits_{t \in [0, 1]} \mathrm{g}(s, t) \,ds \, dt \approx \sum\limits_{x=0}^{S-1} \sum\limits_{y=0}^{T-1} \mathrm{g}[x, y] \underbrace{\Delta s}_{\frac{1}{S}} \underbrace{\Delta t}_{\frac{1}{T}} = \underbrace{\frac{1}{S\,T}\sum\limits_{x=0}^{S-1} \sum\limits_{y=0}^{T-1} \mathrm{g}[x,y]}_{\tiny \mbox{precomputed mean}}$
• Thus, for the 2D probability density function, we get
$\mathrm{p}(s, t) = \frac{\mathrm{g}(s, t)}{I}$

### Importance Sampling of an Environment Map

• The marginal density with respect to $t$ is calculated by
$\mathrm{p}_t(t) = \int\limits_{s \in [0, 1]} \mathrm{p}(s, t) \,ds \approx \mathrm{p}_t[y] = \frac{1}{I} \underbrace{\frac{1}{S}\sum\limits_{x=0}^{S-1} \mathrm{g}[x, y]}_{\tiny \mbox{precomputed row averages}}$
• The corresponding cumulative distribution function (CDF) is:
$\mathrm{P}_t(x) = \int\limits_{0}^x \mathrm{p}_t(x) \,dx \approx \frac{1}{T}\sum\limits_{y=0}^Y\mathrm{p}_t[y]$
• The CDF cannot be inverted because the following function cannot be solved for the uniformly distributed random variable $u \in [0, 1]$
$u = \frac{1}{T}\sum\limits_{y=0}^Y\mathrm{p}_t[y]$
Therefore, the shader iterates over $y$ until $u$ is reached. The corresponding $Y$ is the image row at which the sample is taken

### Importance Sampling of an Environment Map

• Now that the image row $Y$ of the sample value was found, the conditional probability distribution $\mathrm{p}_s(s|t)$ must be used to find the image column
• Conditional probability density function is:
$\mathrm{p}_s(s|t) = \frac{\mathrm{p}(s, t)}{\mathrm{p}_t(t)} \approx \mathrm{p}_s[x] = \frac{\mathrm{g}[x, Y]}{\frac{1}{S}\sum\limits_{x=0}^{S-1} \mathrm{g}[x, Y]}$
• The corresponding cumulative distribution function (CDF) is
$\mathrm{P}_s(x) = \int\limits_{0}^x \mathrm{p}_s(x)\,dx \approx \frac{1}{S}\sum\limits_{x=0}^X\mathrm{p}_s[x]$
• This CDF cannot be inverted as well because the following function cannot be solved for the uniformly distributed random variable $v \in [0, 1]$
$v = \frac{1}{S}\sum\limits_{x=0}^X\mathrm{p}_s[x]$
Therefore, the shader iterates over $x$ until $v$ is reached. The corresponding $X$ is the image column at which the sample is taken

### Example: Environment Map Sampling in GLSL

vec2 sampleEnvmap(sampler2D envmap, sampler2D rowAvg,
float totalAvg, vec2 random) {

ivec2 texSize = textureSize(rowAvg, 0);
float sumY = 0.0;
float prevSumY = 0.0;
float ra = 1.0;
int y = 0;
for(y = 0; y < texSize.y; y++) {
ra = texelFetch(rowAvg, ivec2(0, y), 0).r;
float pt = ra / totalAvg;
sumY += pt / float(texSize.y);
if(sumY >= random.y) {
break;
}
prevSumY = sumY;
}
float subPixelY = (random.y - prevSumY) / (sumY - prevSumY);

ivec2 envSize = textureSize(envmap, 0);
float sumX = 0.0;
float prevSumX = 0.0;
int xx = 0;
for(xx= 0; xx < envSize.x; xx++) {
vec3 rgb = pow(texelFetch(envmap, ivec2(xx, y), 0).rgb, vec3(2.2));
float gray = dot(vec3(0.2989, 0.5866, 0.1145), rgb);
float ps = gray / ra;
sumX += ps / float(envSize.x);
if(sumX >= random.x) {
break;
}
prevSumX = sumX;
}
float subPixelX = (random.x - prevSumX) / (sumX - prevSumX);

return vec2((float(xx) + subPixelX) / float(envSize.x),
(float(y) + subPixelY) / float(texSize.y));
}

### Image-based Lighting

• When applied to the rendering equation, the result is:
\begin{align}L_o(\mathbf{v}) &= L_e(\mathbf{v}) + \int\limits_\Omega \mathrm{f}_r(\mathbf{v}, \mathbf{l})\, L_i(\mathbf{l}) \cos(\theta) \, d\omega\\ &= L_e(\mathbf{v}) + \int\limits_{0}^{2\pi}\, \int\limits_{0}^{\pi/2}\underbrace{ \mathrm{f}_r(\mathbf{v}, \mathbf{l})\,L_i(\mathbf{l}) \cos(\theta) \sin(\theta)}_{\mathrm{f}(\theta_n, \phi_n)} \, d\theta \, d\phi\\ &\approx L_e(\mathbf{v}) + \frac{1}{N} \sum_{n=1}^{N} \frac{\mathrm{f}(\theta_n, \phi_n)}{\mathrm{p}(\theta_n, \phi_n)} \end{align}
• Since only $\mathrm{p}(s, t)$ is known and not $\mathrm{p}(\theta, \phi)$, a conversion is required

### Image-based Lighting

• Relationship between texture coordinates $(s, t)$ and spherical coordinates $(\phi, \theta):$
\begin{align} \phi &= 2\pi \, (0.5 - s)\\ \theta &= \pi \, (1.0 - t) \end{align}
• The Jacobian matrix is:
$\mathtt{J} = \begin{bmatrix} \frac{\partial \phi}{\partial s} & \frac{\partial \phi}{\partial t} \\ \frac{\partial\theta}{\partial s} & \frac{\partial \theta}{\partial t} \end{bmatrix} = \begin{bmatrix} -2\pi & 0\\ 0 & -\pi \end{bmatrix}$
• Thus, the determinant of the Jacobian matrix is: $|\mathtt{J}| = 2 \pi^2$
• Substituting into the rendering equation:
\begin{align}L_o(\mathbf{v}) &= L_e(\mathbf{v}) + \int\limits_{0}^{2\pi}\, \int\limits_{0}^{\pi/2}\underbrace{ \mathrm{f}_r(\mathbf{v}, \mathbf{l})\,L_i(\mathbf{l}) \cos(\theta) \sin(\theta)}_{\mathrm{f}(\theta_n, \phi_n)} \, d\theta \, d\phi\\ &\approx L_e(\mathbf{v}) + \frac{1}{N} \sum_{n=1}^{N} \frac{\mathrm{f}(\theta_n, \phi_n)}{\mathrm{p}(s_n, t_n)} 2 \pi^2 \end{align}

### Are there any questions?

Please notify me by e-mail if you have questions, suggestions for improvement, or found typos: Contact