Graphics Programming
Shadows

• Shadows are very important for the realism of the scene and for estimating the distances between objects
• However, our currently used lighting model does not take shadows into account
• All light sources in the direction of the surface normal (half-space) were previously assumed to be visible
• This has the advantage that the lighting calculation is only dependent on the current polygon (or fragment).
• However, when calculating shadows, all other polygons must be known, since any other polygon can block the light source for the current fragment
• Testing the visibility of a light source is therefore a difficult task and is not directly supported by the OpenGL pipeline.
• This chapter presents some methods that enable real-time shadow calculations

Projection onto the $z=0$ plane

• Idea: projection of the object onto a planar surface
• For example, a simple parallel projection onto the plane $z=0$ with the following matrix:
  $\mathtt{T}_{\tiny \mbox{shadow}} = \begin{bmatrix}1 & 0& 0 & 0\\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 &1\end{bmatrix}$
• This transformation must be applied in the world coordinate system. That means:
  $\underline{\tilde{\mathbf{P}}} = \mathtt{A} \, \underbrace{\mathtt{T}_{\mathrm{\small cam}}^{-1} \, \mathtt{T}_{\tiny \mbox{shadow}} \, \mathtt{T}_{\mathrm{\small obj}}}_{\mathtt{T}_{\mathrm{\small modelview}}} \, \underline{\mathbf{P}}$
• Draw the projected object in black in addition to the original scene

• Projection of the occluding object on a planar surface given the position of a point light source
• Draw the projected object in black in addition to the original scene

$x$

$z$

Vertex $\mathbf{P}=(p_x, p_y, p_z)^\top$

Projected vertex $\tilde{\mathbf{P}}=(\tilde{p}_x, \tilde{p}_y, \tilde{p}_z)^\top$

Light source $\mathbf{L}=(l_x, l_y, l_z)^\top$

• Given the position of the light source $\mathbf{L}=(l_x, l_y, l_z)^\top$, the projection of the vertex $\tilde{\mathbf{P}}=(\tilde{p}_x, \tilde{p}_y, \tilde{p}_z)^\top$ on the plane $z=0$ is calculated from the original vertex $\mathbf{P}=(p_x, p_y, p_z)^\top$ using the intercept theorem (see figure):
  $\frac{\tilde{p}_x - l_x}{l_z} = \frac{p_x - l_x}{l_z - p_z} \Leftrightarrow \tilde{p}_x = \frac{p_x - l_x}{l_z - p_z} l_z+ l_x = \frac{p_x l_z - l_x l_z + l_x l_z - l_x p_z}{l_z - p_z} = \frac{l_z p_x - l_x p_z}{l_z - p_z}$

• This means:
  $\tilde{p}_x = \frac{l_z p_x - l_x p_z}{l_z - p_z} \quad \quad\tilde{p}_y = \frac{l_z p_y - l_y p_z}{l_z - p_z} \quad \quad\tilde{p}_z = 0$
• When using homogeneous coordinates, this mapping can also be written as a projective $4 \times 4$ matrix:
  $ \tilde{\underline{\mathbf{P}}} = \begin{pmatrix}\tilde{x}\\ \tilde{y} \\ \tilde{z} \\ \tilde{w} \end{pmatrix}= \underbrace{\begin{bmatrix}l_z & 0& -l_x & 0\\ 0 & l_z & -l_y & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & -1 & l_z\end{bmatrix}}_{\mathtt{T}_{\tiny \mbox{shadow}} } \begin{pmatrix}p_x\\p_y\\p_z\\1\end{pmatrix}$

  $\tilde{\underline{\mathbf{P}}} = \begin{pmatrix}\tilde{x}\\ \tilde{y} \\ \tilde{z} \\ \tilde{w} \end{pmatrix} \in \mathbb{H}^3 \quad \longmapsto \quad \tilde{\mathbf{P}} = \begin{pmatrix} \tilde{p}_x\\ \tilde{p}_y\\ \tilde{p}_z \end{pmatrix} = \begin{pmatrix}\frac{\tilde{x}}{\tilde{w}}\\\frac{\tilde{y}}{\tilde{w}}\\\frac{\tilde{z}}{\tilde{w}} \end{pmatrix} \in \mathbb{R}^3 $

• When applied in the global coordinate system, the overall result is again:
  $\underline{\tilde{\mathbf{P}}} = \mathtt{A} \, \mathtt{T}_{\mathrm{\small cam}}^{-1} \, \mathtt{T}_{\tiny \mbox{shadow}} \, \mathtt{T}_{\mathrm{\small obj}}\, \underline{\mathbf{P}}$

• Projection onto $z=0$:
  $ \mathtt{T}_{\tiny \mbox{shadow}} = \begin{bmatrix}l_z & 0& -l_x & 0\\ 0 & l_z & -l_y & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & -1 & l_z\end{bmatrix}$
• Projection onto $y=0$:
  $ \mathtt{T}_{\tiny \mbox{shadow}} = \begin{bmatrix}l_y & -l_x &0& 0\\ 0 & 0 & 0 & 0 \\ 0 & -l_z & l_y & 0 \\ 0 & -1 & 0 & l_y\end{bmatrix}$
• Projection onto $x=0$:
  $ \mathtt{T}_{\tiny \mbox{shadow}} = \begin{bmatrix}0 & 0 &0& 0\\ -l_y & l_x & 0 & 0 \\ -l_z & 0 & l_x & 0 \\ -1 & 0 & 0 & l_x\end{bmatrix}$

• GSN Composer: PlanarShadow

• Normally, the ambient part of the lighting model should remain visible in the shadow
• However, this means that the scene has to be rendered twice
  • 1st pass: diffuse and specular part including shadows
  • 2nd pass: only the ambient part without shadows

Diffuse and specular part

Ambient part

Result

+

=

• In this step, the scene must be rendered from the point of view of the light source
• This is relatively easy for a spotlight, since it has a 3D position $\mathbf{p}$, a main direction of radiation $\mathbf{d}$ and an outer angle $\beta_{\tiny \mbox{outer}}$ of the light cone
• To position the "light"-camera with $\mathtt{T}_{\mathrm{\tiny cam/light}}^{-1}$ we can us the function gluLookAt (see Part 6, Chapter 1)

  gluLookAt(eyex, eyey, eyez, refx, refy, refz, upx, upy, upz);

  • The eye point $\mathbf{c}_{\mathrm{\small eye}}$ of the camera corresponds to the 3D position $\mathbf{p}$ of the spotlight
  • The targeted reference point $\mathbf{p}_{\mathrm{\small ref}}$ is given by:
    $\mathbf{p}_{\mathrm{\small ref}} = \mathbf{p} + \mathbf{d}$
• The projection matrix $\mathtt{A}_{\mathrm{\tiny cam/light}}$ can be generated with the function gluPerspective (see Part 6, Chapter 1)

  gluPerspective(fovy, aspect, near, far)

  • Thereby, the camera's field of view fovy $=\beta_{\tiny \mbox{outer}}$

Light position

Shadow Map

0.0

1.0

0.0

1.0

• To project the shadow map, we need to determine the corresponding texture coordinates for each vertex
• After applying
  $\underline{\tilde{\mathbf{P}}} = \mathtt{A}_{\mathrm{\tiny cam/light}} \, \mathtt{T}_{\mathrm{\tiny cam/light}}^{-1} \, \mathtt{T}_{\mathrm{\small obj}}\, \underline{\mathbf{P}}$
  and perspective division, the $x$, $y$, and $z$ coordinates of a vertex are in the range [-1.0, 1.0]
• However, the texture coordinates $s$ and $t$ must be in the range [0.0, 1.0]. This can be achieved by an additional transformation:
  $\begin{pmatrix}\tilde{s}\\\tilde{t}\\\tilde{p}\\\tilde{q}\end{pmatrix} = \underbrace{\begin{bmatrix}0.5 & 0 & 0 & 0.5\\0 & 0.5 & 0 & 0.5\\0 & 0 & 0.5 & 0.5\\0 & 0 & 0 & 1.0\end{bmatrix} \, \mathtt{A}_{\mathrm{\tiny cam/light}} \, \mathtt{T}_{\mathrm{\tiny cam/light}}^{-1}}_{\mathtt{T}_{\tiny \mbox{shadow}}} \,\,\mathtt{T}_{\mathrm{\small obj}}\, \underline{\mathbf{P}}$

• After perspective division, the result is:
  $\begin{pmatrix}\tilde{s}\\\tilde{t}\\\tilde{p}\\\tilde{q}\end{pmatrix} = \underbrace{\begin{bmatrix}0.5 & 0 & 0 & 0.5\\0 & 0.5 & 0 & 0.5\\0 & 0 & 0.5 & 0.5\\0 & 0 & 0 & 1.0\end{bmatrix} \, \mathtt{A}_{\mathrm{\tiny cam/light}} \, \mathtt{T}_{\mathrm{\tiny cam/light}}^{-1}}_{\mathtt{T}_{\tiny \mbox{shadow}}} \,\,\mathtt{T}_{\mathrm{\small obj}}\, \underline{\mathbf{P}}$
  $\begin{pmatrix}s\\t\\p\end{pmatrix} = \begin{pmatrix}\frac{\tilde{s}}{\tilde{q}}\\ \frac{\tilde{t}}{\tilde{q}} \\ \frac{\tilde{p}}{\tilde{q}} \end{pmatrix} \in \mathbb{R}^3 $
• Where $s$ and $t$ are the sought-after texture coordinates and $p$ is the true distance between the vertex and the light position in depth buffer scaling (all three values lie in the range [0.0, 1.0])
• The depth buffer scaling depends on the near and far plane selected when creating $\mathtt{A}_{\mathrm{\tiny cam/light}}$

• The true distance of a fragment from the light source is given by:
  $z_{\mathrm{\small true}} = p$
• The distance to the nearest object as seen from the light source can be found by looking it up in the shadow map:
  $z_{\mathrm{\small nearest}} =$ texture(shadowmap, vec2(s,t));
• If $z_{\mathrm{\small nearest}} < z_{\mathrm{\small true}}$, the fragment is occluded by another polygon and thus in shadow

Point in shadow

Point in light

Light position

Shadow Map

Framebuffer

Camera

$z_{\mathrm{\small nearest}}$

$z_{\mathrm{\small true}}$

$z_{\mathrm{\small nearest}}$

$z_{\mathrm{\small true}}$

• GSN Composer: ShadowMap

Vertex shader:

#version 300 es
precision highp float; 
precision highp int; 

in vec3 position; // input vertex position from mesh
in vec2 texcoord; // input vertex texture coordinate from mesh
in vec3 normal;   // input vertex normal from mesh

uniform mat4 cameraLookAt; // camera look at matrix
uniform mat4 cameraProjection; // camera projection matrix
uniform mat4 spotLightTransform; // transformation matrix for the spot light
uniform mat4 meshTransform; // mesh transformation
uniform mat4 meshTransformTransposedInverse; // transposed inverse meshTransform
uniform mat4 shadowTransform; // shadow map transformation

out vec2 tc; // output texture coordinate of vertex
out vec3 wfn; // output fragment normal of vertex in world coordinate system
out vec3 vertPos; // output 3D position in world coordinate system
out vec4 shadowCoord; // output texture coordinate in shadow map

void main(){
  tc = texcoord;
  wfn = vec3(meshTransformTransposedInverse * vec4(normal, 0.0));
  vec4 vertPos4 = meshTransform * vec4(position, 1.0);
  vertPos = vec3(vertPos4) / vertPos4.w;
  
  //texture coordinates in shadow map
  shadowCoord = shadowTransform * vertPos4;
  
  gl_Position = cameraProjection * cameraLookAt * vertPos4;
}

Fragment shader:

#version 300 es
precision highp float;
precision highp int;
out vec4 outColor;

in vec2 tc; // texture coordinate of pixel (interpolated)
in vec3 wfn; // fragment normal of pixel (interpolated)
in vec3 vertPos; // fragment vertex position (interpolated)
in vec4 shadowCoord; // texture coordinate in shadow map  (interpolated)

uniform int mode; // debug mode
uniform sampler2D diffuseTex; // diffuse texture
uniform float ambientFactor; // ambient factor
uniform vec4 specularColor; // specular color
uniform float shininess; // specular shininess exponent
uniform vec4 lightColor; // color of light
uniform vec3 lightPosition; // light position in world space
uniform vec3 lightDirection; // spot light direction in world space
uniform float lightCutoff; // spot light cut off
uniform float lightExponent; // spot light exponent
uniform float lightAttenuation; // quadratic light attenu.
uniform vec3 cameraPos; // camera position in world space
uniform float shadowBias; // shadow bias
uniform sampler2D shadowMap; // shadow map


vec3 blinnPhongBRDF(vec3 lightDir, vec3 viewDir, vec3 normal, 
            vec3 phongDiffuseCol, vec3 phongSpecularCol, float phongShininess) {
  vec3 color = phongDiffuseCol;
  vec3 halfDir = normalize(viewDir + lightDir);
  float specDot = max(dot(halfDir, normal), 0.0);
  color += pow(specDot, phongShininess) * phongSpecularCol;
  return color;
}

vec3 blinnPhongShading(vec3 n, vec3 viewDir, vec3 lightDir, vec4 lightColor, 
                       float lightAttenuation, vec4 diffuseColor, vec4 specColor, 
                       float shininess, float ambientFactor) 
{
  // ambient Term
  vec3 radiance = ambientFactor * diffuseColor.rgb;
  // irradiance contribution from light
  float irradiance = max(dot(lightDir, n), 0.0); 
  if(irradiance > 0.0) { // if receives light
    vec3 brdf = blinnPhongBRDF(lightDir, viewDir, n, 
                                diffuseColor.rgb, specColor.rgb, shininess);
    radiance += brdf * irradiance * lightColor.rgb * lightAttenuation;
  }
  return radiance;
}

void main() {
  vec3 lightVec = lightPosition - vertPos;
  float d = length(lightVec); // distance to light
  vec3 lightDir = normalize(lightVec);
  vec3 normal = normalize(wfn.xyz);
  vec3 viewDir = normalize(cameraPos - vertPos);

  // compute shadow;
  vec4 shadowCoordDiv = shadowCoord / shadowCoord.w;
  float nearestZ = texture(shadowMap, shadowCoordDiv.st).z;
  float trueZ = shadowCoordDiv.z - shadowBias;
  float shadowVal = nearestZ < trueZ ? 0.5 : 1.0;

  float shadow = shadowVal;
  if(dot(lightDir, normal) <= 0.0) {
    shadow = 0.5;
  }

  // attenuation due to spot
  float attenuation = abs(dot(lightDir, lightDirection));
  if(attenuation < lightCutoff) {
    attenuation = 0.0;
  } else {
    attenuation = pow(attenuation, lightExponent);
    // atttenuation due to quadratic distance fall off
    attenuation *= 1.0 / (lightAttenuation * d * d);
  }

  vec4 diffuseColor = texture(diffuseTex, tc);
  vec4 color = ambientFactor * diffuseColor * lightColor;
  color.a = 1.0;

  if(shadow > 0.75) {
    color.rgb = blinnPhongShading(normal, viewDir, lightDir, 
                  lightColor, attenuation, diffuseColor, specularColor, 
                  shininess, ambientFactor);
  }

  outColor = clamp(color, 0.0, 1.0);
}

• Due to the finite resolution of the shadow map and the quantisation (rounding) of the depth values, inaccuracies arise when comparing the depths $z_{\mathrm{\small nearest}}$ and $z_{\mathrm{\small true}}$
• This is sought to be compensated by a "Shadow Bias" (also referred to as "Polygon Offset", "Depth Bias")
  

  without offset suitable offsettoo large offset
• A too small offset can cause "Shadow Leaking", If the offset is too large, the shadow appears in the wrong place and objects start to "fly"

• Advantages
  • Can be used on any receiver surface
  • Self-shadowing
  • Relatively quick to calculate
• Disadvantages
  • Staircase effects on shadow edges (aliasing)
  • Shadow bias is scene-dependent
  • Initially only for spotlights
• Extensions:
  • Directional light source ⟶ Cascaded Shadow Maps (CSM)
  • Point light source ⟶ Omnidirectional Shadow Mapping