Control Keys

→ move to next slide (also Enter or Spacebar).
← move to previous slide.
 d  enable/disable drawing on slides
 p  toggles between print and presentation view
CTRL  +  zoom in
CTRL  -  zoom out
CTRL  0  reset zoom

Slides can also be advanced by clicking on the left or right border of the slide.

Human Senses

  • Humans take in information with their senses:
    • Hearing
    • Vision
    • Smell
    • Taste
    • Touch
  • Are there more senses?
    • Sense of temperature
    • Pain sensation
    • Sense of balance

Components of Multimedia Documents

  • Today's multimedia documents usually consist of information for the senses "hearing" and "vision":
  • Text
    • Unformatted text
    • Text in various formats: HTML, PDF, PS, Latex, Word, Powerpoint, ...
  • Images
    • 2D vector graphics: Consisting of geometric primitives, such as lines, circles, text blocks, ...
    • Raster images: Consisting of pixel data

Components of Multimedia Documents

  • Audio
    • with low bandwidth (e.g., for human speech)
    • with medium bandwidth (CD quality)
    • with high bandwidth (e.g. audio DVD quality)
  • Video
    • Typically temporal sequence of raster images
    • Due to their large storage requirements, videos are almost always stored or transmitted in compressed form

Text

Character Encoding

  • To represent text on a computer, each character must be encoded in binary
  • Depending on how many bits are used per character, different numbers of characters can be encoded
  • For example:
    • 7 Bits: 27 = 128 different characters
    • 8 Bits: 28 = 256 different characters
    • 16 Bits: 216 = 65536 different characters

ASCII-Code

  • The ASCII code (American Standard Code for Information Interchange) is a 7-bit character encoding, introduced by the American Standards Association (ASA) in 1963
  • However, each character is stored as 1 byte (=8 bits), i.e. the most significant (8th) bit is always zero
  • There are 128 characters in total, of which 95 are printable and 33 are control characters

ASCII-Code

  • All 128 ASCII characters are specified in the following table
  • Move the mouse over a character to see a brief description
Hex Code …0 …1 …2 …3 …4 …5 …6 …7 …8 …9 …A …B …C …D …E …F
0… NUL SOH STX ETX EOT ENQ ACK BEL BS HT LF VT FF CR SO SI
1… DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US
2… SP ! " # $ % & ' ( ) * + , - . /
3… 0 1 2 3 4 5 6 7 8 9 : ; < = > ?
4… @ A B C D E F G H I J K L M N O
5… P Q R S T U V W X Y Z [ \ ] ^ _
6… ` a b c d e f g h i j k l m n o
7… p q r s t u v w x y z { | } ~ DEL
  • Example:
    The character "A" has the hexadecimal value (41)16
    (41)16 = (01000001)2 = (65)10
Source: Wikipedia

ASCII-Code

  • The control characters come from a time when the ASCII code was used to control teletypewriters (electrically controlled typewriters)
  • Today, many of these control characters have lost their relevance
  • The only important thing is the control character for a new line: "LF" (Line Feed, ASCII (0A)16)
  • However, when using the Windows operating system, the "Line Feed" character must be preceded by a "Carriage Return": "CR LF" (=ASCII (0D)16 (0A)16)

ISO 8859

ISO 8859-1 Western European (Latin-1)
ISO 8859-2 Central European (Latin-2)
ISO 8859-3 South European (Latin-3)
ISO 8859-4 North European (Latin-4)
ISO 8859-5 Latin/Cyrillic
ISO 8859-6 Arabic
ISO 8859-7 Greek
ISO 8859-8 Hebrew
ISO 8859-9 Turkish (Latin-5)
ISO 8859-10 Nordic (Latin-6)
ISO 8859-11 Thai
ISO 8859-12 verworfen
ISO 8859-13 Baltic Rim (Latin-7)
ISO 8859-14 Celtic (Latin-8)
ISO 8859-15 Westeuropäisch (Latin-9)
ISO 8859-16 South-Eastern Eur. (Latin-10)
  • With ASCII coding, only 7 of the 8 bits of a byte are used
  • The remaining number range (128 to 255) can therefore be used for additional characters
  • The International Organization for Standardization defines a total of 15 ASCII extensions in ISO 8859
  • For example, ISO 8859-1 contains the characters that are important for us in Germany: "ä", "ö", "ü", "ß"

Unicode

  • When using ISO 8859 to exchange texts, wrong characters might be displayed. This easily happens when the sender and receiver do not use the same ISO 8859-x standard use for decoding
  • In addition, ISO 8859 does not include all characters from different cultures
  • The aspiration of Unicode is to define a single universal encoding that covers all relevant characters
  • The Unicode was standardized by the ISO as ISO-10646

Unicode

  • The Unicode consists of 17 levels (representable with 5 bits)
  • Each level has 16 bits and can theoretically encode 216 = 65536 characters
  • In total, a Unicode character requires 5 + 16 = 21 bits
  • Most of the currently used characters can be found in level 0, the Basic Multilingual Plane (BMP).
  • A Unicode character is often specified as a "U+" and a hexadecimal number of at least 4 digits
  • Examples:
    U+00E4 for the "ä"
    U+1300C for the Egyptian hieroglyph hiero_A10
Image source: Egyptian Hieroglyphs (Unicode_block)

Unicode

  • The encoding of all possible characters is an ongoing process, i.e. the number of characters is constantly growing
  • Most fonts only provide a small subset of the characters defined in Unicode
  • If a character does not exist in a font, it is often simply taken from another font
  • The website https://www.decodeunicode.org/ aims to display all characters currently encoded in Unicode

Unicode

  • Unicode was designed with continuity in mind
  • Out of the 21 bits of the Unicode, the first 7 bits correspond to the ASCII code and the first 8 bits to the ASCII extensions ISO 8859-1 (Latin 1)
unicode

UTF

utf
  • The UTF "Universal Transformation Format" is used to encode Unicode characters
  • UTF-32 encodes a 32-bit Unicode character by padding the 21 Unicode bits with zeros
  • UTF-16 encodes all bits of the Basic Multilingual Plane (BMP) with 16 bits, only for the other planes 32 bits required

UTF-8

utf
  • UTF-8 encodes the first 7 Unicode bits (equivalent to ASCII) with 8 bits,
    the first 11 Unicode bits with 16 bits, etc.
  • A UTF-8 encoded text containing only ASCII characters is therefore fully compatible with ASCII
  • UTF-8 is widely used today, especially on the Internet (quasi-standard of character encoding).

Web Pages

  • Web pages on the World Wide Web (WWW) combine media (such as text, audio, images and video) and allow the user to receive information interactively or provide user-generated content
  • Web pages are created using Hypertext Markup Language (HTML)
  • So-called hyperlinks can refer to other websites
  • The hyperlinks create a net-like structure (hence "web")

Hypertext Markup Language (HTML)

  • HTML web pages are structured using HTML elements
  • An HTML element is usually described by a pair of tags, which consists of a start tag and an end tag that contain the actual content
  • For example, for a paragraph with text: 
    <p>This is some text</p>
  • However, there are also HTML elements that have no content. Then there is no explicit end tag
  • For example, a forced line break: 
    <p>This is more text <br> with a line break</p>
  • Additional attributes can be specified in the start tags
  • For example, the target of a hyperlink 
    <p>Please visit my <a href="otherSite.html">other website</a> with more text</p>

A Web Page with Text

  • Here is an example of a simple HTML5 webpage with text:
<!doctype html>

<html>
<head>
  <meta charset="utf-8">
  <title>A simple HTML5 example</title>
</head>

<body>
  <h1>This is a heading</h1>
  <p>This is some text</p>
  <h2>This is a smaller heading</h2>
  <p>This is more text <br> with a line break</p>
  <p>Please visit my <a href="otherSite.html">other website</a> with more text</p>
</body>
</html>

Source code: simple.html

A Web Page with Text

  • If the source is saved as an ASCII file, Unicode characters can be specified by their hexadecimal code: &#x00E4;
  • If it is ensured that the file can always be edited with editors that use UTF-8, the file can also be saved directly as UTF-8
  • The encoding is specified via charset
<!doctype html>

<html>
<head>
  <meta charset="utf-8">
  <title>International order</title>
</head>

<body>
  <h1>International order</h1>
  <p>Hello German user: Thorm&#x00E4;hlen with &auml;. Please pay 1000 &#x20AC;.</p>
  <p>Hello Chinese user: &#x8B5A; &#x5929; &#x7FF0;. Please pay 1000 &#x00A5;.</p>
</body>
</html>

Source code: charset.html

Changing the Visual Appearance with CSS

  • Cascading Style Sheets (CSS) allow changing the visual appearance of a website
  • The big advantage is that content and appearance can be treated separately
<!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <title>A simple HTML5 and CSS example</title>
    <style>
      body {  
        font-family: Verdana, Arial, Helvetica, sans-serif;
        background-color: #003366;
        color: #FFFFFF;
      } 
      p {
        color: #808080;
      }
      .myClass {
        color: #00FF00;
        font-style: italic;
      }
      
      #myUniqueId {
        color: #00FFFF;
      }
    </style>
  </head>

  <body>
    <h1>This is a heading</h1>
    <p>This is some text</p>
    <p class="myClass">This is text of type "myclass"</p>
    <p>This is more normal text</p>
    <p class="myClass">This is more text of type "myclass"</p>
    <p id="myUniqueId" class="myClass">This text of type "myclass" 
                                       has an ID that makes it unique</p>
  </body>
</html>

Source code: simplecss.html

Changing the Visual Appearance with CSS

  • CSS styles can also be defined in a separate file
<!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <title>A simple HTML5 and CSS example</title>
    <link rel="stylesheet" href="styles.css">
  </head>

  <body>
    ...
  </body>
</html>

The HTML Elements<div> and <span>

  • The HTML elements<div> and <span> can be used to structure a HTML document and are also important when assigning CSS styles
  • The difference between the two is that the <div> element elements are arranged in blocks while the <span> elements are arranged in a line 
<!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <title>A simple HTML5 and CSS example</title>
    <style type="text/css">
      body {  
        font-family: Verdana, Arial, Helvetica, sans-serif;
        background-color: #000000;
        color: #FFFFFF;
      } 
      .myRedDivs {
        background-color: #FF0000;
      }
      .myGreenDivs {
        background-color: #00FF00;
      }
      .myBlueSpan {
        background-color: #0000FF;
      }
    </style>
  </head>

  <body>
    <div class="myRedDivs">
      <h1>This is a heading</h1>
      <p>This is some text</p> 
    </div>
    <div class="myGreenDivs">
      <h1>This is a heading</h1>
      <p>This is some text</p> 
    </div>
    <div class="myRedDivs">
      <h1>This is a <span class="myBlueSpan">heading</span></h1>
      <p>This is some text</p> 
    </div>
  </body>
</html>

Source code: css_div_span.html

Images

Vector Graphics

  • One way to define a 2D image is by a Vector Graphic
  • The term stems from the fact that the coordinates can be written as vectors
  • For a point $\mathbf{p}$ in a 2D coordinate system, for example,

    $\mathbf{p}=\begin{pmatrix} x\\y\end{pmatrix} = \begin{pmatrix} 4.5\\2.33\end{pmatrix}$

Vector Graphics

  • With vector graphics, objects are described by combining basic objects (such as line, rectangle, circle, triangle, polygon)
vectordraw
$x$-coordinate
$y$-coordinate
triangle
rectangle

Vector Graphics

public void paint(Graphics g) {
  Graphics2D g2d = (Graphics2D) g; // cast to Graphics2D
  g2d.draw(new Rectangle2D.Double(20.75, 45.0, 60.0, 30.00) );
  g2d.fill(new Rectangle2D.Double(67.33, 60.0,  8.0, 15.00) ); 
  g2d.fill(new Rectangle2D.Double(35.00, 55.0, 20.3, 11.00) );
  GeneralPath triangle = new GeneralPath(); 
  triangle.moveTo(10.75, 46.0);
  triangle.lineTo(90.75, 46.0);
  triangle.lineTo(50.75, 10.0);
  triangle.closePath();
  g2d.fill(triangle);
 }
vectordrawsmall

Full source code:
MyGraphics2DHouse.java

Vector Graphics

  • A representation as a vector graphic has the advantage that the objects can be arbitrarily scaled
  • For example, an analog plotter can approach the coordinates accurately and thus draw a very accurate picture of the shapes
worldmapfull
World map
worldmapdetail
Detail magnification
Quelle: John Harvey, Blank map of the world Equirectangular, Creative Commons License

Interactive Demonstration: Vector Graphics

your browser does not support SVG

Scaling: 100%:

Source: Johnny Automatic, Robot Dog, Public Domain

Creating Vector Graphics

To create 2D vector graphics, there are many free and commercial programs:

Storing Vector Graphics

Popular file formats for storing 2D vector graphics are:

A Website with 2D Vector Graphics

  • There are three ways to insert 2D vector graphics into web pages
    • Option 1: <svg> element
    • Option 2: <img> element
    • Option 3: Draw via JavaScript in a <canvas> element

A Website with 2D Vector Graphics

  • Option 1: <svg> element
<!doctype html>

<html>
  <head>
    <meta charset="utf-8">
    <title>An example for inline SVG</title>
  </head>

  <body>
    <h1>An example for inline SVG</h1>
    <p>This is some text</p>
    <svg width="100" height="100" style="border:1px solid black;">
      <rect x="20.75" y="45.0" width="60.0" height="30.0" style="fill:none; stroke-width:1; stroke:rgb(0,0,0);" />
      <rect x="67.33" y="60.0" width="8.0" height="15.0" style="fill:rgb(0,0,0); stroke:none;" />
      <rect x="35.0" y="55.0" width="20.3" height="11.0" style="fill:rgb(0,0,0); stroke:none;" />
      <polygon points="10.75,46.0 90.75,46.0 50.75,10.0" style="fill:rgb(0,0,0); stroke:none;" />
    </svg> 
  </body>
</html>

Source code: svg.html

A Website with 2D Vector Graphics

  • Option 2: <img> element
<!doctype html>

<html>
  <head>
    <meta charset="utf-8">
    <title>An example for embedding an SVG object</title>
  </head>

  <body>
    <h1>An example for embedding an SVG object</h1>
    <p>This is some text</p>
    <img src="./house.svg"  style="width:200px; height:200px" alt="house">
  </body>
</html>

Source code: svg_import.html

A Website with 2D Vector Graphics

  • Option 3: Draw via JavaScript in a <canvas> element
<!doctype html>

<html>
  <head>
    <meta charset="utf-8">
    <title>An example for drawing in an HTML canvas element</title>
    <script>
      function draw() {
        var c = document.getElementById("myCanvas");
        var ctx = c.getContext("2d");
        ctx.lineWidth = "1";
        ctx.strokeStyle = "#000000";
        ctx.beginPath();
        ctx.rect(20.75, 45.0, 60.0, 30.00);
        ctx.stroke();
        ctx.beginPath();
        ctx.rect(67.33, 60.0, 8.0, 15.00);
        ctx.fill();
        ctx.beginPath();
        ctx.rect(35.00, 55.0, 20.3, 11.00);
        ctx.fill();
        ctx.beginPath();
        ctx.moveTo(10.75, 46.0);
        ctx.lineTo(90.75, 46.0);
        ctx.lineTo(50.75, 10.0);
        ctx.closePath();
        ctx.fill();
       }
    </script> 
  </head>


  <body onload="draw();">
    <h1>An example for drawing in an HTML canvas element</h1>
    <p>This is some text</p>
    <canvas id="myCanvas" width="100" height="100">
      Your browser does not support the HTML5 canvas tag.
    </canvas>
  </body>
</html>

Source code: canvas.html

Raster Graphics

  • Today's screens provide individual pixels (Pixel="Picture Elements"), which are arranged on a fixed grid
  • Printers with linear paper feed (no plotters) apply their color also on a discrete dot matrix
  • Raster images have a color value for each pixel of their grid. They are therefore fully defined by:
    • Image width: Number of pixels in the x direction
    • Image Height: number of pixels in the y-direction
    • and a data field with (width x height) color values

Raster Graphics

  • Here is an example of a raster image with 9 x 6 pixels (counting starts at 0)
  • In the following it is assumed that the pixel grid intersects with the center of the pixel area
pixeldraw
$x$-coordinate
$y$-coordinate
pixel area
pixel grid
outer edge of raster graphics

Raster Graphics

  • Digital images are typical examples for raster graphics
  • For example, image data of real objects are recorded with a digital camera with photosensitive pixel areas
    • 1280 × 1024 (1,3 megapixels)
    • 2560 × 2048 (5,2 megapixels)
    • 5120 × 4096 (21 megapixels)
  • Typical conventions for raster graphics:
    • Counting of pixels starts at 0
    • $x$-coordinate axis to the right
    • $y$-coordinate axis downwards

Raster Graphics

  • Very often per pixel an RGB color value (red, green, blue) is stored (or RGBA value, whereby the additional alpha value indicates the opacity)
  • RGB color space:
  • The image data is stored sequentially in memory
  • Most formats use the convention that the pixel data is stored row-wise

Skeleton of an own raster graphics class in Java:

class RasterImage {
protected int imageWidth;
protected int imageHeight;
protected int[] imageData;

public RasterImage(int width, int height) { 
  imageData = new int[width*height]; ...
}
public int getPixel(int x, int y) { 
  return imageData[y*imageWidth+x];
}
public void setPixel(int pixelValue, int x, int y) { 
  imageData[y*imageWidth+x] = pixelValue;
}
...
};

Skeleton of an own raster graphics class in C++:

template<typename T> class RasterImage {
protected:
  unsigned imageWidth;
  unsigned imageHeight;
  T* imageData;
public:
  RasterImage(unsigned width, unsigned height) { 
    imageData = new T[width*height]; ...
  }
  const T getPixel(unsigned x, unsigned y) const { 
    return imageData[y*imageWidth+x];
  }
  void setPixel(T pixelValue, unsigned x, unsigned y) { 
    imageData[y*imageWidth+x] = pixelValue;
  }
  ...
};

Java BufferedImage

MyPaintPanel extends JPanel {
private BufferedImage img = null;
public void createGUI() {
  try { img = ImageIO.read(new File("./horse.jpg")); } 
  catch (IOException e) { System.out.println("failed to load image"); }
  ...
}
public void paint(Graphics g) {
  super.paint(g); Graphics2D g2d = (Graphics2D) g;
  int oldRgb = img.getRGB(10, 100); // read a pixel
  int rgb = 0xFFFF0000; // Alpha-Red-Green-Blue
  img.setRGB(10, 100, rgb); // draw red pixel
  g.drawImage(img, 0, 0, null);
}
}

Source code of the example: MyBufferedImage.java

Advantages of Raster Graphics

  • Well suited for complex scenes with irregular shapes or many colors
  • Memory requirement is independent of scene complexity
  • Direct display on todays screens

Disadvantages of Raster Graphics

  • When displaying very simple objects memory requirements are higher compared to a vector graphic representation
  • At large magnification individual pixels become visible
  • This effect is also called jaggies oder aliasing
worldmapdetailpixel
Detail magnification vector graphics
worldmapdetail
Detail magnification raster graphics
Source: John Harvey, Blank map of the world Equirectangular, Creative Commons License

Creating and Editing of Raster Graphics

There are many free and commercial image editing programs:

Storing Raster Graphics

Popular file formats for raster graphics are:

  • PNG (Portable Network Graphics), lossless compression
  • JPEG, smaller files because of lossy compression
  • TIFF (Tagged Image File Format), typically lossless compression, high color depth (up to 32 bit)
  • TGA (Targa Image File), uncompressed or lossless compression
  • GIF (Graphics Interchange Format), lossless compression, allows animation sequences

A Web Page with Raster Images

  • Here is an example of a simple HTML5 web page with raster images:
<!doctype html>
<html>
<head>
  <meta charset="utf-8">
  <title>How to add an image</title>
</head>
<body>
  <h1>How to add an image</h1>
  <p>This is an image 
  <img src="./horse.jpg" alt="horse"> 
  that is inline with the text</p>
  <hr>
  <p>This is an image 
  <img src="./horse.jpg" alt="horse" style="vertical-align: middle;"> 
  that is inline with the text and vertically centered</p>
  <hr>
  <p>This is an image 
  <img src="./horse.jpg" alt="horse" style="display: block;"> 
  that is displayed as a block (no text at any side)</p>
</body>
</html>

Source code: img.html

Manipulating Pixels in an HTML <canvas> element

  • An example for manipulating pixels in an HTML <canvas> element
<!doctype html>

<html>

<head>
  <meta charset="utf-8">
  <title>An example for manipulating pixels in an HTML canvas element</title>
  <script>
    function draw() {
      var c = document.getElementById("myCanvas");
      var ctx = c.getContext("2d");
      var width = c.width;
      var height = c.height;
      var image = ctx.getImageData(0, 0, width, height);
      var imageData = image.data;
      for (var x = 0; x < width; x++) {
        for (var y = 0; y < height; y++) {
          if (x < 50 && y < 50) {
            imageData[x * 4 + y * width * 4 + 0] = 255; // red;
            imageData[x * 4 + y * width * 4 + 1] = 255; // green;
            imageData[x * 4 + y * width * 4 + 2] = 0;   // blue;
            imageData[x * 4 + y * width * 4 + 3] = 255; // alpha;
          } else {
            imageData[x * 4 + y * width * 4 + 0] = 0;   // red;
            imageData[x * 4 + y * width * 4 + 1] = 0;   // green;
            imageData[x * 4 + y * width * 4 + 2] = 0;   // blue;
            imageData[x * 4 + y * width * 4 + 3] = 255; // alpha;
          }
        }
      }
      ctx.putImageData(image, 0, 0);
    }
  </script>
</head>


<body onload="draw();">
  <h1>An example for manipulating pixels in an HTML canvas element</h1>
  <p>This is some text</p>
  <canvas id="myCanvas" width="100" height="100">
    Your browser does not support the HTML5 canvas tag.
  </canvas>
</body>

</html>

Source code: canvas_pixels.html

To be continued ...

  • Now, we have already discussed text and images as components of multimedia documents
  • We continue with an introduction to audio and video in
    Part 1, Chapter 2 of the lecture

Are there any questions?

questions

Please notify me by e-mail if you have questions, suggestions for improvement, or found typos: Contact

More lecture slides

Slides in German (Folien auf Deutsch)

Legal Notice, Privacy Policy,