traditional graphics pipeline

COS 350 - Computer Graphics

graphics pipeline

• sequence of operations to generate an image using object-order processing
  • primitives processed one-at-a-time
  • software pipeline: e.g. Renderman (old)
    • high-quality and efficiency for large scenes
  • hardware pipeline: e.g. graphics accelerators
    • lower-quality solution for interactive applications
• will cover algorithms of modern hardware pipeline (pre RTX)
  • but evolve drastically every few years
  • we will only look at triangles

graphics pipeline

• handles only simple primitives by design
  • point, lines, triangles, quads (as two triangles)
  • efficient algorithm
• complex primitives by tessellation
  • complex curves: tessellate into line strips
  • complex surfaces: tessellate into triangle meshes
• "pipeline" name derives from architecture design
  • sequences of stages with defined input/output
  • easy-to-optimize, modular design

graphics pipeline

• object-local algorithm
  • processes only one-surface-at-a-time
• various effects have to be approximated
  • shadows: shadow volume and shadow maps
  • reflections: environment mapping or screen-space
  • hard to implement
• advanced effects cannot be implemented without hacks
  • soft shadows
  • blurry reflections and diffuse-indirect illumination

graphics pipeline stages

only a subset of modern pipeline stages (tessellation, geometry, vertex post-process, primitive assemly)

graphics pipeline stages

• vertex processing
  • input: vertex data (position, normal, color, etc.)
  • output: transformed vertices in homogeneous canonical view-volume, colors, etc.
  • applies transformation from object-space to clip-space
  • passes along material and shading data
• clipping and rasterization
  • turns sets of vertices into primitives and fills them in
  • output: set of fragments with interpolated data

graphics pipeline stages

• fragment processing
  • output: final color and depth
  • traditionally mostly for texture lookups
    • lighting was computed for each vertex
  • today, computes lighting per-pixel
• framebuffer processing
  • output: final picture
  • hidden surface elimination
  • compositing via alpha-blending

vertex processing

vertex processing

• transform vertices from model to clip space

vertex processing

• other geometry tasks
  • deformation: skinning, mesh blending
  • low-quality lighting
  • pass other properties to next stages of pipeline
  • the only place to algorithmically alter shape
• programmable hardware unit
  • algorithm can be changed at run-time by application

clipping and rasterization

clipping and rasterization

• remove (partial) objects not in the view frustum
  • efficiency: cull later stages of the pipeline
  • correctness: perspective transform can cause trouble
  • often referred as culling when full objects removed

clipping to ensure correctness

in front of eye	behind eye

point clipping

• point-plane clipping
  • test if the point is on the right side of the plane
  • by taking dot-product with the plane normal
  • can be performed in homogeneous coordinates

• point-frustum clipping
  • point-plane clipping for each frustum plane

line clipping

• segment-plane clipping
  • test point-plane clipping for endpoints
  • if endpoints are clipped, clip whole segment
  • if endpoints are not clipped, accept whole segment
  • if one endpoint is clipped, clip segment
    • compute segment-plane intersection
    • create shorter segment

line clipping

• segment-frustum clipping
  • clip against each plane incrementally
  • guarantee to create the correct segment

• more efficient algorithms available
  • previous incremental approach might try too hard
  • provide early rejection for common cases
  • so, only clip when necessary

polygon clipping

• convex polygons similar to line clipping
  • clip each point in sequence
    • remove outside points
    • create new points on boundary
  • clipped triangles are not necessarily triangles

culling

• further optimize by rejecting "useless" triangles
• backface culling
  • if triangle face is oriented away from camera, cull it
  • only ok for closed surfaces
• early z-culling
  • if triangle is behind existing scene, cull it
  • uses z-buffer introduced later on

viewport transformation

• transform the canonical view volume to the pixel coordinates of the screen
• also rescale \(z\) in the \([0...1]\) range
  • we will see later why
• perspective divide is often performed here

rasterization

• approximate primitives into pixels
  • pixel centered at integer coordinates
• determine which pixels to turn on
  • no anti-aliasing (jaggies): pixel in the primitive
  • consider anti-aliasing for some primitives
  • input: vertex position in homogeneous coordinates
• interpolate values across primitive
  • color, normals, position at vertices
  • input: any vertex property

See S10_Algorithms for more details on rasterization

fragment processing

fragment processing

• compute final fragment colors, alphas, and depth
  • depth is often untouched if no special effects
  • final lighting computations
  • lots of texture mapping: see later
• programmable hardware unit
  • algorithm can be changed at run-time by application

lighting computation

• where to evaluate lighting?
  • flat: at vertices but do not interpolate colors
  • Gouraud: at vertices, with interpolated color
  • Phong: at fragments, with interpolated normals

lighting computation - flat shading

• compute using normals of the triangle
  • same as in raytracing
• flat and faceted look
• correct: no geometrical inconsistency

lighting computation - Gouraud shading

• compute light at vertex position
  • with vertex normals
• interpolate colors linearly over the triangle

lighting computation - Phong shading

• interpolate normals per-pixels: shading normals
• compute lighting for each pixel
  • lighting depends less on tessellation

lighting computation comparison

Gouraud	Phong
artifacts in highlights	good highlights

lighting computation

• per-pixel lighting is becoming ubiquitous
  • much more robust
  • move lighting from vertex to fragment processing
    • new hardware architectures allow for this
    • we introduce Gouraud for historical reasons
  • raytracing can have this by using shading normals

lighting computation

• shading normals introduce inconsistencies
  • lights can come from "below" the surface

framebuffer processing

framebuffer processing

• hidden surface elimination
  • decides which surfaces are visible
• framebuffer blending
  • composite transparent surfaces if necessary

hidden surface removal - painter alg.

• sort objects back to front
• draw in sorted order
• does not work in many cases

hidden surface removal - painter alg.

• sort objects back to front
• draw in sorted order
• does not work in many cases

hidden surface removal - z buffer

• brute force algorithm
• for each pixel, keep distance to closest object
• for each object, rasterize updating pixels if distance is closer
  • opaque objects: works in every case
  • transparent objects: cannot properly composite

hidden surface removal - z buffer

z-buffer
color buffer

hidden surface removal - z buffer

z-buffer
color buffer

which z distance

• use z value after homogeneous xform
  • linear interpolation works
  • storage non-linear: more precision around near frame

which z distance

• use z value after homogeneous xform
  • linear interpolation works
  • storage non-linear: more precision around near frame

hidden surface removal - raycasting

• for each ray, find intersection to closest surface
  • works for opaque and transparent objects
• loops over pixels and then over surfaces
  • inefficient
  • would like to loop over surfaces only once

hidden surface removal - scanline

• for each scanline, sort primitives
  • incremental rasterization
  • sorting can be done in many ways
  • needs complex data structures
  • works for opaque and transparent objects

hidden surface removal - REYES

• for each primitives, turn into small grids of quads
• hit-test quads by ray-casting
• keep list of sorted hit-points per pixel
  • like z-buffer but uses a list
  • works for opaque and transparent objects
• hybrid between raycast and z-buffer
  • very efficient for high complexity
    • when using appropriate data-structures
  • solves many other problems we will encounter later

framebuffer processing

• hidden surface elimination using Z-buffer
• framebuffer blending using \(\alpha\)-compositing
  • but cannot sort fragments properly
  • incorrect transparency blending
  • need to presort transparent surfaces only
    • like painter's algorithm, so not correct in many cases

traditional graphics pipeline

why?

why graphics pipelines?

• simple algorithms can be mapped to hardware
• high performance using on-chip parallel execution
  • highly parallel algorithms
  • memory access tends to be coherent
  • one object at a time

traditional graphics pipeline architecture

• multiple arithmetic units
  • NVidia Geforce GTX Titan: 2688 stream processors
• very small caches
  • not needed since memory accesses are very coherent
• fast memory architecture
  • needed for color/z-buffer traffic
• restricted memory access patterns
  • no read-modify-write
  • bound to change hopefully
• easy to make fast: this is what Intel would love!
• research into using for scientific computing

graphics pipelines vs. raytracing

what about RTX?

• the relatively new RTX architecture changes Everything!^†
• adds raytrace abilities into graphics pipeline
  • true reflections, refractions, shadows
  • indirect lighting and ambient occlusion
  • lens and motion blur
• considered a hybrid image render approach
• still very limited and requires some "black magic"
  • raytraced image contains very low samples and requires denoising filters
  • requires acceleration structures

† kind of