Computer graphics: how computers make images

Explore the representations and algorithms behind digital images—from pixels and color to geometric transformations, rasterization, lighting, and animation.

Pixels and image representations

A digital image can be represented as a grid of pixels, each storing one or more values. A raster image with width w and height h contains w × h pixels. A 1920-by-1080 image has more than two million pixels to store or process.

Raster formats are useful for photographs and detailed images, but scaling them up can reveal blocky pixel edges. Vector graphics describe shapes, paths, and text mathematically, allowing them to be rendered at different sizes. Vector and raster representations serve different needs; vector content is often rasterized when displayed.

Resolution and memory

For an uncompressed image with four 8-bit channels per pixel, storage is roughly width × height × 4 bytes, before metadata and row padding. A 1000-by-1000 image would therefore require about four million bytes in this simplified representation. Compression can reduce file size, but compressed files must be decoded before many pixel operations.

More pixels can capture finer detail, but they consume memory, storage, and processing time. Resolution should match the viewing size and task. Upscaling a small raster image cannot recreate detail that was never captured, although interpolation can make it appear smoother.

Color, channels, and transparency

A common display representation stores red, green, and blue intensity values (RGB). If each channel has 8 bits, each has 256 possible values, giving about 16.7 million RGB combinations. The mapping from numeric values to displayed light depends on color spaces, transfer functions, and the display.

Images may use different color models. CMYK is used in many print workflows; grayscale stores intensity; indexed color maps pixel values to a palette. Converting between color spaces can change appearance if gamut and profiles are ignored.

Alpha and compositing

An alpha channel represents opacity or coverage. Combining a translucent foreground color with a background requires a compositing rule. With premultiplied alpha, color channels are multiplied by alpha before storage, which can simplify filtering and blending. Mixing premultiplied and straight-alpha data incorrectly can create dark or bright fringes around edges.

Transparency is not a property every image format or display path preserves. An image editor may show a checkerboard to represent transparent pixels; the checkerboard is not part of the image itself.

Coordinates and transformations

Graphics uses coordinate systems to locate points and describe objects. A 2D screen coordinate often places the origin at the top-left, with x increasing right and y increasing down. A mathematical coordinate system may use y increasing upward. Converting between them requires a transformation.

Common 2D transformations include translation, rotation, scaling, and shear. Homogeneous coordinates represent these transformations with matrices, allowing several operations to be combined through matrix multiplication. Order matters: rotating then translating generally differs from translating then rotating.

point = (x, y)
scaled = (sx * x, sy * y)
translated = (scaled.x + tx, scaled.y + ty)

This simple sequence scales a point relative to the origin and then moves it. To scale around an object's center, translate the center to the origin, scale, and translate back. Understanding the chosen coordinate space prevents common mistakes in positioning and animation.

World, view, and screen space

A 3D scene commonly uses world space for object locations, view space relative to a camera, and clip or screen space for projection to the display. Each transformation changes how coordinates are interpreted. A point may be behind the camera or outside the view and therefore not appear in the final image.

From shapes to pixels: rasterization

A display is a grid, but geometric shapes can cross pixel boundaries. Rasterization determines which pixels are covered by a shape and what color each receives. A line or triangle is not always aligned with whole pixel rows, so simple binary coverage can create jagged edges called aliasing.

Antialiasing estimates partial coverage or samples more points to make edges appear smoother. This improves visual quality but costs computation and cannot restore detail missing from the original geometry. Different techniques trade image quality, performance, and memory.

Visibility and depth

In a 3D scene, several surfaces may project to the same pixel. A depth buffer stores distance information so the renderer can choose the nearest visible surface. Transparent surfaces are more complicated because they can reveal content behind them and often need sorting or specialized blending.

Rasterization is efficient and common in real-time graphics, but it represents geometry as fragments on a grid. The order of drawing and the state used for each fragment can affect the result, so graphics programs explicitly manage buffers, depth tests, and blending rules.

The 3D graphics pipeline

A real-time 3D pipeline transforms objects into screen-space fragments and colors them. A simplified pipeline looks like this:

1. Modeling: describe objects with vertices, surfaces, and attributes.
2. Vertex processing: transform positions and prepare per-vertex data.
3. Primitive assembly: group vertices into triangles, lines, or other primitives.
4. Rasterization: determine which screen fragments each primitive covers.
5. Fragment processing: compute candidate colors using materials, textures, and lighting.
6. Output: apply depth and blending rules and write to a frame buffer.

Modern graphics APIs expose programmable stages through shaders. A vertex shader can transform positions; a fragment shader can calculate colors. GPU hardware processes many vertices or fragments in parallel, but data movement, overdraw, memory bandwidth, and synchronization can still limit performance.

Graphics APIs differ and evolve. The pipeline is a conceptual model; actual implementations may add stages or perform work in different ways. Start with a simple scene and validate each coordinate transform before adding complex effects.

Lighting, shading, and materials

Shading estimates how a surface should appear under light. A simple model may combine ambient light, diffuse reflection, and specular highlights. Surface normals describe orientation; the angle between a normal and a light direction affects diffuse brightness.

A material describes visual properties such as color, roughness, reflectivity, or transparency. These are models, not literal physical objects. A physically based rendering workflow tries to keep material and lighting parameters consistent with measured behavior, though approximations remain.

Shadows require determining whether light can reach a surface. Reflection and refraction require modeling light paths or approximating them. More realistic effects typically cost more computation, so real-time applications use simplified techniques, precomputed data, or selective detail.

Linear color and display encoding

Many color values in image files are encoded nonlinearly to better use limited bit precision. Lighting calculations are generally meaningful in linear-light space. A graphics pipeline that performs blending or shading in the wrong color space can produce inaccurate brightness and edges. Color management helps ensure data is interpreted and displayed consistently.

Textures, filtering, and sampling

A texture is an image or data map applied to geometry. Texture coordinates associate points on a surface with positions in the texture. A renderer samples the texture to determine color, normal, or another material attribute at a fragment.

When a textured object is far away, many texture pixels may contribute to one screen pixel. When it is close, one texture pixel may cover many screen pixels. Filtering estimates appropriate values across scales. Mipmaps store progressively smaller versions of a texture to reduce aliasing and improve sampling efficiency at a distance.

Sampling artifacts can appear as shimmering, moiré patterns, or blurry detail. Filtering, antialiasing, and adequate texture resolution help, but each technique has costs. Use image quality tests at the actual viewing sizes and motion conditions where the graphics will be used.

Animation, frame rate, and interaction

Animation updates a scene over time and renders a sequence of frames. A frame rate of 60 frames per second allows about 16.7 milliseconds per frame, including application logic, graphics work, and presentation. Missing a frame deadline can cause visible stutter even if the average frame rate seems high.

Use elapsed time rather than assuming every frame takes exactly the same duration. If movement advances a fixed distance per frame, the animation speed changes when the frame rate changes. A common approach updates positions based on delta time, while fixed-step simulation can improve consistency for physics.

Animation should respect user preferences such as reduced motion. Rapid movement or large parallax effects can cause discomfort. Provide controls, avoid unnecessary motion, and ensure important information remains available without animation.

Rendering trade-offs and uses

Rendering creates an image from scene data. Real-time rendering prioritizes producing frames quickly enough for interaction, as in games and visualization. Offline rendering can spend much longer per frame to improve realism, as in film production and scientific imagery.

Ray tracing follows simulated light paths to model reflection, refraction, and shadows. It can produce realistic effects but may require substantial computation. Rasterization is usually faster for interactive scenes; hybrid methods combine both approaches.

Graphics also supports more than entertainment: medical imaging, maps, data visualization, computer-aided design, and scientific simulation all rely on visual representations. A visualization can mislead if scales, color, perspective, or uncertainty are represented poorly. Preserve the relationship between the image and the underlying data.

A compact glossary

Pixel

A sample in a raster image, usually storing color or other channel values.

Rasterization

Converting geometric primitives into fragments or pixels.

Shader

A programmable stage that processes graphics data such as vertices or fragments.

Texture

An image or data map sampled to determine surface properties.

Aliasing

Visual artifacts caused by representing continuous shapes with discrete samples.

Frame buffer

Memory holding image data prepared for display.

References and further reading

1. Peter Shirley et al. Fundamentals of Computer Graphics, 4th edition. CRC Press, 2015. Publisher information.
2. Scratchapixel. Computer Graphics and Rendering Tutorials. Free lessons.
3. Khronos Group. OpenGL Wiki. Graphics programming reference.
4. WebGL Fundamentals. Interactive graphics tutorials.

Try it yourself

1. Estimate the uncompressed memory needed for a 1280-by-720 image with four 8-bit channels.
2. What sequence of translations and scaling would enlarge a shape around its center?
3. Explain why a depth buffer is useful in a 3D scene.
4. Describe one visual artifact caused by undersampling and a technique that can reduce it.
5. Design a simple frame-time budget for an interactive animation targeting 30 frames per second.

Remember: graphics turns mathematical descriptions and sampled data into images, balancing correctness, quality, and computation.