This page is an overview of some of the fundamentals in Graphics Programming. While there are some code samples, this section doesn't go through the specific C++ Code and OpenGL API calls used to access the Render Pipeline, and is just meant to be a conceptual overview of the project. To see the actual code, please checkout the project on GitHub (CSC4900 Graphics Engine Building )
There are a lot of technical elements to graphics programming, especially when it comes to 3D Math. Before getting into the details, ensure that you have a basic understanding of linear algebra, and working with vectors, matrices, and matrix transformation. You don't need to know exactly how the underlying math works, but just know the basics of how to use it. I'm going to assume for the sake of space that you have a basic understanding of these topics, though I will go over some topics in future sections.
You will need a good grasp of programming fundamentals. While I don't go into the specifics of OpenGL code here, the engine is built in C++, and any sample code in this section uses C-like syntax. Having a good grasp of C++ and the fundamentals of computer memory and memory management will be useful in better understanding the sections below.
When rendering a 3D object, we create what are known as meshes. Meshes are a collection of vertices (3D positions) that we wrap into triangles.
Here's a representation of a square mesh, also known as a quad:
In this example, the square is made up of 4 vertices and 2 triangles.
For convenience, I labled the vertices A,B,C,D.
Let's say that C is the origin of our graph such that:
A = (0,1,0), B = (1,1,0), C = (0,0,0) and D = (1,0,0)
There are 2 ways of representing these meshes in memory. The first is relatvely simple, just store an array of vertices, with every 3 vertices making up a triangle.
vec3[] vertices = {
A, B, D, // The first triangle
A, D, C // The second triangle
}
However, you'll notice that using this method, we end up storing some vertices twice,
and when you start creating larger 3D meshes, this problem get's exponentially worse.
It would be better to store each vertex only once. The second way of storing a mesh solves
this problem.
In the second method, we use 2 arrays. The first array is a list of all the vertices in the mesh,
the second is a list of indices that represent the triangles of the array.
vec3[] vertices = {
A, B, C, D // A list of the vertices with no duplicates
}
int[] triangles = {
1, 2, 4, // The first triangle
1, 4, 3 // The second triangle
}
One thing to note when creating 3D meshes, the direction in which you draw the triangles is important.
All triangles must be wrapped in a clockwise direction, otherwise they will render inside out.
The direction of the triangle is based on the order you store it in.
So, { A, D, B } is backwards since it is wrapped counter-clockwise,
instead, you want to store it as { A, B, D }.
Also, as long as they are wrapped clockwise, the order doesn't matter,
so { A, B, D } is equivilant to { D, A, B } since they are wrapped in the same directions.
If you've taken a course in Linear Algebra, you've probably heard of matrix transformation. If you are
like me, however, you probably wondered how these transformations could be useful. In graphics programming
matrix transformations are absolutely essential to everything we do.
In this section, I am not going to go over how matrices or matrix transformations work. Instead, I will
simply explain how we use them in when writing code. If you'd like to see a bit more of the math behind it,
check out Learn OpenGL. He goes over the
basics of transformations and how they work. For a more in-depth look into matrix math, there are plenty of
resources and tutorials on the internet.
There are 3 core matrices used in rendering: the model matrix, view matrix, and projection matrix. Each of these
matrices transforms points from one space to another, allowing graphics programmers to easily swap between different
spaces. The model matrix transforms from
local space to world space (Local Space refers to the mesh objects vertices origin. World Space refers to
what the Graphics Programmer defines as the scenes actual space, based around the origin). The
view matrix converts from world space to view space (View space refers to where the space about the camera's
position in the world). The projection matrix converts the view space into clip space (Clip space
is what the camera can actually see in the world). Finally, after all that is done, one more autmatic transformation
is performed to convert clip space into actual screen space.
Here's a good diagram of this
By using these transformations, we can now create 3D meshes in their own "model" space, and easily convert each of it's vertices to world
space. We can also take each of these vertices, and then convert them into coordinates that fit within the viewport of
the users device. With these transformations, we now have the mathematical tools that we need in order to turn a 3D object into
a 2D image on a screen.
In order to render objects using the GPU, we first have to get the mesh data
to the GPU. In OpenGL, we do this using buffers. A GPU buffer is simply an array of bytes
stored in the GPU memory for use in shader programs. Buffers can be used to store
all sorts of data, but generally, we use it for storing vertex information.
This vertex info is going to be stored in an array of values on the GPU. The
most basic thing you need to store is just the position data of each vertex.
In the buffer below, you simply have a an array of floats, where every 3 floats
represents a vertex.
However, when rendering a mesh, you often need more than just the vertex positions
but sometimes want to store other information as well.
For this reason, when we store vertices on the GPU, we also have the ability to store other
data with each vertex. This data could be a color, a texture position (also known as a UV), a normal value (important
for lighting), or any other data that can be represented as primitve data types with static size.
Here's an example of the array when storing more data:
When creating these arrays, it's important to keep track of the stride
of your vertex data. The stride is the distance from the start of one vertex to
the start of the next vertex. Since you might store different types of data (like floats
or ints), the stride is simply the count of bytes your vertex takes up. In the example
above, the stride is 9 * sizeof(float) or 36 bytes (since a float is 4 bytes).
Now the question is, how do we access and use this vertex data? In the next section we will cover
how to create shaders for running programs on the GPU itself.
Now that we have our Mesh, and we've moved it to the GPU, we have to actually render it
using the GPU. Shaders are programs stored on the GPU that can be run in parallel for large
sets of data. This is what makes the GPU so efficient for rendering items on the screen. We could have
thousands of vertices on the screen at a any given time. If we were to render each of these vertices
one at it time, it would take forever to render even a single frame, let alone 60 frames each second.
The GPU, however, has potentially thousands of cores that can all run in parallel. However, while it has a
lot of cores, each of the cores is significantly smaller than a typical CPU core. As a result, programs
run on the GPU are limited in size, and should primarily be used for performin mathmatical operations with as
few branches as possible.
There are six stages to the render pipeline that all run in a linear sequence:
For most graphics programmers, we only really care about the Vertex Shader and the Fragment Shader
stages of the pipeline, everything else is done automatically for us (though occasionally we may modify
the geometry shader, but I won't be going over that).
Before we get into the specifics of the Vertex and Fragment Shader, lets go over a few general concepts
in he shader world: input, output, and uniforms.
Inputs and Outputs:
Every stage of the shader process has a series of inputs and outputs. These inputs are the outputs created
after the previous stage of the render pipeline completed. In general, you can think of the render pipeline
as being a conveyer with a bunch of machines that moves the data along from one machine to another.
Each of these machines takes in the data and transform it, until at the very end you have an image that
can be drawn to the screen. So, the input of one machine is the output of another.
Uniforms:
When writing shaders, there is often some data that will be the same for every vertex in a mesh.
These are called uniforms. Uniforms can be used for all sorts of things. For example, you may want to
color your whole object a single color. Rather than store a color value on every vertex, you can store one
uniform color, and use that on each vertex in the shader. Uniforms are used all over the place, because
often an object, while made up of many vertices, has object-wide properties that we don't want to store
in every vertex.
The Vertex Shader is the first stage of the render pipeline and run over every vertex in your mesh.
Since the vertex shader is the first stage of the pipeline it's input is provided from the CPU in the
form of a vertex buffer. The primary output of the vertex shader is the screen-view position of the
vertex (the normalized (x,y,z) position in relation to the OpenGL window also known as normalized
device coordinates).
To achieve this output, we use the the matrix transformations above. However, all of these Matrices are
used for transformations are kept on the CPU. This is where Uniforms come in handy. Before running the vertex
shader, I make sure to create a Uniform for each of the matrices (model, view, and projection). Then, I simply
multiply the vertex position by the matrices in the proper order.
Here's the Vertex Shader used for rendering a mesh written in GLSL 3:
#version 330 core
layout (location = 0) in vec3 pos; //Vertex Position Input
layout (location = 1) in vec2 uv0; //Vertex Texture Position Attribute
out vec2 texCoord; //Output Texture Coordinate to fragment shader
uniform mat4 model; //The mesh model Matrix
uniform mat4 view; //The scene view matrix
uniform mat4 projection; //The scene projection matrix
void main()
{
//Set the texure Texture Coordinate output
texCoord = vec2(uv0.x, uv0.y);
//Set the default gl_position output
gl_Position = projection * view * model * vec4(pos, 1.0f);
}
As you can see, the shader looks a lot like a simple C program. The key difference is the variables declared at
the top are inputs and outputs. The 'pos' and 'uv0' are the vertex attribute input given. Inputs are defined
using the 'in' keyword. The 'layout' keyword is used to tell the shader at what position in the vertex the input is
located at. After the input comes the use defined output, which in this case is the texture coordinate (the 2D position
in the texture that the vertex should display). There are also a few default ouputs, like gl_position is a that don't
have to be declared. Finally, the Uniforms are declared, which in this case are the 3 transformation matrices we will
be using to get the screen position of the vertex (gl_position).
After finishing in the Vertex Shader, the outputs are passed into the fragment shader.
The Fragment shader is one of the last stages of the render pipeline. The fragment shader is run for every fragment on the
OpenGL window. For simplicity, you can think of these fragments as being the individual pixels on the screen (though sometimes
they are different sizes). The primary output of the fragment shader is the color that the current fragment should be rendered
as using an RGBA (red,green,blue,alpha) color encoding. We represent this color as a 4D Vector, where each position is a value
between 0 and 1.
Here's the Fragment shader associated with the Vertex Shader above:
#version 330 core
in vec2 texCoord; //The texture coordinate given to use by the vertex shader
out vec4 FragColor; //The output color that the fragment will be drawn with
uniform vec4 ourColor; //A Uniform that we use to modify the color of the entire object
uniform sampler2D ourTexture; //The texture to draw on the surface of the object
void main()
{
//Set the color output to the texture color multiplied by object color
FragColor = texture(ourTexture, texCoord) * ourColor;
}
This time, you'll notice I didn't define my inputs using a location. Instead, I just have to make sure my input
name matches the output name of the vertex shader. The compiler will automatically assign the outputs of the
vertex shader to the inputs of the fragment shader. I have only 1 output in the fragment shader, and thats the
fragment Color I want to draw to the screen.
Using these 2 Shader's, we can create a Shader Program and compile it on the GPU. Once we do this, OpenGL
give's us a Shader ID, which we can use to tell the GPU which shader program we want to use when rendering our objects.