OpenGL ES: Rendering Pipeline
Examples are written for Android; other platforms will have something similar.
OpenGL ES: Introduction
OpenGL ES is an API for 3D graphics created by Khronos Group for the C language, and it provides detailed information on how to implement this API. Basically, this information refers to how to draw pixels and lines. Device manufacturers use this information to create their own graphics processor (GPU).
OpenGL ES versions 1.0 and 1.1 use a fixed-function pipeline programming model, which means that it is not possible to include programs in these pipelines. But starting with OpenGL 2.0, Vertex and Fragment stages of the pipeline are programmable. Shaders are applied in these stages (small programs written in GLSL) that allow to describe streaming rendering elements, and for longer and more complex algorithms to be implemented using a single rendering pass.
All OpenGL ES methods must be called from the same thread, otherwise unexpected errors may occur because multithreading is not supported.
For mobile devices, when the application is in pause state, the OpenGL ES context may be lost, so you have to be careful not to lose any information. For example, textures that are associated with that context may be automatically deleted, so they must be recreated in an Activity's resume method.
Frame Buffer
Any display is based on a raster that is a two-dimensional pixel grid. Based on this information, the display updates its state several tens of times per second (this is called the refresh rate and is measured by Hertz, for example 60 Hertz in 1 second).
The graphics processor (GPU) has access to a memory area called VRAM. VRAM has a special memory space that stores color information (RGB(A)) for each pixel in the raster. This area of memory is called the frame buffer, and the entire image displayed is called the frame. Any pixel of the raster corresponds to a specific memory address of the frame buffer and vice versa. There is a one-to-one correspondence between the raster and the frame buffer.
The address of the pixel whose coordinates are x and y in the raster can be calculated using the following formula:
int address = x + y * rasterWidth;
Each raster pixel has two characteristics: position (x and y) and color. The color is determined by the following three components: R (red), G (green), B (blue) and may have the fourth component, A (alpha), which defines the transparency of the pixel. The values of these components are 'float' type in the range [0, 1].
RGB color values 0-255 correspond to the values 0-1 in OpenGL ES.
For example: RGB (255, 153, 0) is equal to RGB (1, 0.6, 0) in OpenGL ES, where 1 = 255 /255, 0.6 = 153 /255, 0 = 0 /255.
The following OpenGL ES methods are called to clean the Frame Buffer:
GLES20.glClearColor (float red, float green, float blue, float alpha;
GLES20.glClear (int mask); mask = GLES20.GL_COLOR_BUFFER_BIT,
GLES20.GL_COLOR_BUFFER_BIT indicates that you are referring to the frame buffer. OpenGL ES also has other buffers.
Alpha Component
Transparency only plays a role in blending the information of two pixels. In order for OpenGL ES to blend pixel information during drawing, you must first enable it. To do this, you must call the following methods before you call the draw method:
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA;
The formula corresponding to GLES20.GL_ONE_MINUS_SRC_ALPHA :
red = src.red * src.alpha + dest.red * (1 — src.alpha)
green = src.green * src.alpha + dest.green * (1 — src.alpha)
blue = src.blue * src.alpha + dest.blue * (1 — src.alpha)
Where src is the new information about the pixel and
dest is the old information about the pixel.
After calling the first method, each time you call the draw method to write new information to the frame buffer, the old information is not replaced with new information, but mixed with it. The second method determines which formula is used for blending.
src = (1, 0.6, 0),
src.alpha = 0.5,
dest = (0, 0, 0)
red = 1 * 0.5 + 0 * (1 — 0.5) = 0.5
green = 0.6 * 0.5 + 0 * (1 — 0.5) = 0.3
blue = 0 * 0.5 + 0 * (1 — 0.5) = 0
result = (0.5, 0.3, 0)
To disable blending, the following method is called:
GLES20.glDisable(GLES20.GL_BLEND);
Projection in Computer Graphics
There are two types of projection in Computer Graphics: Orthographic projection (for 2D graphics) and Perspective projection (for 3D graphics).
Perspective projection is the projection of points of a three-dimensional object by straight lines drawn through them from some given point (Camera) to an intersection with the plane of projection (Near clip plane), as a result of which the three-dimensional object is represented by the two-dimensional projection of points onto the plane of projection.
Orthographic Projection is the projection of points of a three-dimensional object which are constructed by extending perpendiculars from points on the object to the plane of projection (Near clip plane), as a result of which the three-dimensional object is represented by the two-dimensional projection of points onto the plane of projection.
Rendering Geometry
OpenGL ES works mainly with triangles to render a object. Any three-dimensional object can be represented with triangle mesh.
As the number of triangles in the mesh increases, the representation of object becomes more accurate, but this affects performance.
In order for OpenGL ES to render a triangle on the raster, it must know the corresponding addresses of the triangle vertices in the frame buffer. Therefore, you must first define a mapping area that is part of the object space whose points will be displayed on the raster and the points that do not belong to it will be ignored. The mapping area is determined by a Near clip plane and by a Far clip plane.
In the case of 2D graphics it is a rectangular parallelepiped, which can be determined by the following method:
Matrix.ortho(float[] m, int mOffset, float left, float right, float bottom, float top, float near, float far);
and truncated cone in 3D graphics, defined by the method:
Matrix.frustumM(float[] m, int offset, float left, float right, float bottom, float top, float near, float far);
OpenGL ES treats the origin of the three-dimensional spatial coordinate system as the bottom-left corner of the display (raster).
When you specify the vertex coordinates in the object space, OpenGL ES first finds the vertex projection on the Near clip plane, then finds coordinates of the pixel in the raster that corresponds to this projection (there is a mutual correspondence between raster and Near clip plane), and finally the address in the frame buffer that corresponds to the coordinates of this pixel.
You can also create a mutual correspondence not to the whole raster, but to some part of it and Near clip plane. This is done by the following method:
GLES20.glViewport (int x, int y, int width, int height);
where x and y are the coordinates of the top left corner of the raster in its 2D pixel grid, and the width and height dimensions are expressed in pixels.
Rendering
Textures in OpenGL are 2D grid of elements known as texels, which contain colour and alpha values. In OpenGL, textures are applied to triangles and drawn on the screen.
The vertices of a triangle, in addition to the object coordinates, also have texture coordinates that are used to map a texture (image) to a triangle. The texture coordinates of the vertex determine the texel on the texture to which the vertex should be mapped. The texture coordinates are 2-dimensional. The object coordinates of a vertex are usually denoted by the letters x, y, z and the texture coordinates u, v, or s, t. Regardless of the image size, the left bottom corner of the texture has texture coordinates (0.0) and the top right corner (1.1). Object coordinates and the texture coordinates of the vertices of triangle are selected independently from each other.
To display a triangle, it is first projected onto Near clip plane using the object coordinates of the vertices. Then, during the rasterization, the projection points are selected, which should be stored in the frame buffer aligned in a 2D grid of a certain rows and columns corresponding to the display's resolution. For each selected projection point, the corresponding frame buffer address is calculated based on the presence of a mutual correspondence between the raster and Near clip plane. Finally, information about the color is stored at this address. The color is determined by the texture coordinates of the triangle point which projection has been selected.
Texture Filters
Since the triangle projection stored in the frame buffer can occupy more pixels on the raster than the number of texels corresponding to the triangle (this is called magnification), or vice versa (this is called minification), you need to tell OpenGL ES how it should resize the texture. Filters control the magnification and compression of the texture. Filtering is not applied in the case of a one-to-one correspondence.
Filters are defined using the method:
GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, property, value);
property can be:
- GLES20.GL_TEXTURE_MAG_FILTER (texture magnified, enlarged)
- GLES20.GL_TEXTURE_MIN_FILTER (texture minimized, shrinked)
value can be :
- GLES20.GL_NEAREST,
- GLES20.GL_LINEAR.
GL_NEAREST type always selects the closest texel in the texture zone that will be associated with the pixel from the raster of frame buffer. This is the main and fastest rendering filter that causes rendering to be "pixelated".
GL_LINEAR type of filter will use four closest texels to associate with the pixel from the raster, and then find their average value for the final color (linear interpolation of the color between the four closest texels). This type has a better rendering than GL_NEAREST, especially for close-up view. But it is slower, due to the interpolation. The 'pixelized' rendering disappear due to the linear interpolation. The result of this filter is similar to a 'blur' effect.
You apply a filter of the first type if you need a mosaic image, and a filter of the second type if you want a smoother version.
