3D visualisation using Qt3D: part 1

Introduction

In my company we are developing a machine vision + AI based assistant which can provide visual hints about road obstacles to drivers of various vehicles.

For the visual display of such information we are developing a 3d rendered environment which displays AI-identified obstacles by rendering them into a 3D scene.

For the HMI we have to meet the following requirements:

• High runtime performance
• Easy portability across multiple platforms (Linux, Windows, Android and potentially iOS)
• Easy import of 3D assets in various formats

These requirements has led us to choose Qt3D, a standard module within the Qt framework.

This toolkit satisfies our needs:

• It's written in C++, so performance is top-notch
• Portability across platforms is relatively straightforward
• It supports a lot of open and proprietary standards for asset import

Working with Qt3D has made us face some very interesting challenges, which I will try to disclose in a series of articles. Qt3D is a very powerful technology which is however poorly documented, especially as soon as you try to dive deeper than the surface provided with the examples.

These articles will show the pain & gain achieved through long investigations in missing documentation, little information found on forums, and reverse engineering in Qt3D source code.

Our initial objective

In this first incarnation of our example we will write a simple software which:

• Creates a Qt3D window
• Loads a 3d object into it
• Creates a camera and a light
• Allow the user to manipulate the camera with the mouse (using a camera manipulator), thus making it possible to view the 3d object from different points of view

This is what we can expect from the code we will go through:

Creating a simple 3D model

In this series of examples we will use some simple models created with blender. These are not intended to be Blender tutorials though (I wouldn't be able to do anything of the kind anyway). We use Blender as a basic tool to demonstrate what we want to achieve in C++.

So, in Blender we creare a simple scene starting from the default one. We delete the cube and we add a torus, renaming the torus to TorusObject and its mesh to TorusMesh. We also add an azure texture to the torus:

Objects

Looking at the outliner (top-right area of the Blender screen) we see what we are creating:

The outliner shows a tree with a root, "Collection", under which we can find 3 objects:

• A camera
• A light
• A torus called "TorusObject", under which we have a mesh called "TorusMesh" which in turn contains a material called "Material.001"

Transforms

Each object has an associated Transform, which defines its position, orientation and size:

The numbers above define the position and orientation of the camera in the Blender scene:

The camera is represented above by an orange pyramid. The orange filled triangle indicates which is the "up" direction of the camera.

Geometry

Under "TorusObject" we have two more datablocks, TorusMesh and further below "Material.001"

TorusMesh is the name of the polygon mesh used to represent the object's geometry, which can be seen and modified using Blender's "Edit Mode":

The object is represented by a mesh of polygons which can normally be triangles or quadrangles. The mesh contains vertices (the orange dots), edges (lines connecting two vertices) and faces (polygons defined by edges)

In the following picture we see a vertex being selected (white dot):

Here we see an edge being selected (white line) :

And finally a face being selected (orange quadrangle):

Please note that vertices include geometrical information, i.e. they have an XYZ position in space.

On the contrary, edges and faces only include topological information: an edge is defined by which vertices it's connecting, and a face is defined by which edges surround it.

This basic information will be useful, much further ahead in this series of tutorials, to understand the internal representation used by Qt3D to store geometry, and how this can be manipulated in real time.

Materials

Under TorusMesh we can see a data block called "Material.001". Materials define the visual appearance of meshes, with information such as surface colour, shininess, transparency, and in general any kind of visual information about the surface.

Here is a screenshot showing some properties for Material.001

Saving and exporting

Once we are done with creating our model, we can save it. I save it as object1.blend (.blend is the extension for Blender models).

Although Qt3D can theoretically import Blender models out of the box, it fails to do so (at least with all the models I tried).

After experimenting with various combinations of Blender export -> Qt3D import combinations, I found that the most robust solution is Collada. It works well for meshes, materials and textures (more about textures in later articles). Other export/import combinations fail rather miserably in material management and texture management.

So we export the object as object1.dae (.dae is the extension for Collada files).

Finally some coding

Before we dig into details, we need to understand a few fundamental concepts.

Scenes in Qt3D work with a concept which is quite similar to what is shown in Blender's outliner: a scene to be rendered is represented by a tree. Each node in the tree can be an object to be displayed, a camera, a light, a material, and so on. We will see later how these entities are represented internally by Qt3D, but for the time being suffice it to say that nodes are represented by instances of the Qt3DCore::QEntity class.

Entities in the tree can be created programmatically, by using Qt3D APIs, or by importing existing assets created by external modelling programs such as Blender. We will follow a hybrid approach: we will create the camera and a light programmatically, and we will import the object1.dae scene. That scene also includes a camera and a light, but we won't be using those objects. We will only care about the mesh.

These are the steps we need to follow:

1. Create a Qt Widgets application (MainWindow template)
2. Create an instance of Qt3DCore::QEntity representing the tree's root
3. Create an OpenGL surface into which Qt3D will render (instance of the Qt3DExtras::Qt3DWindow class) and assign the root entity to the Qt3DWindow
4. Create a camera, place it somewhere, make it point to some interesting viewpoint
5. Create a light and place it somewhere
6. Import the 3D model

Create a Qt Widgets application

This step is pretty much a no-brainer, use the wizard to create a Qt Widgets application:

Then choose a project name and file location of your choice (I used Qt3DTests), and make sure you select QMainWindow as base class

This will generate the source code for the MainWindow class in MainWindow.cpp and MainWindow.h

First of all we must add support for Qt3D to the project. To do so we open the project file (Qt3DTests.pro in my case). At the beginning of the project you should see the default Qt modules enabled by the wizard:

QT += core gui

You then need to add the components required by Qt3D:

QT += core gui
QT += 3dcore 3drender 3dinput 3dextras

Failing to do so will result in build errors already at compile time.

Create the tree root

The tree root is the parent of all entities which compose the scene graph (camera, light, torus).

Nodes of the scene graph in Qt3D are represented by instances of the class Qt3DCore::QEntity. Therefore we add a private member in MainWindow.h, also adding the appropriate include statement. Non relevant code is omitted:

#include <Qt3DCore/QEntity>
class MainWindow : public QMainWindow
{
// ... 
private:
    Qt3DCore::QEntity *rootEntity;
// ...
};

Then we allocate rootEntity in MainWindow's constructor. Non relevant code is omitted here, too:

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent)
    , ui(new Ui::MainWindow)
{
    Qt3DCore::QEntity *rootEntity = new Qt3DCore::QEntity();

Create an OpenGL surface into which Qt3D will render

We now need to create an on-screen OpenGL surface for rendering. This can be achieved by creating an instance of the Qt3DExtras::Qt3DWindow. Here we create the window and assign a bluish kind of colour. Let's not focus too much on the details, we will understand what the frame graph is much much later in this series of tutorials.

    Qt3DExtras::Qt3DWindow *view = new Qt3DExtras::Qt3DWindow();
    view->defaultFrameGraph()->setClearColor(QRgb(0x1d1d4f));

We can then assign the rootEntity to the view. In this way the Qt3D renderer will show the scene represented by rootEntity, once this has been properly populated with contents:

    view->setRootEntity(rootEntity);

The Qt3DWindow class represents a native on-screen OpenGL surface, the implementation of which is platform-dependent. It can be used to render OpenGL commands, it can be displayed on the screen, but it can't be used as a QWidget. In fact, it's not a QWidget subclass at all.

To integrate it within our QWidget-based application we must build an adapter which wraps the Qt3DWindow into a QWidget. Qt provides an API, createWindowContainer, exactly for this purpose. Here we create the window container, we assign some size constraints and a focus policy, and assign the container as the central widget of the MainWindow. In this way the view is embedded within the MainWindow and it will follow its position and size.

    QWidget *container = QWidget::createWindowContainer(view);
    QSize screenSize = view->screen()->size();
    container->setMinimumSize(QSize(200, 100));
    container->setMaximumSize(screenSize);
    container->setFocusPolicy(Qt::NoFocus);

    setCentralWidget(container);

Create a camera, place it somewhere, make it point to some interesting viewpoint

We can create a camera by creating an instance of the Qt3DRender::QCamera class.

However, there's no real need to do so: the Qt3DExtras::Qt3DWindow class already creates a default camera for us. So we just need to assign some parameters like focal length and field of view, place it somewhere, and choose where it's looking at:

Qt3DRender::QCamera *cameraEntity = view->camera();

// X+ -> right
// Y+ -> away
// Z+ -> up
cameraEntity->lens()->setPerspectiveProjection(
    45.0f, 16.0f/9.0f, 0.1f, 1000.0f);
cameraEntity->setPosition(QVector3D(0, -5, 5.0f));
cameraEntity->setUpVector(QVector3D(0, 0, 1));
cameraEntity->setViewCenter(QVector3D(0, 0, 0));

We can optionally add an interactive controller to the camera, which will allow us to move and orbit around the scene. For this purpose Qt3D provides some "camera controller" classes, we are going to use QOrbitCameraController which allows us to navigate around the scene using LMB/RMB + drag:

Qt3DExtras::QOrbitCameraController *camController = new 
    Qt3DExtras::QOrbitCameraController(rootEntity);
camController->setCamera(cameraEntity);

QOrbitCameraController is derived from QAbstractCameraController which is, in turn, derived from QEntity. As such, it belongs to the scene graph like anything which needs to be used or visualised, and its parent is rootEntity.

Create a light and place it somewhere

The process to create a light is simple but involves a concept we have not seen so far: components.

The first thing we need to do is to create an instance of Qt3DCore::QEntity to represent our light:

Qt3DCore::QEntity *lightEntity = new Qt3DCore::QEntity(rootEntity);

Please notice we are passing "rootEntity" to the constructor. This optional parameter represents the parent entity. Therefore, lightEntity is going to be a child of rootEntity, which is necessary if we want the light to appear in the scene graph.

Up to this point, lightEntity is just a node in the tree. It's neither a light nor an object to be rendered. It does not contain any information about what it is, about its position, and so on. All of this information is added to QEntity by another class (and its derived classes) : Qt3DCore::QComponent. This is an abstract class which provides several concrete implementations:

The list goes actually deeper, as some of the subclasses (like Qt3DRender::QMaterial) are abstract too.

A Qt3DCore::QEntity instance contains a QVector<Qt3DCore::QComponent *> which defines what the object actually does and how it's placed and oriented in space.

So, the first need we need to do is to add an instance of Qt3DRender::QPointLight to lightEntity's component list. Qt3DRender::QPointLight is a subclass of Qt3DRender::QAbstractLight which is, in turn, a subclass of Qt3DCore::QComponent (see the list above).

Qt3DRender::QPointLight *light = new Qt3DRender::QPointLight(lightEntity);
light->setColor("white");
light->setIntensity(1);

lightEntity->addComponent(light);

After creating the light we must place it somewhere. This is achieved by adding a QTransform component to the light entity:

Qt3DCore::QTransform *lightTransform = new Qt3DCore::QTransform( 
    lightEntity);
lightTransform->setTranslation(QVector3D(-20, 20, 20));
lightEntity->addComponent(lightTransform);

Import the 3D model

This step is relatively straightforward: Qt3D provides a class called Qt3DRender::QSceneLoader which takes care about importing models into a variety of formats. Among the various export/import formats I have tried, Collada (.dae) seems to be the most complete in terms of correct geometry, material, and texture management.

To use the scene loader we must first create a QEntity and attach the scene loader to it. Then we just pass the path to the scene which needs to be imported:

    sceneLoaderEntity = new Qt3DCore::QEntity(rootEntity);
    loader = new Qt3DRender::QSceneLoader(sceneLoaderEntity);
    loader->setObjectName("object1.dae");
    sceneLoaderEntity->addComponent(loader);
    loader->setSource(
        QUrl(QString("file:../Qt3DTests/Models/object1.dae")));

We then run the application, and our torus is visible in all its sheer beauty:

We can move and orbit around it by dragging with LMB (move) or RMB (orbit) pressed. Notice we are moving the camera, not the object: therefore we will see the "bright" or the "dark" side of the torus, depending from where we look at it.

Next steps

In the next episode of this tutorial we will dig deeper into the Qt3D structure, and see how the scene is represented internally.

Where to get source code

This series of tutorials is stored on GitHub:

https://github.com/gpiasenza/Qt3DTests

If you want to get the exact version used in this article, please checkout the tag STEP_001

https://github.com/gpiasenza/Qt3DTests/releases/tag/STEP_001