GUI integration

tl;dr

Go to corner.builders. On the bottom there is a new panel with few controls. Try them! Moving in the scene is by pinch on phone or mouse scroll.

Where we are

This newsletter is dedicated in building a tool to project images on walls. More info on the previous editions and in this post. We already have the website, made a demo, and setup our ci/cd pipeline.

Goal of this session

Our app will need some reactive user interaction, for example for defining the room, placing the point of view, selecting the image, placing the image, etc. We need to use something to help us generating buttons, slidders, input box etc and manage interaction. The goal of this session is not to have a definitive UI in place, but to select the tool, and be ready to add easily user-interactions. There exists a multitude of web-framework like React, Vue, Angular that help design User-Interface and propagate user action.

Immediate GUI

Our approach will be different than the traditional web-frameworks. We will use some GUI (Graphical User Interface) integration to generate the GUI directly in the html canvas. There exists 2 main schools of GUI approach, Immediate and Retained. We are going to chose an immediate GUI based on the simplicity, directness, functional-style, rapid-prototyping nature of immediate GUI : egui.

For example, creating, rendering, and incrementing a variable with an egui button click is as simple as this:

if ui.button("Click!").clicked() {
    var += 1;
}        

UX

The so-called User Xperience. This is not my area of expertise and feel free to comment good ways to make the app a better user experience! For example personally, by the way I hold my phone, I like the user buttons to be on the bottom of the screen and look/read at the top. Are there any user survey about this? I'd like to hear! Also I hate panels of intrusive ads so there will be none.

Functionalities

Room size

While in a later stage, it would be nice to allow for more complex room geometries, maybe allowing CAD file imports, we will start with simple rectangle rooms. The user should be able to choose the triplet size: (Width - Depth - Height).

Point of View

The user should also be able to select the point of view. As a reminder, the point of view is the point in space where the image on the wall should appear undistorted. For simplicity of real-life measurements, the zero point should be in a top corners. A corner because it's easier to measure in the horizontal plane, and at the top, because the ceiling is less cluttered. Imagine you want a point of view on top of a bed, you have a free line to the ceiling, but not the ground (there is the bed in between, duh!)

Color of walls

If walls are of a specific color, it might play a role in the decision of the image, it's placement, etc, so it might be nice to include the color selection in the GUI.

Results

That's it for now! We just integrated a few basic control, more will come later as they are needed! Below you can see a quick demo about the different button. Navigation can be done, with mouse (click and drag) + scroll wheel to move forward/backward, or pinch in/out on mobile. Try it yourself on the website: corner.builders! You will notice now the floating projecting shape not attached to the wall anymore. That's normal for now, we will handle it later! Also please don't mind the default colors, I put them randomly!

Next step

Prepare the image input: a way to lock/unlock the camera on the viewpoint, orientation + size of the image input, validation. After this we should be ready to start projecting!

Stay tuned!