Scan-to-Mesh: Getting to precision

Over the weekend, I was experimenting with Autodesk's ReCap 360 Pro and ReMake for large point cloud and photo meshing. The tutorial videos for these platforms have a fast learning curve. I am especially fond of the ReMake Series. This is a pretty remarkable editing platform...from a distance. Unfortunately, I am still working on actually being able to use it due to ReMake not recognizing that I am logged into my Autodesk account. I am quite sure it will be resolved soon and when it is, I will be blogging about that too.

One of the drawbacks to these Autodesk cloud-based meshing services is that users are limited in the number of scans that can be processed at one time (20); and photos (250).

Scan Limitation in ReCap 360 Pro

Hiding enough scans to reach the 20 scan number limit does the trick. Again, and unfortunately, ReCap suggests that you "optimize" for better performance before uploading to the cloud...this can take several hours in itself and requires that you be connected to the internet.

One of the marketing angles we have at ToPa 3D, is that we take on large processing projects. That is sort of our niche. So for scan projects that have hundreds of scans or thousands of photos that we need to model, we look toward solutions that can manage it relatively quickly. That turned our attention to the new meshing tools in FARO Scene - a platform that puts users in control of the mesh, showing your results in real-time.

FARO Scene just released 6.1 which has a more sophisticated meshing option which, after some trial and error, I found to be quite good and relatively fast. We will still have to use other software such as Polyworks, Geomagic, or (eventually) ReMake for a deviation analysis to verify the quality of the mesh against the point clouds, but for creating meshes from point clouds, FARO Scene works well. Lets start with what didn't work...

A stunning watertight bubble generated in FARO Scene 6.1...which has no purpose...and serves no one...at all...whatsoever.

The new FARO Scene 6.1 (or just "Scene") mesh tools are simple with limited options. Yet, that's not such a bad thing actually. The above example didn't work because it was an open floor plan so the watertight mesh tool didn't know how to handle it. More sophisticated meshing software would understand, with the right filters in place, that what I was hoping to do was close gaps within a certain tolerance, ignoring the vast open spaces. Scene doesn't have that level of intricacy, but we are still giving it a go...

FARO has worked diligently on 'octree' technology. This visualization technology allows users to view hundreds or thousands of point clouds in one viewport. Amazing. And, it works extremely well. Our last project consisted of around 350 laser scans and once converted into the FARO Scene tool "Project Point Cloud" (PPC) - which by the way, took about 20 hours to convert - resulted in a beautiful, photo-textured point cloud with balanced colors and the ability to navigate with my 3D mouse like a first-person shooter video game. Zippy!

"Noise" in the scan data (equipment and a guy operating it) - FARO Scene

Because PPC optimizes the navigation experience so well, it becomes quite easy to delete noise out of the point clouds. This is a necessary first step before going into the meshing workflow because Scene has a limitation on the number of polygons the meshing tool can generate. The less point cloud data you have, the more mesh polygons can go to other, more useful data, thereby putting the limited polygons you have available to the details you need.

Selecting the noise is easy with the polygon selector in the 3D view of Scene. I choose to view the point cloud with an orthographic camera option and a clipping box to square it up to my viewport for a clean deletion. Once you have drawn a selection polygon around the noise data, simply right-click-->selection-->delete selected... and...

No more noise!

This process is tedious but with the use of clipping boxes, it greatly simplifies the work. A word of caution though. PPC is fickle; in fact Scene has been historically fickle and prone to crash, therefore it's important to save AND update PPC (a checkbox on the save dialog window) relatively frequently - PPC and Scene in general can be unforgiving should you try to do too many things without saving periodically...newer updates however have proven that Scene is getting much more stable.

Assuming you were able to successfully clean all the data in this way and Scene cooperated, even though it took a quite a bit of time to keep updating PPC, we can move onto the next step.

Meshing Dialog Window - FARO Scene 6.1 - note: Mesh Name (ensure this is unique because with this workflow, you will be creating a great number of mesh models and later joining them together); Watertight checkbox (skip this unless you are doing statues or something small); Maximum Number of Triangles (The maximum is 40 million - which seems like a lot, but these models still orbit well in most 3D software that can read meshes...note that this number will drop to about 10 million as a maximum for very small object you select for meshing); Detail Reduction Behavior (if you have color scans, use that - it bakes in the photo-textures with no noticeable performance issues) - That's it!

The top ribbon in the image above, 3rd button from the right is the "Mesh Selection" tool. This offers 2 ways to mesh. I don't use either one...

I love clipping boxes. I do everything through these because it ensures the operation is going to honor the clipping box. In my experience, tools do not always do what you suppose they should do. Better safe than sad. So this is what I did:

Created a clipping box around the entire 100,000 sqft area (not cubic) structure to see what would happen. Hours later, I stopped the process. No results.

• Created a clipping box that enveloped about 300k cubic feet and kept my focus on the detail of a sphere I had on the ground in the scan. This was equivalent to approximately 1 floor of 5 floors. Note that the sphere is just a lump in the foreground.

• Created a clipping box that enveloped about 37k cubic feet and kept my focus on the detail of a sphere I had on the ground in the scan. This was equivalent to approximately 1/3rd of 1 floor with 5 floors total for the building. The sphere base is still flanged with mesh faces and would provide a measurement accuracy of probably 1" at best.

• Created a clipping box that enveloped about 9k cubic feet and kept my focus on the detail of a sphere I had on the ground in the scan. This was equivalent to approximately 1/8th floor of 5 floors. Sphere base is getting more refined with 1/2" measured accuracy possible.

• Created a clipping box that enveloped about 4k cubic feet and kept my focus on the detail of a sphere I had on the ground in the scan. This was equivalent to approximately 1/16th floor of 5 floors. Sphere is refined enough for 1/2" or better measured accuracy. This is where I decided to stop so my clipping box slices would be in the dozens and not in the hundreds.

So why go to all this trouble? Well, recall that Scene's meshing tool has a limit on the number of polygons that can be meshed at once: 40 million of them. Therefore, if I mesh a large area, Scene will have to distribute 40 million polygons over that entire area - making for relatively large polygons. But by selecting a small area to mesh, Scene will still use 40 million polygons; they will just be smaller and therefore provide you with a much more measurably accurate and photo-realistic mesh-model.

Determined mesh-model slice based on mesh details observed: FARO Scene Screen Shot

Now that I had the right-sized clipping box slice and the mesh was tight enough (polygons were small enough) for the tolerances we were looking for, I was ready for the mesh production workflow.

Enter "Multiple Clipping Boxes" tool...

"Create Clipping Boxes along an axis" tool. Access this tool by right-clicking on your initial clipping box.

To enable this, simply create a clipping box around the point cloud data (in this case, a single floor of a 5-story building). This tool allows users to create an array of clipping boxes within the initial clipping box created. You can then set the parameter to have the array branch out along the x, y, or z axis in one direction or another (or both, using the positive/negative checkboxes). You can check the "Disable Clipping" to see how this array is looking to you and whether you are encompassing all you like. Users can choose to set a number of clipping boxes along this array that will be equal in size to the initial clipping box created. Finally, for this exercise, choose the drop-down option of "Space between boxes" and set to zero so that there is no gap between the boxes. This is because we will be creating a mesh from each individual box and we want all these detailed meshes to eventually, and seamlessly, join back together.

"Multiple Clipping Boxes" array.

The first thing to do once these are created is to DEACTIVATE ALL clipping boxes.

Even though you right-click on a single clipping box and choose to create a mesh from there - Scene will still try to mesh ALL active clipping boxes. Buggy Bummer... But, now you know. And as many of you Scene users know, canceling, skipping, or any other attempted stopping of any sort of process in Scene doesn't work very well, locks up your computer, and usually results in the necessity of just hard-crashing Scene to start over.

Starting from one end of the array, right click on the Multiple Clipping Box in Structure View.

Choose "Create Mesh using active Clipping Boxes." Remember to de-activate all other clipping boxes before selecting this option.

The area of interest should be the only clipping box active.

"Create Mesh using active Clipping Boxes" started here. For the 4,000 cubit foot area of the point cloud I chose to mesh, this meshing creation process only took about 1 minute per slice. To slice up the entirety of the floor equaled about 20 slices.

I also uniquely named all my mesh slices based on the particular Multiple Clipping Box it belonged to. This kept me straight when my brain started to wander into Blender (the program, not the appliance).

Exporting the meshes was fast and Scene offers users 4 file type options: (.PLY, .STL, .WRL, .OBJ). I was using Blender 2.78 (open source version of 3DS Max) and found that the OBJ type didn't recognize the photo-texture, but the other file types did. I would imagine this is because .OBJ files usually pack their texture maps in a .MTL file and Scene apparently "Bakes" their textures into the mesh. I didn't notice any supporting texture files upon export from Scene.

The meshes looked great in Blender. Because I wanted to preserve any scale and coordinate system in the meshes, I didn't move my imported mesh slices around, bringing them into place one-by-one.

2 mesh models imported in .PLY format into Blender. Note the orange 'seamline' between the slices. This line was seamless.

The purpose of bringing these meshes into Blender (or your favorite mesh modeling platform) was to edit the meshes, preparing them for other applications such as gaming or visualizations that clients can use with VR headsets and so forth; a much lighter weight option than trying to visualize point clouds on the fly.

Before joining the meshes as one single data-set, I edited some mesh artifacts that Scene created when meshing the slices. Straight edges of the slices looked good, but Scene attempted to close the pipes on the edge of each slice with more mesh faces. Therefore I cut the rounded areas off in Blender and extruded to match up with continuing pipes from other slices.

Another shot of the 2 meshes separated by an orange selection 'seamline' - Image from Blender by ToPa 3D

Editing the tip of the pipes where the seamline was.

In Blender, backface culling is enabled by default. This means that when using a selection tool such as a 'lasso', users will only grab what is directly visible to the viewer with objects occluded being ignored. This is really a safety feature to ensure your edits aren't getting carried away. You can however disable this quickly with a button - which is what I did for cutting off the end of the pipe (perhaps TMI...)

After mesh edits were complete, I "joined" all the slices together for a contiguous mesh, ready for exporting to other platforms for visualization.

Seamline gone. Models successfully 'Joined'

Exporting as .OBJ in Blender

I also tried to bring these models back into Scene after edits to compare to the point cloud - ensuring scaling was correct. Scene however only understands .WRL/.VRML model types. This is a limited option and without a specialized converter - that understands the scale of the mesh model it's converting - you will not be able to get these models back into Scene. A better way to verify that scale and position is holding is to just bring everything into a CAD platform and see where they land.

In Summary

Project Point Cloud-to-Mesh

1. Clean noise (it is faster to delete points from PPC than it is from a mesh. In fact, in Scene, mesh-model elements wouldn't delete for me.) Also, clean the entirety of the PPC, save and "Update PPC" before you begin creating the meshes. I tried to clean and create meshes on the fly and Scene kept crashing probably because the edits to the PPC weren't being saved. Warning: Updating PPC can be very time consuming (my initial creation of it with 350 scans took about 20 hours of constant computational time).

2. Determine Mesh Quality you desire visually, with different sized clipping box-to-mesh experiments.

3. Slice Project Point Cloud with Multiple Clipping Box Feature. Note that your navigation with this array of Multiple Clipping Boxes visible will get squirrley - so I make them invisible when navigating, which corrects that issue.

4. Create Mesh from individual slices, de-activating ALL other clipping boxes.

5. Export meshes

6. Import into...(your choice)

Thanks for reading and please share your experience/feedback/workflows as you see fit and Happy Modeling;)

ToPa 3D~

Afterward...

Point Cloud Sphere we've been talking about...

Isolated sphere using clipping box

Meshed Sphere without 'Watertight' checkbox - missing data at the bottom.

Meshed Sphere with 'Watertight' checkbox - Not too shabby!