Discover 3D Point Cloud Processing with Python
- We need to set-up our environment. I recommend to download Anaconda Navigator, which comes with an easy GUI.
- Once downloaded and installed, create an environment (2nd tab on the left > Create button), which allows you to specify a python version (the latest is fine). You know it is selected when the green arrow is next to it.
- Once created, you can then link wanted libraries without conflicts. Very handy! For this, just search packages in what is installed (E.g. NumPy, Matplotlib), and if it is not popping, then select Not installed, check them both and click on Apply to install them. Numpy and Matplotlib are standard libraries that will be useful for this and future projects.
- You are almost set-up, now back to the Anaconda Home Tab, make sure you are in the right environment (Applications on XXX), then you can install Spyder as the IDE (Integrated Development Environment)to start your code project.
🤓 Note: Spyder is one of the best tools for any amateur who is new to python coding. Another great tool is Jupyter, which does a great job of presenting interactive code to higher management for better visualization. We will explore this as well as Google Colab on future posts.
- Once the installation progress bar is done you are ready! You can now launch Spyder. While the GUI may allow several possibilities, to directly obtain results, first write your script (1), execute your script (2) and explore + interact with results in the console (3).
For future experiments, we will use a sampled point cloud that you can freely download from this repository. If you want to visualize it beforehand without installing anything, you can check the webGL version.
In Sypder, let us start by using a very powerful library: NumPy. You can already write your first shortcode in the script area (left window):1 import numpy as np
3 point_cloud= np.loadtxt(file_data_path, skiprows=1, max_rows=1000000)
This shortcode (1) import the library NumPy for further use as a short name “np”; (2) create a variable that holds the string pointing to the file that contains the points; (3) import the point cloud as a variable named point_cloud, skipping the first row (holding, for example, the number of points), and setting a maximal number of rows to run tests without memory shortages.
You can now run your script (green arrow), and save it as a .py file on your hard-drive when the pop-up appears. You can now access the first point of the entity that holds your data (point_cloud) by directly writing in the console:In: point_cloud
You will then get an array containing the content of the first point, in this case, X, Y and Z coordinates.Out: array([0.480, 1.636, 1.085])
These were your first steps with python and point clouds. Now that you know how to load point data, let us look at some interesting processes.
We have a point cloud with 6 attributes: X, Y, Z, R, G, B. It is important to note that when playing with NumPy arrays, the indexes always start at 0. So, if loaded without field names, then getting the second point is done doing:In: point_cloud
Out: array([0.480, 1.636, 1.085, 25, 44, 68])
If from there we want to obtain the Red (R) attribute (the NumPy “column” index is 3), we can do:In: point_cloud
If we want to extract the Z attribute for all the points in the point cloud:In: point_cloud[:,1]
Out: array([2.703, 2.716, 2.712, …, 2.759, 2.741, 2.767])
If we want to extract only X, Y, Z attributes for all the points:In: point_cloud[:,:3]
Out: array([[4.933, 2.703, 2.194],
[4.908, 2.716, 2.178],
[4.92 , 2.712, 2.175],
[5.203, 2.759, 0.335],
[5.211, 2.741, 0.399],
[5.191, 2.767, 0.279]])
Congratulations, you just played around with multi-dimensional indexing 👏. Note that in the example above , the third column (R) is excluded from the selection. Each result can be stored in variables if they are meant to be used more than one time:xyz=point_cloud[:,:3]
Now let us look at some useful analysis. If you want to know the mean height of your point cloud, then you can easily do:In: np.mean(point_cloud,axis=0)
💡 Hint: here, the axis set to 0 is asking to look at each “column” independently. If ignored, then the mean run on all the values, and if set to 1, it will average per row.
If now you want to extract points which are within a buffer of 1 meter from the mean height (we assume the value is stored in mean_Z):In: point_cloud[abs( point_cloud[:,2]-mean_Z)<1]
💡 Hint: In python, and programming in general, there is more than one way to solve a problem. The provided is a very short and efficient way, which may not be the most intuitive. Trying to solve it using a for loop is a great exercise. The aim is a good balance between clarity and efficiency, see the PEP-8 guidelines.
Now you know how to set-up your environment, use Python, Spyder GUI and NumPy for your coding endeavors. You can load point clouds and play with attributes, but you can try other scenarios such as color filtering, point proximities …
Interactive 3D Visualization using Matplotlib
NOTE : This lesson is still under construction...
Matplotlib has the advantage of being easy to set up. Almost anyone that is working in machine learning or data science will already have this installed. There are, however, several reasons why you should avoid using it to visualize point clouds interactively in 3D.
Firstly matplotlib is incredibly slow. It will likely crash your computer if you want to actually visualize all of the points in something like a LIDAR scan.
Secondly, it just doesnt produce very nice point cloud visualizations. If you are processing LIDAR point clouds for instance, it is unlikely you will be able to recognize anything in the scene when using matplotlib.
Mayavi has the disadvantage of being quite tricky to install, but does an amazingly good job at visualizing point cloud data. So i would encourage trying that if you can.
NOTE : This lesson is still under construction...
In order to prevent matplotlib from crashing your computer, it is recomended to only view a subset of the point cloud data. For instance, if you are visualizing LIDAR data, then you may only want to view one in every 25-100 points. Below is some sample code to get you started.
TODO: show images comparing the output of matplotlib vs mayavi.
Guide to real-time visualisation of massive 3D point clouds in Python
Tutorial for advanced visualization and interaction with big point cloud data in Python. (Bonus) Learn how to create an interactive segmentation “software”.
Data visualisation is a big enchilada 🌶️: by making a graphical representation of information using visual elements, we can best present and understand trends, outliers, and patterns in data. And you guessed it: with 3D point cloud datasets representing real-world shapes, it is mandatory 🙂.
However, when collected from a laser scanner or 3D reconstruction techniques such as Photogrammetry, point clouds are usually too dense for classical rendering. In many cases, the datasets will far exceed the 10+ million mark, making them impractical for classical visualisation libraries such as Matplotlib.
This meansthat we often need to go out of our Python script (thus using an I/O function to write our data to a file) and visualise it externally, which can become a super cumbersome process 🤯. I will not lie, that is pretty much what I did the first year of my thesis to try and guess the outcome of specific algorithms🥴.
Would it not be neat to visualise these point clouds directly within your script? Even better, connecting the visual feedback to the script? Imagine, now with the iPhone 12 Pro having a LiDAR; you could create a full online application! Good news, there is a way to accomplish this, without leaving the comfort of your Python Environment and IDE. ☕ and ready?
In the previous article below, we saw how to set up an environment with Anaconda easily and how to use the IDE Spyder to manage your code. I recommend continuing in this fashion if you set yourself up to becoming a fully-fledge python app developer 😆.
If you are using Jupyter Notebook or Google Colab, the script may need some tweaking to make the visualisation back-end work, but deliver unstable performances. If you want to stay on these IDE, I recommend looking at the alternatives to the chosen libraries given in Step 4.
I illustrated point cloud processing and meshing over a 3D dataset obtained by using photogrammetry and aerial LiDAR from Open Topography in previous tutorials. I will skip the details on LiDAR I/O covered in the article below, and jump right to using the efficient .las file format.
Only this time, we will use an aerial Drone dataset. It was obtained through photogrammetry making a small DJI Phantom Pro 4 fly on our University campus, gathering some images and running a photogrammetric reconstruction as explained here.
🤓 Note: For this how-to guide, you can use the point cloud in this repository, that I already filtered and translated so that you are in the optimal conditions. If you want to visualize and play with it beforehand without installing anything, you can check out the webGL version.
We first import necessary libraries within the script (NumPy and LasPy), and load the .las file in a variable called .import numpy as np
import laspy as lp
Nice, we are almost ready! What is great, is that the LasPy library also give a structure to the variable, and we can use straightforward methods to get, for example, X, Y, Z, Red, Blue and Green fields. Let us do this to separate coordinates from colours, and put them in NumPy arrays:points = np.vstack((point_cloud.x, point_cloud.y, point_cloud.z)).transpose()
colors = np.vstack((point_cloud.red, point_cloud.green, point_cloud.blue)).transpose()
🤓 Note: We use a vertical stack method from NumPy, and we have to transpose it to get from (n x 3) to a (3 x n) matrix of the point cloud.
If your dataset is too heavy, or you feel like you want to experiment on a subsampled version, I encourage you the check out the article below that give you several ways to achieve such a task:
Or the following formation for extensive point cloud training:
For convenience, and if you have a point cloud that exceeds 100 million points, we can just quickly slice your dataset using:factor=10
decimated_points_random = points[::factor]
🤓 Note: Running this will keep 1 row every 10 rows, thus dividing the original point cloud's size by 10.
Now, let us choose how we want to visualise our point cloud. I will be honest, here: while visualisation alone is great to avoid cumbersome I/O operations, having the ability to include some visual interaction and processing tools within Python is a great addition! Therefore, the solution that I push is using a point cloud processing toolkit that permits exactly this and more. I will still give you alternatives if you want to explore other possibilities ⚖️.
Solution A (Retained): PPTK
The PPTK package has a 3-d point cloud viewer that directly takes a 3-column NumPy array as input and can interactively visualize 10 to 100 million points. It reduces the number of points that needs rendering in each frame by using an octree to cull points outside the view frustum and to approximate groups of faraway points as single points.
To get started, you can simply install the library using the Pip manager:pip install pptk
Then you can visualise your previously createdvariable from the point cloud by typing:import pptk
import numpy as np
v = pptk.viewer(points)
Don’t you think we are missing some colours? Let us solve this by typing in the console:v.attributes(colors/65535)
🤓 Note: Our colour values are coded on 16bits from the .las file. We need the values in a [0,1] interval; thus, we divide by 65535.
That is way better! But what if we also want to visualise additional attributes? Well, you just link your attributes to your path, and it will update on the fly.
💡 Hint:Do not maximize the size of the window to keep a nice framerate over 30 FPS. The goal is to have the best execution runtime while having a readable script
You can also parameterize your window to show each attributes regarding a certain colour ramp, managing the point size, putting the background black and not displaying the grid and axis information:v.color_map('cool')
Alternative B: Open3D
For anybody wondering for an excellent alternative to read and display point clouds in Python, I recommend Open3D. You can use the Pip package manager as well to install the necessary library:pip install open3d
We already used Open3d in the tutorial below, if you want to extend your knowledge on 3D meshing operations:
This will install Open3D on your machine, and you will then be able to read and display your point clouds by executing the following script:import open3d as o3dpcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(points)
pcd.colors = o3d.utility.Vector3dVector(colors/65535)
pcd.normals = o3d.utility.Vector3dVector(normals)o3d.visualization.draw_geometries([pcd])
Open3D is actually growing, and you can have some fun ways to display your point cloud to fill eventual holes like creating a voxel structure:voxel_grid = o3d.geometry.VoxelGrid.
🤓 Note: Why is Open3d not the choice at this point? If you work with datasets under 50 million points, then it is what I would recommend. If you need to have interactive visualization above this threshold, I recommend either sampling the dataset for visual purposes, or using PPTK which is more efficient for visualizing as you have the octree structure created for this purpose.
Other (Colab-friendly) alternatives: Pyntcloud and Pypotree
If you would like to enable simple and interactive exploration of point cloud data, regardless of which sensor was used to generate it or what the use case is, I suggest you look into Pyntcloud, or PyPotree. These will allow you to visualise the point cloud in your notebook, but beware of the performances! Pyntcloud actually rely on Matplotlib, and PyPotree demands I/O operations; thus, both are actually not super-efficient. Nevertheless, I wanted to mention them because for small point clouds and simple experiment in Google Colab, you can integrate the visualisation. Some examples:### PyntCloud ###
conda install pyntcloud -c conda-forge
from pyntcloud import PyntCloudpointcloud = PyntCloud.from_file("example.ply")
pointcloud.plot()### PyntCloud ###
pip install pypotreeimport pypotree
import numpy as np
xyz = np.random.random((100000,3))
cloudpath = pypotree.generate_cloud_for_display(xyz)
Back to PPTK. To make an interactive selection, say the car on the parking lot, I will move my camera top view (shortcut is ), and I will make a selection dragging a rectangle selection holding +.
💡 Hint:If you are unhappy with the selection, a simple will erase your current selection(s). Yes, you can make multiple selections 😀.
Once the selection is made, you can return to your Python Console and then get the assignment's point identifiers.selection=v.get('selected')
This will actually returns a 1D array like this:
You can actually extend the process to select more than one element at once (+) while refining the selection removing specific points (++ ).
After this, it becomes effortless to apply a bunch of processes interactively over your variable that holds the index of selected points.
Let us replicate a scenario where you automatically refine your initial selection (the car) between ground and non-ground elements.
In the viewer that contain the full point cloud, stored in the variable , I make the following selection :
Then I compute normals for each points. For this, I want to illustrate another key takeaway of using PPTK: The function , which can be used to get a normal for each point based on either a radius search or the k-nearest neighbours. Don’t worry, I will illustrate in-depth these concepts in another guide, but for now, I will run it by using the 6 nearest neighbours to estimate my normals:normals=pptk.estimate_normals(points[selection],k=6,r=np.inf)
💡 Hint:Remember that the variable holds the indexes of the points, i.e. the “line number” in our point cloud, starting at 0. Thus, if I want to work only on this point subset, I will pass it as . Then, I choose the k-NN method using only the 6 nearest neighbours for each point, by also setting the radius parameter to which make sure I don’t use it. I could also use both constraints, or set k to if I want to do a pure radius search.
This will basically return this:
Then, I want to filter AND return the original points' indexes that have a normal not colinear to the Z-axis. I propose to use the following line of code:idx_normals=np.where(abs(normals[...,2])<0.9)
🤓 Note: The , is a NumPy way of saying that I work only on the 3rd column of my 3 x n point matrix, holding the Z attribute of the normals. It is equivalent to . Then, I take the absolute value as the comparing point because my normals are not oriented (thus can point toward the sky or towards the earth centre), and will only keep the one that answer the condition , using the function .
To visualise the results, I create a new viewer window object:viewer1=pptk.viewer(points[idx_normals],colors[idx_normals]/65535)
As you can see, we also filtered some points part of the car. This is not good 🤨. Thus, we should combine the filtering with another filter that makes sure only the points close to the ground are chosen as host of the normals filtering:idx_ground=np.where(points[...,2]>np.min(points[...,2]+0.3))
This is nice! And now, you can just explore this powerful way of thinking and combine any filtering (for example playing on the RGB to get away with the remaining grass …) to create a fully interactive segmentation application. Even better, you can combine it with 3D Deep Learning Classification! Ho-ho! But that is for another time 😉.
Finally, I suggest packaging your script into functions so that you can directly reuse part of it as blocks. We can first define a , that will take as input any point cloud, and format it :def preparedata():
points = np.vstack((point_cloud.x, point_cloud.y, point_cloud.z)
colors = np.vstack((point_cloud.red, point_cloud.green,
normals = np.vstack((point_cloud.normalx, point_cloud.normaly,
Then, we write a display function , that return a viewer object:def pptkviz(points,colors):
v = pptk.viewer(points)
Additionally, and as a bonus, here is the function , to get the current parameters of your camera from the opened viewer:def cameraSelector(v):
And we define the function to automate the refinement of your interactive segmentation:def computePCFeatures(points, colors, knn=10, radius=np.inf):
Et voilà 😁, you now just need to launch your script containing the functions above and start interacting on your selections using , , and more of your creations:import numpy as np
import laspy as lp
import pptk#Declare all your functions hereif __name__ == "__main__":
It is then easy to call the script and then use the console as the bench for your experiments. For example, I could save several camera positions and create an animation:cam1=cameraSelector(v)
#Change your viewpoint then -->
#Change your viewpoint then -->
#Change your viewpoint then -->
cam4=cameraSelector(v)poses = 
v.play(poses, 2 * np.arange(4), repeat=True, interp='linear')
You just learned how to import, visualize and segment a point cloud composed of 30+ million points! Well done! Interestingly, the interactive selection of point cloud fragments and individual points performed directly on GPU can now be used for point cloud editing and segmentation in real-time. But the path does not end here, and future posts will dive deeper into point cloud spatial analysis, file formats, data structures, segmentation [2–4], animation and deep learning . We will especially look into how to manage big point cloud data as defined in the article below.
My contributions aim to condense actionable information so you can start from scratch to build 3D automation systems for your projects. You can get started today by taking a formation at the Geodata Academy.
1. Poux, F., & J.-J Ponciano. (2020). Self-Learning Ontology For Instance Segmentation Of 3d Indoor Point Cloud. ISPRS Int. Arch. of Pho. & Rem. XLIII-B2, 309–316; https://doi.org/10.5194/isprs-archives-XLIII-B2–2020–309–2020
2. Poux, F., & Billen, R. (2019). Voxel-based 3D point cloud semantic segmentation: unsupervised geometric and relationship featuring vs deep learning methods. ISPRS International Journal of Geo-Information. 8(5), 213; https://doi.org/10.3390/ijgi8050213
3. Poux, F., Neuville, R., Nys, G.-A., & Billen, R. (2018). 3D Point Cloud Semantic Modelling: Integrated Framework for Indoor Spaces and Furniture. Remote Sensing, 10(9), 1412. https://doi.org/10.3390/rs10091412
4. Poux, F., Neuville, R., Van Wersch, L., Nys, G.-A., & Billen, R. (2017). 3D Point Clouds in Archaeology: Advances in Acquisition, Processing and Knowledge Integration Applied to Quasi-Planar Objects. Geosciences, 7(4), 96. https://doi.org/10.3390/GEOSCIENCES7040096
This tutorial demonstrates basic usage of a point cloud.
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
# examples/Python/Basic/pointcloud.pyimportnumpyasnpimportopen3daso3dif__name__=="__main__":print("Load a ply point cloud, print it, and render it")pcd=o3d.io.read_point_cloud("../../TestData/fragment.ply")print(pcd)print(np.asarray(pcd.points))o3d.visualization.draw_geometries([pcd])print("Downsample the point cloud with a voxel of 0.05")downpcd=pcd.voxel_down_sample(voxel_size=0.05)o3d.visualization.draw_geometries([downpcd])print("Recompute the normal of the downsampled point cloud")downpcd.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.1,max_nn=30))o3d.visualization.draw_geometries([downpcd])print("Print a normal vector of the 0th point")print(downpcd.normals)print("Print the normal vectors of the first 10 points")print(np.asarray(downpcd.normals)[:10,:])print("")print("Load a polygon volume and use it to crop the original point cloud")vol=o3d.visualization.read_selection_polygon_volume("../../TestData/Crop/cropped.json")chair=vol.crop_point_cloud(pcd)o3d.visualization.draw_geometries([chair])print("")print("Paint chair")chair.paint_uniform_color([1,0.706,0])o3d.visualization.draw_geometries([chair])print("")
Visualize point cloud¶
The first part of the tutorial reads a point cloud and visualizes it.
print("Load a ply point cloud, print it, and render it")pcd=o3d.io.read_point_cloud("../../TestData/fragment.ply")print(pcd)print(np.asarray(pcd.points))o3d.visualization.draw_geometries([pcd])
reads a point cloud from a file. It tries to decode the file based on the extension name. The supported extension names are: , , , , , .
visualizes the point cloud. Use mouse/trackpad to see the geometry from different view point.
It looks like a dense surface, but it is actually a point cloud rendered as surfels. The GUI supports various keyboard functions. One of them, the key reduces the size of the points (surfels). Press it multiple times, the visualization becomes:
On OS X, the GUI window may not receive keyboard event. In this case, try to launch Python with instead of .
Voxel downsampling uses a regular voxel grid to create a uniformly downsampled point cloud from an input point cloud. It is often used as a pre-processing step for many point cloud processing tasks. The algorithm operates in two steps:
Points are bucketed into voxels.
Each occupied voxel generates exact one point by averaging all points inside.
print("Downsample the point cloud with a voxel of 0.05")downpcd=pcd.voxel_down_sample(voxel_size=0.05)o3d.visualization.draw_geometries([downpcd])
This is the downsampled point cloud:
Vertex normal estimation¶
Another basic operation for point cloud is point normal estimation.
print("Recompute the normal of the downsampled point cloud")downpcd.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.1,max_nn=30))o3d.visualization.draw_geometries([downpcd])
computes normal for every point. The function finds adjacent points and calculate the principal axis of the adjacent points using covariance analysis.
The function takes an instance of class as an argument. The two key arguments and specifies search radius and maximum nearest neighbor. It has 10cm of search radius, and only considers up to 30 neighbors to save computation time.
The covariance analysis algorithm produces two opposite directions as normal candidates. Without knowing the global structure of the geometry, both can be correct. This is known as the normal orientation problem. Open3D tries to orient the normal to align with the original normal if it exists. Otherwise, Open3D does a random guess. Further orientation functions such as and need to be called if the orientation is a concern.
Use to visualize the point cloud and press to see point normal. Key and key can be used to control the length of the normal.
Access estimated vertex normal¶
Estimated normal vectors can be retrieved by variable of .
print("Print a normal vector of the 0th point")print(downpcd.normals)
To check out other variables, please use . Normal vectors can be transformed as a numpy array using .
print("Print the normal vectors of the first 10 points")print(np.asarray(downpcd.normals)[:10,:])
Check Working with NumPy for more examples regarding numpy array.
Crop point cloud¶
print("Load a polygon volume and use it to crop the original point cloud")vol=o3d.visualization.read_selection_polygon_volume("../../TestData/Crop/cropped.json")chair=vol.crop_point_cloud(pcd)o3d.visualization.draw_geometries([chair])print("")
reads a json file that specifies polygon selection area. filters out points. Only the chair remains.
Paint point cloud¶
paints all the points to a uniform color. The color is in RGB space, [0, 1] range.
© Copyright 2018 - 2019, www.open3d.org
Cloud 3d python point plot
Python - Display 3D Point Cloud
For anybody wondering for an easy way to read and display PLY point clouds in Python I answer my own question reporting what I've found to be the best solution in my case.
Open cmd and type:
This will install Open3D on your machine and you will then be able to read and display your PLY point clouds just by executing the following sample script:
Try pptk (point processing toolkit). The package has a 3-d point cloud viewer that directly takes a 3-column numpy array as input, and is able to interactively visualize 10-100 million points. (It reduces the number of points that needs rendering in each frame by using an octree to cull points outside the view frustum and to approximate groups of far away points as single points)
To visualize 100 randomly generated points in Python,
screenshot of pptk viewer visualizing 100 random points
The documentation website also has a tutorial specifically on visualizing point clouds loaded from .ply files.
You could use https://github.com/daavoo/pyntcloud to visualize the PLY inside a Jupyter notebook:
Suddenly, in the dark, I saw a couple approaching. Tights, especially thin ones, should be pulled slowly, and they are only half dressed. I had to speed up the process - immediately pull on tights, put things in a bag and run through a pine park about 200 meters. When I reached a remote place, I realized that they did not run around in pantyhose in a pine forest, and quick.
- Target window pane frame
- Skunk fu fox
- Amazon swivel desk chair
- Samsung messages wifi texting
- Polychem 36
- Mhw master rank
- G tech drive icons
- City of lakewood
- Scarborough fair lyrics
She grabs her hand over her dripping pussy and starts rubbing her clitoris. Little. Lie here, damn it. I get up and take out from the bins of the Motherland an artificial member for anal sex.