I. Introduction

The vtk library is an open source 3D computer graphics, image processing and visualization library that can be used to perform tasks such as 3D reconstruction and texture mapping. Due to the 3D model needs of the projects in the group, I began to slowly explore how to use this library (PS: I have never done 3D tasks for learning image processing, it is difficult)... The wheel of vtk (whl file) can be found at this link Available at: Archived: Python Extension Packages for Windows - Christoph Gohlke (uci.edu) https://www.lfd.uci.edu/~gohlke/pythonlibs/#vtk

I found that the vtk tutorials on the Internet are basically derived from the book "Advanced VTK Graphics and Image Development" by Zhang Xiaodong. The content is very comprehensive, but there are always knowledge points that I want to find but can't find. This task is to rebuild the inner wall of the tape paper tube , as shown in the figure below:

The curved surface information of this paper tube is known, and the diameter is 76mm. Here I will put the camera lens at the center of the circle to take a picture, the image size is 2592*1944 (500w pixels), the ratio is 4:3, the obtained picture is as follows ( texture.jpg ) :

The task now is to reconstruct the surface corresponding to the camera field of view using the vtk library, and map the above image as a texture to the reconstructed model . How to do it?

Two, numpy and vtk libraries draw custom surfaces

Check the information and find that vtk provides an important function: vtk.vtkSurfaceReconstructionFilter() , this function can help us to perform implicit plane reconstruction of vertex data in vtk.vtkPolyData() format , and the vertex data of vtk.vtkPolyData() can be Use numpy to build. In this way, the idea is:

1. numpy constructs custom surface sampling points (using linspace, etc.)

2. The vtk library calls vtkSurfaceReconstructionFilter for surface reconstruction and drawing

3. Use the vtk library for texture mapping

2.1 numpy input vertex coordinates

After calculation, the field of view of the captured texture and jpg corresponds to a part of the cylindrical surface, which is called an arc surface for the time being . The angle of this arc surface is about 90° , the radius is 76mm , and the height is 80mm . Here, 1 unit length is used as 1mm for drawing. The code for constructing vertex coordinates is as follows (using numpy and vtk.vtkPoints() ):

import vtk
import numpy as np

# 半径
r = 76
# 极坐标系下构建顶点数据
theta = np.linspace(0, np.pi / 2, 61)
x = r * np.cos(theta)
y = r * np.sin(theta)

# vtkPoints格式的点
points = vtk.vtkPoints()
vertices = vtk.vtkCellArray()

i = 0
for z in range(80):
    for xi, yi in zip(x, y):
        points.InsertPoint(i, xi, yi, z)
        # vertices.InsertNextCell(1)
        # vertices.InsertCellPoint(i)
        i += 1

The 3D image of the vertex array can be drawn through the data of these similar sampling points (the code is not given, it is only used as an example):

With these vertex data, the next step is to call vtkSurfaceReconstructionFilter to rebuild our custom surface.

2.2 vtkSurfaceReconstructionFilter surface reconstruction

With the vertex data in numpy.ndarray format in 2.1, first convert it to the vtk data format in vtk.vtkPolyData() format, mainly using the SetPoints function:

polyData = vtk.vtkPolyData()
polyData.SetPoints(points)

Next, it's time for vtkSurfaceReconstructionFilter and vtkContourFilter to come on stage. Just use the vtkPolyData converted above as input to get the reconstructed surface object:

polyData = vtk.vtkPolyData()
polyData.SetPoints(points)
# polyData.SetVerts(vertices)

surf = vtk.vtkSurfaceReconstructionFilter()
# 输入polyData数据
surf.SetInputData(polyData)
surf.SetNeighborhoodSize(20)
# 此处SetSampleSpacing数值小精度高但运行时间长，数值大则反之
surf.SetSampleSpacing(1.0)
surf.Update()

# 轮廓信息
contour = vtk.vtkContourFilter()
contour.SetInputConnection(surf.GetOutputPort())
contour.SetValue(0, 0.0)
contour.Update()

At this point, we have actually built a custom surface and loaded it into the contour variable in the vtk.vtkContourFilter() format. At this point, we can use the mapper ( mapper ) and actor ( actor ) of the vtk library to draw the appearance of the contour. There are many online tutorials for this part, and you can check for gaps by yourself:

texturemap = vtk.vtkTextureMapToPlane()
texturemap.SetInputData(contour.GetOutput())

# 设置纹理映射原点
texturemap.SetOrigin(76, 0, 0)
# 设置两个坐标轴的方向（根据自定义曲面的信息来输入）
texturemap.SetPoint1(0, 76, 0)
texturemap.SetPoint2(76, 0, 80)

# 映射器
mapper = vtk.vtkPolyDataMapper()
# 映射器输入自定义曲面模型信息
mapper.SetInputConnection(texturemap.GetOutputPort())
# 这一句十分关键，不然后面的纹理映射可能会失败
mapper.ScalarVisibilityOff()

# 演员
actor = vtk.vtkActor()
# 演员添加映射器
actor.SetMapper(mapper)

# 绘制
ren = vtk.vtkRenderer()
# 添加演员
ren.AddActor(actor)

#绘制窗口
renWin = vtk.vtkRenderWindow()
renWin.AddRenderer(ren)
renWin.SetSize(800, 800)

# 交互窗口
iren = vtk.vtkRenderWindowInteractor()
iren.SetRenderWindow(renWin)
iren.Initialize()
renWin.Render()
iren.Start()

A custom surface 3D image without texture looks like this:

We're more than halfway there! Next, you only need to map the texture onto this pure white plane like sticking a film on a curved screen ~

3. Texture mapping

It is not difficult to perform texture mapping on a custom surface. Here, vtk.vtkTextureMapToPlane() is used for model construction, that is, the texture image texture.jpg is tiled on a custom surface along a certain plane. The most important thing is to set three coordinates SetOrigin, SetPoint1 and SetPoint2 , which respectively represent the origin of the mapping and the directions of the two coordinate axes. The position of the origin and the orientation of the axis are different, and the direction of the mapped texture may also be inverted or mirrored. The three-dimensional coordinate law is hand-drawn here, and you can understand it:

Note that the origin position is often not (0, 0, 0) in the original coordinate system! , need to be determined according to the coordinates of your own graphics. Next, you can directly perform texture mapping. The pattern shown in texture.jpg will be mapped to the custom arc surface according to the plane direction and size of the above picture . As mentioned above, it is like pasting a protective film on a curved screen. ...The code is as follows, and the subsequent drawing steps are the same as above , so I won’t go into details :

reader = vtk.vtkJPEGReader()
reader.SetFileName(r'texture.jpg')

texture = vtk.vtkTexture()  # 定义一个纹理类
texture.SetInputConnection(reader.GetOutputPort())
texture.InterpolateOn()

texturemap = vtk.vtkTextureMapToPlane()
texturemap.SetInputData(contour.GetOutput())
texturemap.SetOrigin(76, 0, 0)
texturemap.SetPoint1(0, 76, 0)
texturemap.SetPoint2(76, 0, 80)

# 映射器
mapper = vtk.vtkPolyDataMapper()
# 映射器输入自定义曲面模型信息
mapper.SetInputConnection(texturemap.GetOutputPort())
# 这一句十分关键，不然后面的纹理映射可能会失败
mapper.ScalarVisibilityOff()

# 演员
actor = vtk.vtkActor()
# 演员添加映射器
actor.SetMapper(mapper)
#演员添加纹理
actor.SetTexture(texture)

So far you're done, the 3D effect is as follows:

Texture Mapping 3D Effects

4. Summary

First put the code of the whole process for your reference:

import vtk
import numpy as np

# 半径
r = 76
# 极坐标系下构建顶点数据
theta = np.linspace(0, np.pi / 2, 61)
x = r * np.cos(theta)
y = r * np.sin(theta)
# vtkPoints格式的点
points = vtk.vtkPoints()
vertices = vtk.vtkCellArray()
i = 0
for z in range(80):
    for xi, yi in zip(x, y):
        points.InsertPoint(i, xi, yi, z)
        # vertices.InsertNextCell(1)
        # vertices.InsertCellPoint(i)
        i += 1

polyData = vtk.vtkPolyData()
polyData.SetPoints(points)
# polyData.SetVerts(vertices)
surf = vtk.vtkSurfaceReconstructionFilter()
# 输入polyData数据
surf.SetInputData(polyData)
surf.SetNeighborhoodSize(20)
# 此处SetSampleSpacing数值小精度高但运行时间长，数值大则反之
surf.SetSampleSpacing(1.0)
surf.Update()
# 轮廓信息
contour = vtk.vtkContourFilter()
contour.SetInputConnection(surf.GetOutputPort())
contour.SetValue(0, 0.0)
contour.Update()

reader = vtk.vtkJPEGReader()
reader.SetFileName(r'texture.jpg')
texture = vtk.vtkTexture()  # 定义一个纹理类
texture.SetInputConnection(reader.GetOutputPort())
texture.InterpolateOn()
texturemap = vtk.vtkTextureMapToPlane()
texturemap.SetInputData(contour.GetOutput())
texturemap.SetOrigin(76, 0, 0)
texturemap.SetPoint1(0, 76, 0)
texturemap.SetPoint2(76, 0, 80)

# 映射器
mapper = vtk.vtkPolyDataMapper()
# 映射器输入自定义曲面模型信息
mapper.SetInputConnection(texturemap.GetOutputPort())
# 这一句十分关键，不然后面的纹理映射可能会失败
mapper.ScalarVisibilityOff()

# 演员
actor = vtk.vtkActor()
# 演员添加映射器
actor.SetMapper(mapper)
actor.SetTexture(texture)

# 绘制
ren = vtk.vtkRenderer()
# 添加演员
ren.AddActor(actor)

#绘制窗口
renWin = vtk.vtkRenderWindow()
renWin.AddRenderer(ren)
renWin.SetSize(800, 800)

# 交互窗口
iren = vtk.vtkRenderWindowInteractor()
iren.SetRenderWindow(renWin)
iren.Initialize()
renWin.Render()
iren.Start()

You can modify the custom plane here to your own model, such as topographic map and other data. I have just come into contact with 3D knowledge, including texture mapping and coordinate relations, etc. I will simply share it, and the method is not perfect. Welcome to discuss and learn together !

Python calls the vtk library and numpy to draw custom surfaces and perform texture mapping