[Record] USSOCOM Urban3D data set reading and processing

Introduction to Urban3D dataset content

The Urban3D data set images are orthographic RGB images with a resolution of 50cm.
Use aws to download data from SpaceNet . The folder structure is:

|- 01-Provisional_Train
	|- GT
		|- GT中包含GTC，GTI，GTL.tif文件，GTL为ground truth building footprint。
	|- Inputs
		|- Inputs中包含DSM，DTM，RGB.tif文件，DSM为Digital Surface Models，DTM为Digital Terrain Models，normalized DSM (nDSM = DSM - DTM)
|- 02-Provisional_Test
|- 03-Sequestered_Test
|- 04-Unused_Data
|- AOI_polygons
|- Pretrained_Models
   |- 包含前6名参赛团队的模型

The size of each .tif is 2048*2048.

Urban3D dataset reading

from torchvision import transforms
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
import sys
rgb = Image.open("/01-Provisional_Train/Inputs/JAX_Tile_016_RGB.tif")
dsm = Image.open("/01-Provisional_Train/Inputs/JAX_Tile_016_DSM.tif")
gtl = Image.open("/01-Provisional_Train/GT/JAX_Tile_016_GTL.tif")
print(rgb.size, dsm.size, gtl.size) >> (2048, 2048) (2048, 2048) (2048, 2048)
print(np.array(rgb).shape, np.array(dsm).shape, np.array(gtl).shape) >> (2048, 2048, 3) (2048, 2048) (2048, 2048)
print(np.array(rgb).dtype, np.array(dsm).dtype,  np.array(gtl).dtype) >> uint8 float32 uint8

fig = plt.figure()
plt.subplot(131)
plt.imshow(np.array(rgb))
plt.subplot(132)
plt.imshow(np.array(dsm))
plt.subplot(133)
plt.imshow(np.array(gtl))

print(np.max(gtl), np.min(gtl)) >> 6, 2

Urban3D dataset Pytorch processing

Using deep learning requires clipping of Urban3D data, which is used here torchvision.transforms.RandomCrop. RandomCropIt can be directly applied to files opened by PIL.Image and torch type data, but it cannot be applied to numpy arrays. For specific content, please refer to numpy, PIL, tensor type used in torchvision.transforms

def type_convert(x):
    x_ = np.array(x).astype(np.float32)
    return x_
def to_tensor(x):
    x_ = np.expand_dims(x, axis=0)
    x_ = torch.from_numpy(x_)
    return x_
trans = transforms.Compose([
    transforms.RandomCrop(size=256),
    transforms.Lambda(type_convert),
    transforms.Lambda(to_tensor)
])

torch.random.manual_seed(16)
rgb_crop = trans(rgb)
torch.random.manual_seed(16)
dsm_crop = trans(dsm)
torch.random.manual_seed(16)
gtl_crop = trans(gtl)
print(rgb_crop.size(), dsm_crop.size(), gtl_crop.size())
fig = plt.figure()
plt.subplot(131)
plt.imshow(rgb_crop[0,:,:,0])
plt.subplot(132)
plt.imshow(dsm_crop[0,:,:])
plt.subplot(133)
plt.imshow(gtl_crop[0,:,:])

At the same time, one thing that needs to be noted is that in order to ensure the consistency of data and label after RandomCrop, a random number seed needs to be set. For GTL, the value of label needs to be changed.

def type_convert_gtl(x):
    x_ = np.array(x).astype(np.float32)
    x_[x_==6]=1
    x_[x_==2]=0
    return x_