Generating LoD2 3D city models using bimodal deep learning networks

Abstract

In today's world, cities are rapidly expanding, and their populations are constantly increasing. According to the United Nations' population division report, 55% of the world's population lives in urban areas. This proportion is expected to increase to 68% by 2050. As a result, there is an increasing demand for accurate, precise 3D city models to support the sustainable management of the growing urban population.

To meet this growing demand, this dissertation aims to develop techniques to generate 3D building models, which ultimately lead to a city model based on the CityGML 3.0 * standard. This standard defines a conceptual model and exchange format for the representation, storage, and exchange of virtual 3D city models. According to this standard, 3D building models can be generated at four Levels of Detail (LoD), which progressively increase the level of complexity and accuracy. In LoD0, buildings have a 2D representation of their footprints, and in LoD1, they are modeled as block-shaped. In LoD2, buildings have the roof structure, and, finally, detailed architectural elements such as windows, doors, and a full exterior are included in LoD3.

This research focuses on the reconstruction of LoD2 building models. Generating these models automatically from optical imagery is challenging because roof structures and types must be correctly identified, while orthophotos often suffer from low contrast and shadows. Height data can compensate for these limitations; however, optical and height datasets are frequently spatially misaligned, which complicates reconstruction.

The primary objective of this research is to develop a method towards automated LoD2 building reconstruction from bimodal data (RGB imagery and DSM) by detecting roof structural components and types, and converting them into geometrically consistent 3D models to be able to model all types of buildings.

To achieve this, the proposed methodology employs Artificial Intelligence (AI), particularly Deep Learning (DL) techniques, with a focus on bimodal data and feature fusion, and addressing data misalignment. These models are applied to high-resolution aerial imagery and LiDAR point cloud data to extract building roof types and geometric components, thereby generating large-scale 3D modeling of buildings. The resulting models contribute towards the creation of an automated digital twin framework for urban environments.

* https://www.ogc.org/standards/citygml/