I am mainly engaged in the geographic information industry. At present, I mainly research and explore technologies related to the construction of GIS vertical fields such as smart cities, digital twins, smart natural resources, and real-world 3D China. I have certain experience in application system development, architecture, and operation and maintenance. , specialized in Xinchuang industry for more than 5 years. In the course of more than 5 years of work, I will often deal with massive spatial data, and the management and visualization of these massive spatial data will definitely use spatial databases.
This article will combine some experience in work and industry to introduce the YuKon (Yugong) database based on Huawei openGauss jointly launched by SuperMap and Huawei, mainly focusing on the dependence of GIS on spatial databases, the era of real China and digital twin construction Background, the past and present of the YuKon database, its architecture and usage, the role YuKon can play in digital twins and real-world China construction, and finally imagine the future of the spatio-temporal database. I am very happy to take this opportunity to communicate with colleagues here. This article is written in a hurry. Friends are welcome to criticize and correct.
1. Background
1. Digital twin
GIS has been used for many years for high-precision simulation of real-world systems. GIS representations of utility networks, legal parcel boundaries, and transportation networks are used for asset management, maintenance, and planning. Computer information technology, including data warehouses, federated systems, object-oriented data models, combined with virtual reality, interactive user experience, high-precision three-dimensional and four-dimensional models of assets and systems, on top of which comes the concept of digital twins .
In the digital twin, the real world is restored 1:1. These three-dimensional models, such as city models, road models, and models of various urban elements, will generate corresponding model data, which in turn requires storage requirements for model data.
2. Real scene 3D China
The construction of real scene 3D China is facing the new orientation and new demand of the surveying and mapping geographic information business serving economic and social development and ecological civilization construction in the new era, and transforming and upgrading the traditional basic surveying and mapping business. 14th Five-Year Plan for Natural Resources Protection and Utilization. This undoubtedly places great demands on the spatial processing capabilities in the new era.
3. Requirements for autonomous controllability
Geographical information is an important basic and strategic resource of the country, which is directly related to national sovereignty, security and interests, and carries resources, environment, population and other economic construction and social development; geographic information is also an important part of modern military struggle, ensuring Military activities and national defense security. The country needs to hold its autonomy in its own hands and not be controlled by others, and the basic software support capabilities must also be in its hands.
2. YuKon (Yu Gong) past and present
Whether it is based on the needs of digital twins, or the needs of service and real-world China construction, or from the three perspectives of improving the self-controllability of the basic capabilities of the GIS industry. The demand for domestic spatial database is very urgent.
1. What is YuKon?
Yukon expands the storage and management capabilities of geospatial data based on the openGauss database, provides professional GIS (Geographic Information System) functions, and empowers traditional relational databases. Led by SuperMap, in collaboration with massive data, cloud and Enmo, and in-depth cooperation in the openGauss community, it hatched the first 2D and 3D integrated spatial database based on openGauss. In the future, as the community continues to gather innovative forces from the industry, more databases for new scenarios will be incubated. Supports the management of common 2D and 3D data such as points of interest, residential areas, roads, water systems, highway networks, building information, model information, BIM, CIM, underground pipelines, etc., as shown in the following figure:
The YuKon database can manage these data very well, and provides a lot of functional support for spatial storage and analysis, which makes our application development even more powerful. For GIS, spatial database belongs to the research on the underlying technical level. Under the background of the general environment that generally pursues "external" nowadays, it is commendable to do a good job in the "inside" of GIS, so that GIS can really be both good-looking and easy to use . Huawei open-sourced GaussDB, and SuperMap also open-sourced all source codes of Yugon, both of which demonstrate the demeanor of a major manufacturer and are the crystallization of the wisdom of the two major manufacturers.
2. What does YuKon have?
Currently, Yukon's extended modules based on openGauss/PostgreSQL include:
- postgis: PostGIS vector module adapted to openGauss/PostgreSQL;
- postgis_raster: PostGIS raster module adapted to openGauss/PostgreSQL;
- postgis_sfcgal: PostGIS 3D algorithm related modules adapted to openGauss/PostgreSQL;
- yukon_geomodel: Yukon's own 3D model data module;
- yukon_geogridcoder: Yukon's own spatial grid coding module.
It can be seen that in terms of spatial processing, YuKon is extended based on PostGis, and the dependencies between modules are as shown in the figure below:
3. The past and present of YuKon
YuKon is based on Huawei's openGauss database. In September 2020, openGauss provided PostGIS adaptation as an external plug-in for spatial database management. In March 2021, driven by HyperMap, the community established a GIS SIG, openGauss/YuKon. From March 2021 to May 2022, the PostGIS version has been upgraded to expand the core capabilities of space management. In June 2022, SuperMap officially released the YuKon spatial database at the 5th GIS Software Technology Conference. YuKon open source library address: openGauss/Yukon
Huawei's openGauss database has always been in an iterative state, and version 3.0 has been released. Compared with the old version, the new version has the characteristics of lightweight, high performance, high availability, high security, and high intelligence. The compatibility and distributed capabilities of the structural database have been greatly enhanced. It is believed that the two are powerful alliances to jointly build a leading enterprise-level open source database system.
4. YuKon's restrictions on the operating system
Friendly reminder: Up to now, the YuKon4OpenGauss (Yugong) database is running on the Linux operating system, so if you want to try it out, please pay attention to this.
The operating systems supported by openGauss are: centos_x86_64, openeuler_aarch64, openeuler_x86_64, please install on these operating systems first.
If you want to try out the Yugong database, you can refer to the installation manual on the official website. It can also be built based on containers such as Docker.
5. YuKon Spatial Data Processing
a. Vector data management
Use the specified user to connect to the database and install space expansion support at the same time. The key codes are as follows:
# 连接到数据库(openGauss为例)
gsql -d yukontutorial -U yukondemo -W gisdata@123
--创建 postgis 扩展
CREATE EXTENSION postgis;After connecting to the target database, create a spatial table and insert sample data. The key codes are as follows:
-- 创建带geometry列的表
CREATE TABLE global_points (id SERIAL PRIMARY KEY,name VARCHAR(64),location geometry(POINT,3857));
-- 写入10个点对象
INSERT INTO global_points (name, location) VALUES ('Town1', ST_GeomFromEWKT('SRID=3857;POINT (12057531.405152922 4096788.3009192282)') );
INSERT INTO global_points (name, location) VALUES ('Town2', ST_GeomFromEWKT('SRID=3857;POINT (12057879.323089447 4096794.06898925)') );
INSERT INTO global_points (name, location) VALUES ('Town3', ST_GeomFromEWKT('SRID=3857;POINT (12057241.150712628 4096835.404659481)') );
INSERT INTO global_points (name, location) VALUES ('Town4', ST_GeomFromEWKT('SRID=3857;POINT (12057396.37461059 4096885.372392862)') );
INSERT INTO global_points (name, location) VALUES ('Town5', ST_GeomFromEWKT('SRID=3857;POINT (12056940.710538926 4096886.9123589676)') );
INSERT INTO global_points (name, location) VALUES ('Town6', ST_GeomFromEWKT('SRID=3857;POINT (12058128.24460281 4096938.23117457)') );
INSERT INTO global_points (name, location) VALUES ('Town7', ST_GeomFromEWKT('SRID=3857;POINT (12058378.134595742 4097081.4360874156)') );
INSERT INTO global_points (name, location) VALUES ('Town8', ST_GeomFromEWKT('SRID=3857;POINT (12058636.607321415 4097235.1257181372)') );
INSERT INTO global_points (name, location) VALUES ('Town9', ST_GeomFromEWKT('SRID=3857;POINT (12058917.488660585 4097395.7746241256)') );
INSERT INTO global_points (name, location) VALUES ('Town10', ST_GeomFromEWKT('SRID=3857;POINT (12059180.102471316 4097548.7255362184)') );b. Raster data management
Use the specified user to connect to the database and install the spatial raster extension support at the same time. The key codes are as follows:
# 连接到数据库(openGauss为例)
gsql -d yukontutorial -U yukondemo -W gisdata@123
--创建 postgis_raster 扩展
CREATE EXTENSION postgis_raster;After connecting to the target database, create a spatial table and insert sample data. The key codes are as follows:
raster2pgsql -s 0 /SampleData/hillshade.tif -t 256x256 | gsql -d yukontutorial3. Empowering digital twins to help the construction of Real-Scene China
1. The ability of YuKon
YuKon provides integrated storage, computing and management capabilities. Support desktop products such as SuperMap or QGis for data management.
2. The three-dimensional data organization of YuKon
a. Object organization structure
The 3D model object is stored as a GeoModel type, which is composed of a model object (ModelNode) with a local coordinate system and its placed position, posture and other information. The object organization structure is shown in the figure below:
ModelNode is composed of fine layer and LOD layer (represented by PagedLOD, optional) data, and the basic unit of fine layer and LOD layer is PagedPatch; each PagedPatch contains multiple Geodes; Geode is a data package, composed of entity objects (ModelElement ), through the matrix on Geode, the same entity can be placed in different positions to realize instantiated storage of model data. Subclasses of ModelElement include skeleton (ModelSkeleton), material (ModelMaterial) and texture (ModelTexture).
b. Storage strategy
In terms of storage strategy, GeoModel is stored in the main table, and Geode only stores the name of the entity object; the entity object is stored separately in the sub-table of the dataset, and the 64-bit HashCode code based on the name of the entity object is used as the ID of the object.
Use this mode to efficiently manage and query 3D model data.
3. Spatial function support
The foundation of the YuKon database is to extend the PostGIS database, so it has complete support for PostGIS spatial functions, and some function extensions have been made.
For the planning industry, for users who need to accurately calculate the spherical area, the data type of the ellipse arc is extended.
4. GeoSOT coding basic ability
Visualization tools: DBeaver, QGIS or SuperMap iDesktopX; this article uses SuperMap iDesktopX to show the 3D scene effect. Calculate geosotgrid for the geometry column of two-dimensional point, line, and area data, and convert it to a geometry object; sample data: building_point_demo, building_line_demo, building_region_demo
---1. 对二维点对象编码
---1.1 创建表存储二维网格结果：building_point_grid22
create table building_point_grid22
(smid serial4 not null ,
smgeometry geometry(polygon, 4490),
asText varchar(765),
grid GeoSOTGrid,idGeo int4) ;
---1.2 为 building_point_demo 表的smgeometry列构造22层网格，并将格网转换成geometry对象，写入
with a as (select smid,ST_GeoSOTGrid(smgeometry,22) as grids from building_point_demo )
insert into building_point_grid22(smgeometry,asText,grid,idGeo)
select ST_GeomFromGeoSOTGrid(unnest(grids)),ST_AsText(unnest(grids)),unnest(grids),smid from a;
---2. 对二维线对象编码
---2.1 创建表存储二维网格结果：building_line2d_grid22
create table building_line2d_grid22
(smid serial4 not null ,
smgeometry geometry(polygon, 4490),
asText varchar(765),
grid GeoSOTGrid,idGeo int4) ;
---2.2 为 builing_line_demo 表的smgeometry列构造22层网格，并将格网转换成geometry对象，写入
with a as (select smid,ST_GeoSOTGrid(smgeometry,22) as grids from builing_line_demo )
insert into building_line2d_grid22(smgeometry,asText,grid,idGeo)
select ST_GeomFromGeoSOTGrid(unnest(grids)),ST_AsText(unnest(grids)),unnest(grids),smid from a;
---3. 对二维面对象编码
---3.1 创建表存储二维网格结果：building_region2d_grid22
create table building_region2d_grid22
(smid serial4 not null,
smgeometry geometry(polygon, 4490),
asText varchar(765),
grid GeoSOTGrid,idGeo int4);
---3.2 为 building_region_demo 表的 smgeometry 列构造22层网格，并将格网转换成geometry对象，写入
with a as (select smid,ST_GeoSOTGrid(smgeometry,22) as grids from building_region_demo )
insert into building_region2d_grid22(smgeometry,asText,grid,idGeo)
select ST_GeomFromGeoSOTGrid(unnest(grids)),ST_AsText(unnest(grids)),unnest(grids),smid from a;The newly generated building_point_grid22, building_line2d_grid22 and building_region2d_grid22 table visualization effects are as follows:
4. The Future of Spatial Databases
Spatial data focuses on the refinement and precise management of cities, serves all walks of life, and promotes the continuous development of our country. Every city and every individual generates a large amount of data every moment, many of which are accompanied by spatiotemporal data. Focusing on the efficient management of spatio-temporal data is a major topic at present and in the future.
With massive spatial data, there is also an efficient visualization engine. After quickly tiling the massive data in the database, render it in the front-end browser. Taking vector big data visualization as an example, it is a slice-free data model designed to quickly display large-scale vector data (above tens of millions). In the database, one SQL statement can be used to construct a vector pyramid, and at the same time complete the data thinning, simplification and tile caching. It is also possible to obtain the required tiles through one SQL statement: output standard mvt-pbf pure data according to wmts rules Vector tiles or output png rasterized vector tiles on demand. These still need to continue to evolve in the future technological development process.
The planning of the subsequent version of Yugong Spatial Database: First, support more spatial data models, such as point cloud, trajectory and other data; open source more spatial data expansion models, so that more users can use them; Second, continue to Collect user experience feedback based on the community, and will continue to improve peripheral business requirements around the core storage and management system of the current spatial database, such as the development of tools such as data migration, task progress, and resource monitoring, to help users use the product better ; Third, HyperMap has partnered with a number of database vendors with the openGauss community, and will further increase the integration of related resources in the future to achieve cross-border integration and innovation, and work together to improve the ecology.
V. Summary
That's all for this article. This article will combine some experience in work and industry to introduce the YuKon (Yugong) database based on Huawei openGauss jointly launched by SuperMap and Huawei, mainly focusing on the dependence of GIS on spatial databases, the era of real China and digital twin construction Background, the past and present of the YuKon database, its architecture and usage, the role YuKon can play in digital twins and real-world China construction, and finally imagine the future of the spatio-temporal database. I am very happy to take this opportunity to communicate with colleagues here. Since this article is written in a hurry, all friends are welcome to criticize and correct.
References for this article:
1. Product introduction of Yukon product website — Yukon 1.0 documentation
2. Zhou Qin-Yukon: A new open source 2D and 3D integrated spatial database.pdf
3. Hu Zhengce-YuKon empowers spatial intelligence and jointly builds the openGauss open source database ecosystem.pdf
5. Spatial database open source road, hypergraph + openGauss Fengqi Yugong