3DSEE (3D SEmantic Engine) is a 3D model semantic search engine based on AI technology. It can automatically extract the semantic information of 3D models and store it in the database to help users use natural Efficiently retrieve 3D models by language or keyword. 3DSEE provides a complete secondary development API. Whether using Java, Python, PHP, JavaScript or C#, you can integrate the semantic extraction, warehousing and natural language retrieval capabilities of 3D models in your own applications.
3DSEE official download address:3D model semantic search engine .
The main functions of 3DSEE include:
语义提取: 3DSEE can automatically analyze and understand the semantic information of the 3D model connotation, which allows users to achieve a comprehensive understanding of the model connotation without manually adding tags or metadata.数据入库: 3DSEE uniformly stores the extracted semantic information and related metadata in the database for fast and efficient retrieval. This provides users with a platform to centrally manage and organize 3D models, simplifying the complexity of data management.自然语言检索: Users can search 3D models through natural language or keywords. 3DSEE uses advanced natural language processing technology to enable users to obtain the required 3D models using simple language descriptions, improving the convenience of retrieval.
1. File and directory organization
3DSEE provides an efficient and accurate model retrieval experience by understanding the semantic information input by users. The current version V1.0, the main file and directory organization structure is as follows:
| Development package files | illustrate |
|---|---|
| src/ | Execution file directory |
| src/models/ | AI model directory |
| src/attempt.pyc | Execute method file |
| src/db.pyc | data management files |
| src/main.pyc | Engine entry file |
| src/modules.pyc | Model call file |
| src/preview.pyc | Preview image generation file |
| src/config.json | Configuration file |
| src/requirements.txt | Dependencies that need to be installed |
| installation.md | pytorch3D installation guide |
| LICENSE.md | Development kit license agreement file |
2. Environment installation and deployment
First refer to installation.md to install the basic operating environment.
2.1 Install dependency packages
Open cmd in the src directory and execute the following command to install dependent packages
pip install requirements.txt -r
2.2 Modify configuration file
Modify configuration in config.json
{
"port": 3005, //服务端口号
"host": "127.0.0.1", //服务地址
"dbconfig": {
"name": "default", //数据库名
"host": "192.168.1.139", //数据库地址
"port": 19530, //数据库端口
"collection": "my_models", //集合名，首次启动会自动创建集合
"load_timeout": 30 //加载集合时限
}
}
2.3 Start the service
Open cmd in the src directory and execute the following command to run the service
python -m main
After starting the service, you can visit http://127.0.0.1:3005/docs to view the api document (please refer to the configuration file for the actual port and IP)
3. How to use the sample program
After 3DSEE is started, run the following command in the example directory to start the example program:
npm install
npm run dev
The sample program provides model retrieval and warehousing functions.
3.1 Keyword query
Enter keywords, click the search button to start the query, and return the query results. For example, enter the keywordcar and the query results are as follows:
3.2 Model storage
Fill out the form and click Submit to add the model to the model library.
4. API usage instructions
The current document version number is v1.0, and the update date is 2023/11/29.
The service package provides 2 access interfaces, the list is as follows.
| Interface name | visit url | ask |
|---|---|---|
| Query model | / | GET |
| Add model | / | POST |
4.1 Model Query API
The general call [query model] interface allows you to query models related to their semantics through Chinese or English keywords. The interface is described in detail as follows:
Visit URL: GET /
Request parameters:The main fields of the query parameters are described as follows:
keyword: Required, keyword, such as: carpage: Number of query pages, default value: 1limit: The number returned per page, default value: 10radius: Minimum similarity threshold. The larger the value, the larger the semantic similarity matching range. Default value: 1.
An example of a request is as follows (using curl):
curl http://127.0.0.1:3005/?keyword=car&page=1&limit=10&radius=3
Response result:JSON object, the main fields are described as follows:
id: unique iddistance: Determine the minimum similarity threshold, type: numberentity: json object, the main fields are as follows:data: 3D model custom data, JSON stringpreview: Model preview url, stringpath: 3D model access path, string
Examples of response results are as follows:
[
{
id: 445859173072122400,
distance: 1.2356822490692139,
entity: {
data: '{"name": "r2"}',
path: "D:/preview/models/r2.off",
preview: "http://127.0.0.1:3005/preview/81ac5d31-c11e-45e2-aa94-c39eddf3f459.png",
},
},
{
id: 445946831156557600,
distance: 0.7618597269058228,
entity: {
data: '{"name": "car"}',
path: "http://127.0.0.1:3005/preview/67d8abcf-176c-441c-abb5-3712750bd33c.png",
},
},
];
4.2 Model addition API
Commonly called [Add Model] interface, you can add models to the library. The interface is described in detail as follows:
访问URL： POST /
Request parameters:JSON object, the main fields are described as follows:
data: json object, custom datapath: String, local path of the model, make sure it is accessible,modelType: String, the model type to be added, currently supported: 'obj', 'off', ply', 'glb'
An example request is as follows (using curl):
curl -X POST -D '{"data": {}, "path": "./model.obj", "modelType": "obj"}' http://127.0.0.1:3005/
Response result: Returns a string indicating the operation result information, for example:
"保存成功"Original link:3D model semantic search engine - BimAnt