Godot 4 source code analysis - dynamically import image files

Trying to compile an e-book software with Godot 4, the initial effect has come out, and through the pipeline communication interface, you can get, set attributes, and call functions, and it seems to be able to handle various matters.

In fact, external factors work through internal factors. If the inside is not understood and the functions are not released, some needs cannot be realized.

For example, if you want to dynamically load new books now, this is a practical requirement. If each e-book needs to be exported once, then this software is not universal.

When loading images before, the GDScript code:

# 目标对象上加载图片
func loadPng(obj, pngFileName) -> bool:
	var texture = load(pngFileName) as Texture2D;
	if(texture != null):
		obj.set_texture(texture);
		obj.scale.y = get_viewport_rect().size.y / texture.get_height();
		obj.scale.x = obj.scale.y;
		obj.position.y = get_viewport_rect().size.y / 2;	# 垂直居中
		adjustPos();
		return true;
	return false;

Load another picture directly, the result is unsuccessful

 This has to be studied.

import import

Copy this picture to the Godot development environment, and found that when switching back to Godot, an import dialog box will flash quickly, so what Godot must have done

Go to the resource manager and see a corresponding .import file, open it and look at the content

[remap]

importer="texture"
type="CompressedTexture2D"
uid="uid://rv5gm15xbcaf"
path="res://.godot/imported/DrGraph_Page24.png-1dd935fbb11807a645ea1ea79ec38662.ctex"
metadata={
"vram_texture": false
}

[deps]

source_file="res://Pages/DrGraph_Page24.png"
dest_files=["res://.godot/imported/DrGraph_Page24.png-1dd935fbb11807a645ea1ea79ec38662.ctex"]

[params]

compress/mode=0
compress/high_quality=false
compress/lossy_quality=0.7
compress/hdr_compression=1
compress/normal_map=0
compress/channel_pack=0
mipmaps/generate=false
mipmaps/limit=-1
roughness/mode=0
roughness/src_normal=""
process/fix_alpha_border=true
process/premult_alpha=false
process/normal_map_invert_y=false
process/hdr_as_srgb=false
process/hdr_clamp_exposure=false
process/size_limit=0
detect_3d/compress_to=1

Since it is dynamically generated, it must be written in the code. Find it in the source code, where to start? Checking import and .import are a lot of results. Try content again, look for [remap], few results

 Look at the location and see that there are two functions _reimport_group and _reimport_file of the EditorFileSystem class, so _reimport_file must be used. But this function is not accessible. Fortunately, the source code is in hand, and it is directly public and accessible.

But this needs to be called to GDScript, and some work needs to be done, and it needs to be added to ClassDB.

For simplicity, add it directly to the DllStream class [See Godot 4 Source Code Analysis - Add Pipeline Communication_DrGraph's Blog - CSDN Blog ], add an import function

ClassDB::bind_method(D_METHOD("import", "fileName"), &DllStream::import);

String DllStream::import(String fileName) {
	if (FileAccess::exists(fileName + ".import") == false) 
		EditorFileSystem::get_singleton()->_reimport_file(fileName);
	return fileName;	
}

After running, it is still unsuccessful. Debugging found that _find_file(p_file, &fs, cpos) returns false. And _find_file actually requires the file to be located under the "res://" directory, that is to say, it must be in the project directory.

bool EditorFileSystem::_find_file(const String &p_file, EditorFileSystemDirectory **r_d, int &r_file_pos) const {
	//todo make faster

	if (!filesystem || scanning) {
		return false;
	}

	String f = ProjectSettings::get_singleton()->localize_path(p_file);

	if (!f.begins_with("res://")) {
		return false;
	}
	f = f.substr(6, f.length());
	f = f.replace("\\", "/");

	Vector<String> path = f.split("/");

	if (path.size() == 0) {
		return false;
	}
	String file = path[path.size() - 1];
	path.resize(path.size() - 1);

	EditorFileSystemDirectory *fs = filesystem;

	for (int i = 0; i < path.size(); i++) {
		if (path[i].begins_with(".")) {
			return false;
		}

		int idx = -1;
		for (int j = 0; j < fs->get_subdir_count(); j++) {
			if (fs->get_subdir(j)->get_name() == path[i]) {
				idx = j;
				break;
			}
		}

		if (idx == -1) {
			//does not exist, create i guess?
			EditorFileSystemDirectory *efsd = memnew(EditorFileSystemDirectory);

			efsd->name = path[i];
			efsd->parent = fs;

			int idx2 = 0;
			for (int j = 0; j < fs->get_subdir_count(); j++) {
				if (efsd->name.naturalnocasecmp_to(fs->get_subdir(j)->get_name()) < 0) {
					break;
				}
				idx2++;
			}

			if (idx2 == fs->get_subdir_count()) {
				fs->subdirs.push_back(efsd);
			} else {
				fs->subdirs.insert(idx2, efsd);
			}
			fs = efsd;
		} else {
			fs = fs->get_subdir(idx);
		}
	}

	int cpos = -1;
	for (int i = 0; i < fs->files.size(); i++) {
		if (fs->files[i]->file == file) {
			cpos = i;
			break;
		}
	}

	r_file_pos = cpos;
	*r_d = fs;

	return cpos != -1;
}

This request is a bit excessive. However, there are two methods for program processing: one is to automatically copy the pictures to the project directory, and the other is to bypass this.

Just try one, toss a coin and choose the second on the tail.

Since you choose to bypass, then directly implement the EditorFileSystem::_reimport_file function, which saves the need to make this private function public. The so-called implementation is to copy all the code of EditorFileSystem::_reimport_file, and then change it:

Error DllStream::_import(String destFileName) {
	HashMap<StringName, Variant> params = HashMap<StringName, Variant>();	
	String importer_name; //empty by default though
	ResourceUID::ID uid = ResourceUID::INVALID_ID;
	Variant generator_parameters;

	Ref<ResourceImporter> importer = ResourceFormatImporter::get_singleton()->get_importer_by_extension(destFileName.get_extension());
	if (importer.is_null()) 
		ERR_FAIL_V_MSG(ERR_FILE_CANT_OPEN, "BUG: File queued for import, but can't be imported, importer for type '" + importer_name + "' not found.");

	//mix with default params, in case a parameter is missing

	List<ResourceImporter::ImportOption> opts;
	importer->get_import_options(destFileName, &opts);
	for (const ResourceImporter::ImportOption &E : opts) {
		if (!params.has(E.option.name)) { //this one is not present
			params[E.option.name] = E.default_value;
		}
	}

	if (ProjectSettings::get_singleton()->has_setting("importer_defaults/" + importer->get_importer_name())) {
		//use defaults if exist
		Dictionary d = GLOBAL_GET("importer_defaults/" + importer->get_importer_name());
		List<Variant> v;
		d.get_key_list(&v);

		for (const Variant &E : v) 
			params[E] = d[E];
	}

	//finally, perform import!!
	String base_path = ResourceFormatImporter::get_singleton()->get_import_base_path(destFileName);

	List<String> import_variants;
	List<String> gen_files;
	Variant meta;
	Error err = importer->import(destFileName, base_path, params, &import_variants, &gen_files, &meta);

	ERR_FAIL_COND_V_MSG(err != OK, ERR_FILE_UNRECOGNIZED, "Error importing '" + destFileName + "'.");

	//as import is complete, save the .import file

	Vector<String> dest_paths;
	{
		Ref<FileAccess> f = FileAccess::open(destFileName + ".import", FileAccess::WRITE);
		ERR_FAIL_COND_V_MSG(f.is_null(), ERR_FILE_CANT_OPEN, "Cannot open file from path '" + destFileName + ".import'.");

		//write manually, as order matters ([remap] has to go first for performance).
		f->store_line("[remap]");
		f->store_line("");
		f->store_line("importer=\"" + importer->get_importer_name() + "\"");
		int version = importer->get_format_version();
		if (version > 0) {
			f->store_line("importer_version=" + itos(version));
		}
		if (!importer->get_resource_type().is_empty()) {
			f->store_line("type=\"" + importer->get_resource_type() + "\"");
		}

		if (uid == ResourceUID::INVALID_ID) {
			uid = ResourceUID::get_singleton()->create_id();
		}

		f->store_line("uid=\"" + ResourceUID::get_singleton()->id_to_text(uid) + "\""); //store in readable format

		if (err == OK) {
			if (importer->get_save_extension().is_empty()) {
				//no path
			} else if (import_variants.size()) {
				//import with variants
				for (const String &E : import_variants) {
					String path = base_path.c_escape() + "." + E + "." + importer->get_save_extension();

					f->store_line("path." + E + "=\"" + path + "\"");
					dest_paths.push_back(path);
				}
			} else {
				String path = base_path + "." + importer->get_save_extension();
				f->store_line("path=\"" + path + "\"");
				dest_paths.push_back(path);
			}

		} else {
			f->store_line("valid=false");
		}

		if (meta != Variant()) {
			f->store_line("metadata=" + meta.get_construct_string());
		}

		if (generator_parameters != Variant()) {
			f->store_line("generator_parameters=" + generator_parameters.get_construct_string());
		}

		f->store_line("");

		f->store_line("[deps]\n");

		if (gen_files.size()) {
			Array genf;
			for (const String &E : gen_files) {
				genf.push_back(E);
				dest_paths.push_back(E);
			}

			String value;
			VariantWriter::write_to_string(genf, value);
			f->store_line("files=" + value);
			f->store_line("");
		}

		f->store_line("source_file=" + Variant(destFileName).get_construct_string());

		if (dest_paths.size()) {
			Array dp;
			for (int i = 0; i < dest_paths.size(); i++) {
				dp.push_back(dest_paths[i]);
			}
			f->store_line("dest_files=" + Variant(dp).get_construct_string() + "\n");
		}

		f->store_line("[params]");
		f->store_line("");

		//store options in provided order, to avoid file changing. Order is also important because first match is accepted first.

		for (const ResourceImporter::ImportOption &E : opts) {
			String base = E.option.name;
			String value;
			VariantWriter::write_to_string(params[base], value);
			f->store_line(base + "=" + value);
		}
	}

	// Store the md5's of the various files. These are stored separately so that the .import files can be version controlled.
	{
		Ref<FileAccess> md5s = FileAccess::open(base_path + ".md5", FileAccess::WRITE);
		ERR_FAIL_COND_V_MSG(md5s.is_null(), ERR_FILE_CANT_OPEN, "Cannot open MD5 file '" + base_path + ".md5'.");

		md5s->store_line("source_md5=\"" + FileAccess::get_md5(destFileName) + "\"");
		if (dest_paths.size()) {
			md5s->store_line("dest_md5=\"" + FileAccess::get_multiple_md5(dest_paths) + "\"\n");
		}
	}
	if (ResourceUID::get_singleton()->has_id(uid)) {
		ResourceUID::get_singleton()->set_id(uid, destFileName);
	} else {
		ResourceUID::get_singleton()->add_id(uid, destFileName);
	}
}

String DllStream::import(String fileName) {
	if (FileAccess::exists(fileName + ".import") == false) 
		_import(fileName);
	return fileName;	
}

After running, it still failed, and then followed up and found that it was

importer->import(destFileName, base_path, params, &import_variants, &gen_files, &meta)

fail. Single-step debugging found that the importers of ResourceFormatImporter is empty. Finding its add_importer function from the source code will add a ResourceImporter. In the EditorNode::EditorNode() constructor, multiple ResourceImporters are added.

	{
		// Register importers at the beginning, so dialogs are created with the right extensions.
		Ref<ResourceImporterTexture> import_texture;
		import_texture.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_texture);

		Ref<ResourceImporterLayeredTexture> import_cubemap;
		import_cubemap.instantiate();
		import_cubemap->set_mode(ResourceImporterLayeredTexture::MODE_CUBEMAP);
		ResourceFormatImporter::get_singleton()->add_importer(import_cubemap);

		Ref<ResourceImporterLayeredTexture> import_array;
		import_array.instantiate();
		import_array->set_mode(ResourceImporterLayeredTexture::MODE_2D_ARRAY);
		ResourceFormatImporter::get_singleton()->add_importer(import_array);

		Ref<ResourceImporterLayeredTexture> import_cubemap_array;
		import_cubemap_array.instantiate();
		import_cubemap_array->set_mode(ResourceImporterLayeredTexture::MODE_CUBEMAP_ARRAY);
		ResourceFormatImporter::get_singleton()->add_importer(import_cubemap_array);

		Ref<ResourceImporterLayeredTexture> import_3d;
		import_3d.instantiate();
		import_3d->set_mode(ResourceImporterLayeredTexture::MODE_3D);
		ResourceFormatImporter::get_singleton()->add_importer(import_3d);

		Ref<ResourceImporterImage> import_image;
		import_image.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_image);

		Ref<ResourceImporterTextureAtlas> import_texture_atlas;
		import_texture_atlas.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_texture_atlas);

		Ref<ResourceImporterDynamicFont> import_font_data_dynamic;
		import_font_data_dynamic.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_font_data_dynamic);

		Ref<ResourceImporterBMFont> import_font_data_bmfont;
		import_font_data_bmfont.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_font_data_bmfont);

		Ref<ResourceImporterImageFont> import_font_data_image;
		import_font_data_image.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_font_data_image);

		Ref<ResourceImporterCSVTranslation> import_csv_translation;
		import_csv_translation.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_csv_translation);

		Ref<ResourceImporterWAV> import_wav;
		import_wav.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_wav);

		Ref<ResourceImporterOBJ> import_obj;
		import_obj.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_obj);

		Ref<ResourceImporterShaderFile> import_shader_file;
		import_shader_file.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_shader_file);

		Ref<ResourceImporterScene> import_scene;
		import_scene.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_scene);

		Ref<ResourceImporterScene> import_animation;
		import_animation = Ref<ResourceImporterScene>(memnew(ResourceImporterScene(true)));
		ResourceFormatImporter::get_singleton()->add_importer(import_animation);

		{
			Ref<EditorSceneFormatImporterCollada> import_collada;
			import_collada.instantiate();
			ResourceImporterScene::add_importer(import_collada);

			Ref<EditorOBJImporter> import_obj2;
			import_obj2.instantiate();
			ResourceImporterScene::add_importer(import_obj2);

			Ref<EditorSceneFormatImporterESCN> import_escn;
			import_escn.instantiate();
			ResourceImporterScene::add_importer(import_escn);
		}

		Ref<ResourceImporterBitMap> import_bitmap;
		import_bitmap.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_bitmap);
	}

EditorNode will be automatically created in editor mode, but for the final running program, there is no ProjectManager and no EditorNode, so if you want to import it, you need to create it yourself.

From the image import process, we can see that the e-book only needs to import png images, which are in Texture format, so only one type can be imported.

		Ref<ResourceImporterTexture> import_texture;
		import_texture.instantiate();
		ResourceFormatImporter::get_singleton()->add_importer(import_texture);

Run, the picture is displayed successfully.

The next step of the job

At this point, the core work is completed, and there are two main tasks for the next step:

One is directory import, that is, you can import all the files in the directory where the pictures of the target e-book are located at one time

The second is forced dynamic import. Non-mandatory import means that as long as there is a corresponding .import file, it does not need to be imported again; mandatory import means that no matter whether the .import file exists or not, it is imported. This is used for dynamic processing. For example, after searching for keywords in the file, the keywords need to be highlighted, resulting in different pictures. These pictures can be placed in the dynamic directory, and they can be forced to import.