Table of contents
Packaging and resource loading framework directory
text
Before starting the text, let’s put the packaging code here. Please note that the previous code has been omitted. Compare it to the previous article yourself. This article starts with executing the packaging code for the first time.
public void PostAssetBuild()
{
//前面的代码省略，和上一篇文章一致
Log($"开始构建......");
BuildAssetBundleOptions opt = MakeBuildOptions();
AssetBundleManifest buildManifest = BuildPipeline.BuildAssetBundles(OutputPath, buildInfoList.ToArray(), opt, BuildTarget);
if (buildManifest == null)
throw new Exception("[BuildPatch] 构建过程中发生错误！");
//本篇的代码从这开始==============================================
// 清单列表
string[] allAssetBundles = buildManifest.GetAllAssetBundles();
Log($"资产清单里总共有{allAssetBundles.Length}个资产");
//create res manifest
var resManifest = CreateResManifest(buildMap, buildManifest);
var manifestAssetInfo = new AssetInfo(AssetDatabase.GetAssetPath(resManifest));
var label = "Assets/Manifest";
manifestAssetInfo.ReadableLabel = label;
manifestAssetInfo.AssetBundleVariant = PatchDefine.AssetBundleDefaultVariant;
manifestAssetInfo.AssetBundleLabel = HashUtility.BytesMD5(Encoding.UTF8.GetBytes(label));
var manifestBundleName = $"{manifestAssetInfo.AssetBundleLabel}.{manifestAssetInfo.AssetBundleVariant}".ToLower();
_labelToAssets.Add(manifestBundleName, new List<AssetInfo>() { manifestAssetInfo });
//build ResManifest bundle
buildInfoList.Clear();
buildInfoList.Add(new AssetBundleBuild()
{
assetBundleName = manifestAssetInfo.AssetBundleLabel,
assetBundleVariant = manifestAssetInfo.AssetBundleVariant,
assetNames = new[] { manifestAssetInfo.AssetPath }
});
var resbuildManifest = BuildPipeline.BuildAssetBundles(OutputPath, buildInfoList.ToArray(), opt, BuildTarget);
//加密代码省略，后面文章讲解
}After calling BuildPipeline.BuildAssetBundles packaging API for the first time (see the seventh line of the code for details), a reference to AssetBundleManifest will be returned.
[Question]:BuildPipeline.BuildAssetBundles packaging API has already helped us create dependency references between AB packages. Why do we need to create reference relationships between AB packages?
[Answer]:The UnityManifest.manifest file generated after executing the BuildPipeline.BuildAssetBundles packaging API records allAB package information and dependencies. but! Enterprise-level project packaging must consider incremental packaging, so we want to know which version of each AB is typed, and we need a mark, such as recording that the AB package is typed from a certain stage of SVN. Therefore, the UnityManifest.manifest file generated by the packaging interface is a semi-finished product.
Let’s start with a formal introduction to the secondary processing of the UnityManifest.manifest file.
string[] allAssetBundles = buildManifest.GetAllAssetBundles(); Get allAssetBundles and use the CreateResManifest method to create a Unity Asset file, and serialize the few data in UnityManifest.manifest into the asset file. The serialization script of asset is ResManifes, as shown below
Secondary processing of UnityManifest.manifest fileThe code is as follows:
//assetList在前面的打包代码里有
//buildManifest第一次打包API返回的文件
private ResManifest CreateResManifest(List<AssetInfo> assetList , AssetBundleManifest buildManifest)
{
string[] bundles = buildManifest.GetAllAssetBundles();
var bundleToId = new Dictionary<string, int>();
for (int i = 0; i < bundles.Length; i++)
{
bundleToId[bundles[i]] = i;
}
var bundleList = new List<BundleInfo>();
for (int i = 0; i < bundles.Length; i++)
{
var bundle = bundles[i];
var deps = buildManifest.GetAllDependencies(bundle);
var hash = buildManifest.GetAssetBundleHash(bundle).ToString();
var encryptMethod = ResolveEncryptRule(bundle);
bundleList.Add(new BundleInfo()
{
Name = bundle,
Deps = Array.ConvertAll(deps, _ => bundleToId[_]),
Hash = hash,
EncryptMethod = encryptMethod
});
}
var assetRefs = new List<AssetRef>();
var dirs = new List<string>();
foreach (var assetInfo in assetList)
{
if (!assetInfo.IsCollectAsset) continue;
var dir = Path.GetDirectoryName(assetInfo.AssetPath).Replace("\\", "/");
CollectionSettingData.ApplyReplaceRules(ref dir);
var foundIdx = dirs.FindIndex(_ => _.Equals(dir));
if (foundIdx == -1)
{
dirs.Add(dir);
foundIdx = dirs.Count - 1;
}
var nameStr = $"{assetInfo.AssetBundleLabel}.{assetInfo.AssetBundleVariant}".ToLower();
assetRefs.Add(new AssetRef()
{
Name = Path.GetFileNameWithoutExtension(assetInfo.AssetPath),
BundleId = bundleToId[$"{assetInfo.AssetBundleLabel}.{assetInfo.AssetBundleVariant}".ToLower()],
DirIdx = foundIdx
});
}
var resManifest = GetResManifest();
resManifest.Dirs = dirs.ToArray();
resManifest.Bundles = bundleList.ToArray();
resManifest.AssetRefs = assetRefs.ToArray();
EditorUtility.SetDirty(resManifest);
AssetDatabase.SaveAssets();
AssetDatabase.Refresh();
return resManifest;
}
Here is the code to serialize the data:
/// <summary>
/// design based on Google.Android.AppBundle AssetPackDeliveryMode
/// </summary>
[Serializable]
public enum EAssetDeliveryMode
{
// ===> AssetPackDeliveryMode.InstallTime
Main = 1,
// ====> AssetPackDeliveryMode.FastFollow
FastFollow = 2,
// ====> AssetPackDeliveryMode.OnDemand
OnDemand = 3
}
/// <summary>
/// AssetBundle打包位置
/// </summary>
[Serializable]
public enum EBundlePos
{
/// <summary>
/// 普通
/// </summary>
normal,
/// <summary>
/// 在安装包内
/// </summary>
buildin,
/// <summary>
/// 游戏内下载
/// </summary>
ingame,
}
[Serializable]
public enum EEncryptMethod
{
None = 0,
Quick, //padding header
Simple,
X, //xor
QuickX //partial xor
}
[Serializable]
[ReadOnly]
public struct AssetRef
{
[ReadOnly, EnableGUI]
public string Name;
[ReadOnly, EnableGUI]
public int BundleId;
[ReadOnly, EnableGUI]
public int DirIdx;
}
[Serializable]
public enum ELoadMode
{
None,
LoadFromStreaming,
LoadFromCache,
LoadFromRemote,
}
[Serializable]
public struct BundleInfo
{
[ReadOnly, EnableGUI]
public string Name;
[ReadOnly, EnableGUI]
[ListDrawerSettings(Expanded=false)]
public int[] Deps;
[ReadOnly]
public string Hash;
[ReadOnly]
public EEncryptMethod EncryptMethod;
// public ELoadMode LoadMode;
}
public class ResManifest : ScriptableObject
{
[ReadOnly, EnableGUI]
public string[] Dirs = new string[0];
[ListDrawerSettings(IsReadOnly = true)]
public AssetRef[] AssetRefs = new AssetRef[0];
[ListDrawerSettings(IsReadOnly = true)]
public BundleInfo[] Bundles = new BundleInfo[0];
}
}As you can see from the picture, the CreateResManifest method creates our own set of resource and AB package index relationships.
ResManifes serialization (code below) file stores 3 types of data,
List of all resource folders
AB package List number where the resource is located, List number of the folder where the resource is located
The name of the AB package, the name of the dependent package, the version number MD5, and the encryption type.
[Question]: Why do you need to serialize this asset file?
Before answering the question, let me ask a question: Resource loading is definitely for developers. How do developers find the ab package in which the desired resource is located?
[Answer]: When the project starts, we need to use this asset file to create a reference information for all resources. After the project starts, we need to load this asset. The loading code is as follows .
protected virtual ResManifest LoadResManifest()
{
string label = "Assets/Manifest";
var manifestBundleName = $"{HashUtility.BytesMD5(Encoding.UTF8.GetBytes(label))}.unity3d";
string loadPath = GetAssetBundleLoadPath(manifestBundleName);
var offset = AssetSystem.DecryptServices.GetDecryptOffset(manifestBundleName);
var usingFileSystem = GetLocation(loadPath) == AssetLocation.App
? FileSystemManagerBase.Instance.MainVFS
: FileSystemManagerBase.Instance.GetSandboxFileSystem(PatchDefine.MainPackKey);
if (usingFileSystem != null)
{
offset += usingFileSystem.GetBundleContentOffset(manifestBundleName);
}
AssetBundle bundle = AssetBundle.LoadFromFile(loadPath, 0, offset);
if (bundle == null)
throw new Exception("Cannot load ResManifest bundle");
var manifest = bundle.LoadAsset<ResManifest>("Assets/Manifest.asset");
if (manifest == null)
throw new Exception("Cannot load Assets/Manifest.asset asset");
for (var i = 0; i < manifest.Dirs.Length; i++)
{
var dir = manifest.Dirs[i];
_dirToIds[dir] = i;
}
for (var i = 0; i < manifest.Bundles.Length; i++)
{
var info = manifest.Bundles[i];
_bundleMap[info.Name] = i;
}
foreach (var assetRef in manifest.AssetRefs)
{
var path = StringFormat.Format("{0}/{1}", manifest.Dirs[assetRef.DirIdx], assetRef.Name);
// MotionLog.Log(ELogLevel.Log, $"path is {path}");
if (!_assetToBundleMap.TryGetValue(assetRef.DirIdx, out var assetNameToBundleId))
{
assetNameToBundleId = new Dictionary<string, int>();
_assetToBundleMap.Add(assetRef.DirIdx, assetNameToBundleId);
}
assetNameToBundleId.Add(assetRef.Name, assetRef.BundleId);
}
bundle.Unload(false);
return manifest;
}Looking at the code above, you can see that this asset file is also included in the bundle and is a separate ab package. Take another look at the title of this article: "Using Manifest to build resource index twice". Then, the bundle where this asset is located is the core of this article! ! !
Let’s talk about how developers load resources in the project. First, the developer will call a Loader to load resources. If the AB package loading mode is used (local resource loading is not discussed), then a resource path will be passed in, and Loading success callback
Loader.Load("Assets/Works/Resource/Sprite/UIBG/bg_lihui",callbackFunction)
//成功后回调
void callbackFunction(资源文件)
{
//使用资源文件
}We know that this resource index file will be loaded when the project starts, so of course the framework knows all the resource paths and the AB package names it references, so when loading resources, it will naturally find the corresponding AB package, and the resource index file also records the AB package. Interdependencies, when loading the target AB package, just load all dependent packages recursively.
How to use this secondary built resource index file in the project has been explained clearly above. Now let’s start with how to hot-upload all AB packages when the project starts.
The CreatePatchManifestFile method is to create an AB package download list. Please note that the old list will be loaded before creating a new list, and the MD5 generated by the AB package will be compared to see if there is any change. If there is no change, the version number of the old list will continue to be used. For example : Assume that the UI_Login default is generated in version 1, and this time it is packaged in version 2. Since UI_Login is compared and found that the MD5 has not changed in this package, the AB package version where UI_Login is located still writes 1, other changes, and new additions Write 2 for the resource version number.
/// <summary>
/// 1. 创建补丁清单文件到输出目录
/// params: isInit 创建的是否是包内的补丁清单
/// useAAB 创建的是否是aab包使用的补丁清单
/// </summary>
private void CreatePatchManifestFile(string[] allAssetBundles, bool isInit = false, bool useAAB = false)
{
// 加载旧文件
PatchManifest patchManifest = LoadPatchManifestFile(isInit);
// 删除旧文件
string filePath = OutputPath + $"/{PatchDefine.PatchManifestFileName}";
if (isInit)
filePath = OutputPath + $"/{PatchDefine.InitManifestFileName}";
if (File.Exists(filePath))
File.Delete(filePath);
// 创建新文件
Log($"创建补丁清单文件：{filePath}");
var sb = new StringBuilder();
using (FileStream fs = File.OpenWrite(filePath))
{
using (var bw = new BinaryWriter(fs))
{
// 写入强更版本信息
//bw.Write(GameVersion.Version);
//sb.AppendLine(GameVersion.Version.ToString());
int ver = BuildVersion;
// 写入版本信息
// if (isReview)
// {
// ver = ver * 10;
// }
bw.Write(ver);
sb.AppendLine(ver.ToString());
// 写入所有AssetBundle文件的信息
var fileCount = allAssetBundles.Length;
bw.Write(fileCount);
for (var i = 0; i < fileCount; i++)
{
var assetName = allAssetBundles[i];
string path = $"{OutputPath}/{assetName}";
string md5 = HashUtility.FileMD5(path);
long sizeKB = EditorTools.GetFileSize(path) / 1024;
int version = BuildVersion;
EBundlePos tag = EBundlePos.buildin;
string readableLabel = "undefined";
if (_labelToAssets.TryGetValue(assetName, out var list))
{
readableLabel = list[0].ReadableLabel;
if (useAAB)
tag = list[0].bundlePos;
}
// 注意：如果文件没有变化使用旧版本号
PatchElement element;
if (patchManifest.Elements.TryGetValue(assetName, out element))
{
if (element.MD5 == md5)
version = element.Version;
}
var curEle = new PatchElement(assetName, md5, version, sizeKB, tag.ToString(), isInit);
curEle.Serialize(bw);
if (isInit)
sb.AppendLine($"{assetName}={readableLabel}={md5}={sizeKB}={version}={tag.ToString()}");
else
sb.AppendLine($"{assetName}={readableLabel}={md5}={sizeKB}={version}");
}
}
string txtName = "PatchManifest.txt";
if (isInit)
txtName = "InitManifest.txt";
File.WriteAllText(OutputPath + "/" + txtName, sb.ToString());
Debug.Log($"{OutputPath}/{txtName} OK");
}
}The generated AB package list looks like this.
The first line is the SVN version number
The second line is the number of AB packages
Starting from the third line is the resource package information, separated by = sign to separate the valid data, respectively
MD5.unity3d = Resource path = HashId of resource path = Package body KB size = SVN version number = Start hot update mode
Finally, write this InitManifest.txt into bytes and send it to the server to compare the data packets.
In the loading part of this series of articles, I will formally explain the loading of AB packages. This article will only give a brief introduction.
first step:
When the client starts, it enters the download list state machine. Http first downloads the InitManifest.txt or InitManifest.bytes file and parses the AB package list.
Below is the code to parse the AB package list.
public class PatchElement
{
/// <summary>
/// 文件名称
/// </summary>
public string Name { private set; get; }
/// <summary>
/// 文件MD5
/// </summary>
public string MD5 { private set; get; }
/// <summary>
/// 文件版本
/// </summary>
public int Version { private set; get; }
/// <summary>
/// 文件大小
/// </summary>
public long SizeKB { private set; get; }
/// <summary>
/// 构建类型
/// buildin 在安装包中
/// ingame 游戏中下载
/// </summary>
public string Tag { private set; get; }
/// <summary>
/// 是否是安装包内的Patch
/// </summary>
public bool IsInit { private set; get; }
/// <summary>
/// 下载文件的保存路径
/// </summary>
public string SavePath;
/// <summary>
/// 每次更新都会先下载到Sandbox_Temp目录，防止下到一半重启导致逻辑不一致报错
/// temp目录下的文件在重新进入更新流程时先校验md5看是否要跳过下载
/// </summary>
public bool SkipDownload { get; set; }
public PatchElement(string name, string md5, int version, long sizeKB, string tag, bool isInit = false)
{
Name = name;
MD5 = md5;
Version = version;
SizeKB = sizeKB;
Tag = tag;
IsInit = isInit;
SkipDownload = false;
}
public void Serialize(BinaryWriter bw)
{
bw.Write(Name);
bw.Write(MD5);
bw.Write(SizeKB);
bw.Write(Version);
if (IsInit)
bw.Write(Tag);
}
public static PatchElement Deserialize(BinaryReader br, bool isInit = false)
{
var name = br.ReadString();
var md5 = br.ReadString();
var sizeKb = br.ReadInt64();
var version = br.ReadInt32();
var tag = EBundlePos.buildin.ToString();
if (isInit)
tag = br.ReadString();
return new PatchElement(name, md5, version, sizeKb, tag, isInit);
}
}Step two:
Please note that interrupt and resume downloading is also a very important function. The AB package list records the size of each AB package. When the project is started, the AB packages in the Temp folder will be traversed first. If the size is inconsistent with the one in the list, it will be turned on. Http download function, Http supports breakpoint resumption, and the data segment to be downloaded is defined in the Http Header. If you think this is not safe, you can directly delete the AB package and download it again.
After the AB package list is parsed, switch to the download list state machine and start downloading each file in the list. Please note that when downloading files for hot updates, we can first create a Temp folder. All AB packages before they are downloaded successfully are in Here, after all downloads are successful, cut them all into the PersistentData folder. The PersistentData folder is Unity's built-in sandbox directory, and Unity has read and write permissions.
After all downloads are completed, complete the cutting of the PersistentData folder.
third step:
All resources are in place and the formal business framework is launched.
Question: Why start the formal business framework after the hot update is completed?
At present, most commercial projects are based on Tolua and Xlua frameworks, and many framework layer codes are written in Lua. Lua code is part of the AB package, so it can only be started after the hot update is completed.