In our App development process, the network is indispensable. It is almost difficult to think of apps that do not require network transmission. Therefore, network problems are generally difficult to reproduce offline. Once they are in the hands of users, they will encounter many There are difficult and complicated diseases, so network monitoring is essential. For common problems of users, we also need to add optimization strategies in actual projects.
1 Basic optimization of the network
For some mainstream network request frameworks, such as OkHttp, Retrofit, etc., they actually encapsulate the Http protocol. The most common ones we use are POST or GET requests. If we are doing client development, knowing these basic contents is like You can also write code, but when you really encounter online network problems, you can't figure it out. In fact, the biggest problem is the lack of knowledge about the network, so at the beginning of the article, let's start with some basic network knowledge.
1.1 Types of network connections
In fact, for network connections, we usually initiate a request to the server, and the server returns a corresponding response. However, at the same time, there can only be data transmission in one direction. This connection method is called half-duplex communication.
| type | describe | Example |
|---|---|---|
| simplex | During communication, data can only be sent from one party to another | Common ones include UDP protocol; Android broadcast |
| half duplex | During the communication process, data can be sent from one party A to the other party B, or from one party B to the other party A, but there can only be data transmission from one party at the same time. | Common protocols such as Http |
| full duplex | At any time, there will be two-way data transmission from A to B and B to A. | Common ones include Socket protocol and long connection channel |
Therefore, the Http1.0 protocol is still a half-duplex protocol, because long connections are turned off by default. If you need to support long connections, you need to add a field to the http header: "Connection:Keep-Alive"; in Http 1.1 protocol, long connections are enabled by default. If you need to close long connections, you need to add the http request header field: "Connection:close".
So when or in what scenario do we need to use long connections? In fact, it is very simple. Just remember one thing. If there are requirements for the immediacy of messages in business scenarios, you need to establish a long connection with the server, such as IM chat, video calls and other scenarios.
1.2 DNS resolution
If partners have added trace logs to the network in the project, in addition to timeout errors such as net timeout, they should have also seen exceptions such as UnknowHostException. This is because the DNS resolution failed and the IP address of the server was not obtained through resolution.
When we are at home, our mobile phones or computers will connect to the router's WiFi, and the router can set the DNS server address.
However, if the settings are incorrect or tampered by attacks, DNS resolution will fail, and our app's network requests will be abnormal. Therefore, in this case, we need to add our own DNS resolution strategy.
First, let's look at an example. Suppose we want to request a Baidu domain name to obtain a piece of data, for example:
object HttpUtil {
private const val BASE_URL = "https://www.baidu.comxx"
fun initHttp() {
val client = OkHttpClient.Builder()
.build()
Request.Builder()
.url(BASE_URL)
.build().also {
kotlin.runCatching {
client.newCall(it).execute()
}.onFailure {
Log.e("OkHttp", "initHttp: error $it ")
}
}
}
}
Obviously, Baidu’s domain name is wrong, so an error will be reported when executing a network request:
java.net.UnknownHostException: Unable to resolve host "www.baidu.comxx": No address associated with hostname
So once our domain name is hijacked and modified, the entire service will be down, and the user experience will be poor. Therefore, we can make a small optimization through the custom DNS resolver provided by OkHttp.
public interface Dns {
/**
* A DNS that uses {@link InetAddress#getAllByName} to ask the underlying operating system to
* lookup IP addresses. Most custom {@link Dns} implementations should delegate to this instance.
*/
Dns SYSTEM = hostname -> {
if (hostname == null) throw new UnknownHostException("hostname == null");
try {
return Arrays.asList(InetAddress.getAllByName(hostname));
} catch (NullPointerException e) {
UnknownHostException unknownHostException =
new UnknownHostException("Broken system behaviour for dns lookup of " + hostname);
unknownHostException.initCause(e);
throw unknownHostException;
}
};
/**
* Returns the IP addresses of {@code hostname}, in the order they will be attempted by OkHttp. If
* a connection to an address fails, OkHttp will retry the connection with the next address until
* either a connection is made, the set of IP addresses is exhausted, or a limit is exceeded.
*/
List<InetAddress> lookup(String hostname) throws UnknownHostException;
}
Let's take a look at the source code. The lookup method is equivalent to doing DNS addressing. Once an exception occurs, an UnknownHostException exception will be thrown. At the same time, a SYSTEM method is also defined internally, in which routing addressing is performed through the InetAddress class provided by the system. , also if DNS resolution fails, UnknownHostException will also be thrown.
So we take two steps. First, we use the system capabilities for routing. If it fails, then we use a custom strategy.
class MyDNS : Dns {
override fun lookup(hostname: String): MutableList<InetAddress> {
val result = mutableListOf<InetAddress>()
var systemAddressList: MutableList<InetAddress>? = null
//通过系统DNS解析
kotlin.runCatching {
systemAddressList = Dns.SYSTEM.lookup(hostname)
}.onFailure {
Log.e("MyDNS", "lookup: $it")
}
if (systemAddressList != null && systemAddressList!!.isNotEmpty()) {
result.addAll(systemAddressList!!)
} else {
//系统DNS解析失败，走自定义路由
result.add(InetAddress.getByName("www.baidu.com"))
}
return result
}
}
In this way, after the resolution of www.baidu.comxx fails, the domain name www.baidu.com will be used to replace it, thereby avoiding the problem of network request failure.
1.3 Interface data adaptation strategy
I believe many partners often encounter this situation when debugging interfaces with the server: the interface document indicates that this field is of type int, and the result is the string ""; or in some cases, I need the server to return An empty array, but null is returned. In this case, when we parse the data, whether we use Gson or Moshi, the parsing will fail. If not handled properly, it will seriously cause a crash.
So to deal with this data format mismatch problem, we can simply make some adaptations to Gson, such as the List type:
class ListTypeAdapter : JsonDeserializer<List<*>> {
override fun deserialize(
json: JsonElement?,
typeOfT: Type?,
context: JsonDeserializationContext?
): List<*> {
return try {
if (json?.isJsonArray == true) {
Gson().fromJson(json, typeOfT)
} else {
Collections.EMPTY_LIST
}
} catch (e: Exception) {
//
Collections.EMPTY_LIST
}
}
}
If json is a List array type data, then it will be converted into a List array normally; if not, it will be parsed into an empty array.
class StringTypeAdapter : JsonDeserializer<String> {
override fun deserialize(
json: JsonElement?,
typeOfT: Type?,
context: JsonDeserializationContext?
): String {
return try {
if (json?.isJsonPrimitive == true) {
Gson().fromJson(json, typeOfT)
} else {
""
}
} catch (e: Exception) {
""
}
}
}
For String type fields, it will first be judged whether it is a basic type (String, Number, Boolean). If it is a basic type, then it can be converted normally.
GsonBuilder()
.registerTypeAdapter(Int::class.java, IntTypeAdapter())
.registerTypeAdapter(String::class.java, StringTypeAdapter())
.registerTypeAdapter(List::class.java, ListTypeAdapter())
.create().also {
GsonConverterFactory.create(it)
}
In this way, when creating GsonConverterFactory, you can use our strategy for data adaptation. However, in the test environment, we do not recommend using this because problems on the server cannot be found. You can use this strategy to avoid online problems after going online.
2 HTTPS protocol
The difference between http protocol and https protocol is that there is an extra "s". Don't underestimate this "s". It can ensure the reliability of http data transmission. So what is this "s"? It is the SSL/TLS protocol. .
From the picture above, the SSL/TLS protocol will be used before entering the TCP protocol.
2.1 Symmetric encryption and asymmetric encryption
Since HTTPS can ensure the reliability of transmission, it means that it encrypts the data. In the past, HTTP protocol data was transmitted in plain text, and the data was easily stolen and impersonated. Therefore, in the subsequent optimization, the data was encrypted and transmitted. The HTTPS protocol was born.
There are two common encryption methods: symmetric encryption and asymmetric encryption.
2.1.1 Symmetric encryption
First of all, symmetric encryption, you can know the specific principle from the name, look at the picture below:
The key for symmetric encryption and decryption is a key, which needs to be agreed upon by both parties. The sender encrypts the data using the secret key, and the receiver uses the same secret key to decrypt and obtain the transmitted data.
Therefore, using symmetric encryption is very simple and parses data quickly, but the security is relatively poor because both parties need to agree on the same key. The transmission of the key is at risk of being hijacked, and unified storage is also at risk of being attacked.
Therefore, in response to this situation, asymmetric encryption emerged.
2.1.2 Asymmetric encryption
Asymmetric encryption will have two keys: private key + public key. Anyone can know the public key. The sender can use the public key to encrypt data, and the receiver can use the private key that only he knows to decrypt and obtain the data.
So since everyone knows the public key, can the private key be directly derived from the public key? The answer is that it is currently impossible, but it may be possible in the future. It depends on whether cryptography masters or hackers around the world can solve this problem.
Summarize the advantages and disadvantages of the two encryption methods:
| Encryption type | advantage | shortcoming |
|---|---|---|
| Symmetric encryption | The process is simple and the decryption speed is fast | Not safe, key management is risky |
| asymmetric encryption | Only you know the private key | The process is cumbersome and the decryption speed is slow |
2.2 Security Guarantee of Public Keys
Through the introduction of asymmetric encryption in Section 2.1, although it seems to be more secure, the transfer of public keys is a bit too ideal. Let's look at the following scenario.
If the public key is hijacked during transmission, then the sender will get the hacker's public key, and all subsequent data transmissions will be hijacked. So the question becomes, how to ensure that the public key obtained by the sender must be the recipient's public key ? square?
Let’s take a simple example: We picked up a bank card on the road and want to withdraw the money inside. Then the bank counter is actually the recipient, and the bank card is the public key. Then the bank will give us the money directly? ? Definitely not, we either need an ID card or a password to prove that the bank card is ours, so the security guarantee of the public key is the CA certificate (which can be understood as our ID card).
Then first the recipient needs to apply for an ID card and generate a digital signature through the CA agency. The specific generation rules are as follows:
Then what is finally sent to the recipient is the following digital certificate, which contains: digital signature + public key + personal information of the recipient, etc.
Then after the sender receives the digital certificate, it will check whether the digital certificate is legal. The detection method is as follows:
If it is not a fake certificate, the possibility is almost 0, because it is simply impossible to forge a digital signature of a domain name, and the CA organization is not just for nothing , so as long as the certificate is authenticated, the public key can be guaranteed security.
2.3 HTTPS transmission process
In fact, an Https request contains two Http transmissions. If we request, www.baidu.comthe specific process is as follows:
(1) The client initiates a request to the server to access Baidu, and then establishes a connection with Baidu's server at this time;
(2) At this time, the server has a public key and a private key. The public key can be sent to the client, and then an SSL certificate is sent to the client, which includes: CA signature, public key, and some information from Baidu. For details, see Section 2.2 final picture;
(3) After receiving the SSL certificate, the client decrypts the CA signature to determine whether the certificate is legal. If it is not legal, then disconnect the connection; if it is legal, then generate a random number as the key for symmetric encryption of data . key, which is sent to the server through public key encryption.
(4) After receiving the client's encrypted data, the server decrypts it through the private key and obtains the symmetric encryption key. Then it encrypts Baidu-related data through the symmetric encryption key and sends it to the client.
(5) The client obtains the data from the server through decryption, and the request ends.
In fact, HTTPS requests are not completely asymmetric encryption, but combine the strengths of each company. Because the transmission of symmetric encryption keys is risky, the symmetric encryption keys are transmitted through asymmetric encryption in the early stage, and subsequent data transmission is through symmetric encryption, improving improve the efficiency of data analysis.
But what we need to understand is that HTTPS only protects the security of both parties in the communication. Man-in-the-middle attacks such as packet capture by test partners through Charles will still lead to the risk of data leakage, because forged certificates or untrusted CAs can accomplish.
Android study notes
OkHttp source code analysis notes: https://qr18.cn/Cw0pBD
Android performance optimization: https://qr18.cn/FVlo89
Android vehicle version: https://qr18.cn/F05ZCM
Android reverse security study notes: https://qr18.cn/CQ5TcL
Android Framework underlying principles: https://qr18.cn/AQpN4J
Android audio and video: https://qr18.cn/Ei3VPD
Jetpack family bucket (including Compose): https://qr18.cn/A0gajp
Kotlin: https://qr18.cn/CdjtAF
Gradle: https://qr18.cn/DzrmMB
Flutter: https://qr18.cn/DIvKma
Eight knowledge bodies of Android: https://qr18.cn/CyxarU
Android core notes: https://qr21.cn/CaZQLo
Android interview questions from previous years: https://qr18.cn/CKV8OZ
The latest Android interview questions in 2023: https://qr18.cn/CgxrRy
Android vehicle development position interview exercises: https://qr18.cn/FTlyCJ
Audio and video interview questions:https://qr18.cn/AcV6Ap