@
Next to the previous article "Things Behind the 100 Billion Internet Platform-Want to Go to the Blue Sky to Cover the Moon" , today we will talk about the privacy of users.
With the development of the times and the Internet, people pay more and more attention to personal privacy, but the problem of leakage and abuse of private information is still common. Previously, a "China Personal Information Security and Privacy Protection Report" had extracted 1 million survey data, and 80% of users encountered privacy leaks . For example, Marriott suffered a 383 million privacy data leak in 18 years and then exploded again on March 31, 2020. Out of 5.2 million customer information leaked. The reason behind this is that we won't talk much about it. Apart from the malicious behavior of some rogue companies, there must be a lot of commercial interests. Today we come to talk about how developers should deal with user privacy. I remembered that the developer of the Putonghua proficiency test query system in a province that knew about half a year ago wrote the ID card directly in js. Some netizens laughed and said that this is true. The separation of the front end and the back end is not a thing to support billion-level concurrency .
Before the article begins, a small question is thrown : In addition to the name, ID card, bank card, mobile phone number, what do you think is the user's sensitive information that needs to be encrypted and stored?
What is personal information and what is sensitive information? How should personal information be stored and displayed? Is the redemption code in the game sensitive information? Is the accommodation information sensitive? ? As an excellent developer, we ca n’t just focus on the code, we ca n’t always be the product manager or project manager to let me do this, and we should also master the business knowledge of the industry, including laws and policy regulations, to enhance and broaden us. Business knowledge side.
1. User Information Security Specification
Regarding the construction of information systems, there are actually many standards and regulations in the country and industry, such as the national standard full text disclosure system ( http://openstd.samr.gov.cn/)). Regarding personal information, the latest is the "GB / T 35273-2020 Information Security Technology-Personal Information Security Specification" released this year , which will be officially implemented in 2020-10-01 , replacing the old standard GB / T 35273-2017. The entire specification document mainly embodies seven principles: the principle of unanimous power and responsibility, the principle of clear purpose, the principle of choice and consent, the principle of least sufficient, the principle of openness and transparency, the principle of ensuring safety, and the principle of subject participation .
1.1 Definition and judgment basis of user information and sensitive information
1.1.1 Personal information
Personal information. Refers to all kinds of information recorded in electronic or other ways that can identify the identity of a specific natural person or reflect the activities of a specific natural person, alone or in combination with other information.
Judgment method :
- Identification: From information to individuals, specific natural persons are identified by the particularity of the information itself. Personal information should help identify specific individuals.
- Correlation: From personal to information, if a specific natural person is known, the information (such as personal location information, personal call records, personal browsing records, etc.) generated by the specific natural person in its activities is personal information.
Information that meets one of the above two situations should be determined as personal information.
Examples of personal information : 
Note : The information formed by the personal information controller through the processing of personal information or other information, such as user portraits or feature tags, can identify the identity of a specific natural person or reflect the activities of a specific natural person alone or in combination with other information. Also belongs to personal information.
1.1.2 Personal sensitive information
Personal sensitive information. Refers to personal information that once leaked, illegally provided or misused may endanger personal and property safety, easily lead to damage to personal reputation, physical and mental health, or discriminatory treatment. Generally, personal information of children under 14 years old (inclusive) and information related to the privacy of natural persons are personal sensitive information .
Judgment method :
-
Leakage: Once the personal information is leaked, it will cause the main body of personal information and the organizations and institutions that collect and use personal information to lose the ability to control personal information, resulting in the uncontrollable scope and use of personal information. After leakage, some personal information is directly used in a manner contrary to the will of the personal information subject or associated analysis with other information, which may bring significant risks to the personal information subject's rights and interests, and should be determined as personal sensitive information. For example, the copy of the ID card of the main body of personal information is used by others for real-name registration of a mobile phone number card, bank account opening card, etc.
-
Illegal provision: Some personal information can only cause significant risks to the rights and interests of personal information subjects because they are spread outside the authorized consent of the personal information subjects, and should be determined as personal sensitive information. For example, sexual orientation, deposit information, history of infectious diseases, etc.
-
Misuse: When certain personal information is used beyond the reasonable limit of authorization (such as changing the processing purpose, expanding the processing range, etc.), it may bring significant risks to the rights and interests of the personal information subject and should be determined as personal sensitive information. For example, when the authorization of the main body of personal information is not obtained, the health information is used for insurance company marketing and determining the level of individual premiums.
Examples of personal sensitive information
Note : The information formed by the personal information controller through the processing of personal information or other information, such as once leaked, illegally provided or abused GB / T 35273-20206 may endanger the safety of personal and property, easily lead to personal reputation , Physical and mental health is damaged or discriminatory treatment, etc., is personal sensitive information.
1.2 Notes on user information storage
- The storage time of personal information is minimized. After the storage period of personal information is exceeded, personal information should be deleted or anonymized.
- When transmitting and storing personal sensitive information, encryption and other security measures should be adopted; when using cryptographic technology, it is advisable to follow relevant national standards for password management.
- Personal biometric information should be stored separately from personal identity information ;
- In principle, should not be stored original Personal biometric information (e.g., sample, images, etc.) , measures can be taken include, but are not limited to: only the summary information stored personal biometric identification information; implemented using personal biometric information collection terminal identification directly in the , Authentication and other functions; after using facial recognition features, fingerprints, palm prints, irises, etc. to achieve identification, authentication and other functions, delete the original image that can extract personal biometric information.
The entire specification document also mentions the use, display, third-party access, security management, etc. of user information. Interested partners can search for it by themselves.
Second, the implementation of framework technology
2.1 Automatic encryption and decryption of user sensitive information
As mentioned in the first chapter, the user ’s real name, mobile phone number, bank card number, and sensitive information including accommodation need to be encrypted and stored in the database, and then converted into plain text data when the business is in normal use. From the technical implementation point of view, it is nothing more than encrypting when adding, editing, and decrypting when querying. Such operations are still relatively low, and it is likely that one day a new method was added and forgetting to encrypt and decrypt. So most of them will be implemented through the framework, and the principle of implementation is nothing more than reflection machine + interceptor. Next, take Mybatis as an example, the principle is as follows, for specific reference: https://blog.csdn.net/weixin_39494923/article/details/91534658
2.1.1 Automatic encryption and decryption of data through Interceptor
Mybatis provides an interceptor interface Interceptor by default, most of Mybatis' enhanced tools are implemented through this interface. If you want to implement a custom interceptor, you only need to implement the org.apache.ibatis.plugin.Interceptor interface, which has three methods:
Object intercept(Invocation invocation) throws Throwable;
Object plugin(Object target);
void setProperties(Properties properties);
First, a custom annotation @Crypt, which acts on the field, is used to tell the interceptor that the field needs to be encrypted and decrypted.
@Target({ ElementType.FIELD,ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
public @interface Crypt {
}
Next, add a custom interceptor, decrypt when using the selelct method, and encrypt when using the update and add methods.
@Intercepts({ @Signature(type = Executor.class, method = "update", args = { MappedStatement.class, Object.class, }),
@Signature(type = Executor.class, method = "query", args = { MappedStatement.class, Object.class,
RowBounds.class, ResultHandler.class }),
@Signature(type = ResultSetHandler.class, method = "handleResultSets", args = { Statement.class }) })
public class CryptInterceptor implements Interceptor {
@Override
public Object intercept(Invocation invocation) throws Throwable {
Object[] args = invocation.getArgs();
if (args.length <= 0 || invocation.getMethod() == null || args[0] == null) {
return invocation.proceed();
}
String methodName = invocation.getMethod().getName();
if ("update".equals(methodName) && args[1] != null) {
return this.interceptUpdate(invocation);
} else if ("query".equals(methodName) && args[1] != null) {
return this.interceptQuery(invocation);
} else if ("handleResultSets".equals(methodName)) {
return this.interceptHandleResultSets(invocation);
}
return invocation.proceed();
}
private Object interceptHandleResultSets(Invocation invocation) throws Throwable {
Object resultCollection = invocation.proceed();
// 略 将resultCollection的对象中有@Crypt注解的Feild进行解密
return newObject;
}
private Object interceptUpdate(Invocation invocation) throws Throwable {
Object[] args = invocation.getArgs();
Object args1Obj = args[1];
// 略 将args1Obj的对象进行加密
args[1] = newObject;
return invocation.proceed();
}
private Object interceptQuery(Invocation invocation) throws Throwable {
Object[] args = invocation.getArgs();
Object condition = args[1];
// 略 将condition对象进行解密
args[1] = newObject;
return invocation.proceed();
}
}
2.1.2 Automatic encryption and decryption of data through BaseTypeHandler
Under normal circumstances, the Interceptor interface will not intercept Mybatis requests, unless some complex requirements such as "read and write separation". See the execution process of mybatis above, we found that the last step called TypeHander, the role of this class is to perform type conversion between the database and the entity, such as the conversion of MySql varchar to Java Long, and Java Integer to Mysql int, so we can use the BaseTypeHandler class.
@Component
@Alias("CryptHandler")
@MappedTypes(value = {Crypt.class})
public class EncryptHandler extends BaseTypeHandler {
@Override
public void setNonNullParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType)
throws SQLException {
ps.setString(i, encrypt(parameter.toString()));
}
@Override
public String getNullableResult(ResultSet rs, String columnName) throws SQLException {
String columnValue = rs.getString(columnName);
return decrypt(columnValue);
}
@Override
public String getNullableResult(ResultSet rs, int columnIndex) throws SQLException {
String columnValue = rs.getString(columnIndex);
return decrypt(columnValue);
}
@Override
public String getNullableResult(CallableStatement cs, int columnIndex) throws SQLException {
String columnValue = cs.getString(columnIndex);
return decrypt(columnValue);
}
private String encrypt(String parameter) {
// 加密
return parameter;
}
private String decrypt(String columnValue) {
// 解密
return columnValue;
}
}
See above for the complete code, so I won't go into more details. Next, I need to tell Mybatis which fields need to be encrypted and decrypted. In order to simplify writing, define a class Crypt renamed to crypt, the above class EncryptHandler also renamed to EncryptHandler
@Alias("crypt")
public final class Crypt {
}
The above two classes are placed in the cn.itmds.plugin directory, the configuration yml file tells Mybatis to read the renamed configuration
mybatis:
type-aliases-Package: cn.itmds.plugin.dbcrypt
Next, suppose there is a member table whose realname (real name) field needs to be encrypted and decrypted, it is very simple to write:
<sql id="memberConditionSql">
<where>
<if test="id != null">and id = #id}</if>
<!--这个地方只需要指定javaType=crypt，如果上面没有重命名，这个地方需要写成javaType= cn.itmds.plugin.dbcrypt.Crypt，写起来比较麻烦 -->
<if test=realName != null">and real_name = #{realName,javaType=crypt}</if>
</where>
</sql>
<resultMap id="memberDOResultMap" type="MemberDO">
<!--这个地方只需要指定typeHandler=CryptHandler，如果上面没有重命名，这个地方需要写成javaType= cn.itmds.plugin.dbcrypt.CryptHandler，写起来比较麻烦 -->
<!--另外，只需要将需要解密的字段写到这个resultMap里即可，不需要写全部的字段，其他字段系统会自动映射为MemberDO -->
<result column="phone" property="phone" typeHandler="CryptHandler"/>
</resultMap>
2.1.3 MybatisPlus realizes automatic encryption and decryption of data
MyBatis-Plus (abbreviated as MP) is a MyBatis enhancement tool. On the basis of MyBatis, only enhancements and no changes are made. It is born to simplify development and improve efficiency.
With simple configuration, MyBatis-Plus can quickly perform CRUD operations, saving a lot of time. And it also supports Lambda expressions, manipulation of SQL, etc. through objects, so more and more people are using it now. So how can it achieve automatic encryption and decryption of data, super simple. The realization principle is the same as that in 2.1.2, but also through BaseTypeHandler.
1. Added @TableField(typeHandler = EncryptHandler.class), where EncryptHandler is defined in 2.1.2, EncryptHandler.javaand automatic encryption is implemented when adding or modifying.
2. Set on @TableName autoResultMap = true, at this time the automatic decryption of the return value is realized.
Done! Examples:
@Data
@TableName(value = "user_info",autoResultMap = true)
public class UserPO {
/** */
@TableId(type = IdType.AUTO)
private Long id;
/** 真实姓名 */
@TableField(typeHandler = EncryptHandler.class)
private String realName;
}
2.2 Log files automatically filter user sensitive information
In order to facilitate development and debugging and to locate production line problems, the development framework will basically define a log interceptor, intercept all methods of the controller layer and service layer, and print detailed input and output parameters. In 2.1, we mentioned that the encryption and decryption of the user ’s sensitive information is done automatically at the bottom of the dao, so it also causes the user ’s sensitive information to be printed in the log. Next, provide a complete case.
- Define an annotation @ServiceLog, which can be applied to the class or method. Provide a parameter: ignore, the default is false. If true, it means that the method does not need to print logs. For example, there are many methods in a certain class that require logs, but one of the methods is used for file uploads or scheduled tasks are executed once per second. In these scenarios, you do not need to print logs, you can set ignore = true.
@Target({ElementType.TYPE, ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface ServiceLog {
boolean ignore() default false;
}
- Define a global interceptor, print in and out log, use FastJson to convert the object into a string.
@Aspect
@Component
public class ServiceLogAspect {
@Around("@within(cn.itmds.log.ServiceLog)")
protected Object aroundJoinPoint(ProceedingJoinPoint joinPoint) throws Throwable {
MethodSignature signature = (MethodSignature)joinPoint.getSignature();
Method method = signature.getMethod();
ServiceLog serviceLog = method.getAnnotation(ServiceLog.class);
if (null != serviceLog && serviceLog.ignore()) {
return joinPoint.proceed();
}
long beginTime = System.currentTimeMillis();
Class clazz = joinPoint.getTarget().getClass();
String methodName = clazz.getSimpleName() + "." + method.getName();
// 打印请求所有的入参
log.info("Begin|{}|{}", methodName, jsonString(joinPoint.getArgs()));
Object result = null;
try {
result = joinPoint.proceed();
} finally {
// 打印所有的出参
log.info("End|{}|{}ms|{}", methodName, System.currentTimeMillis(),
- beginTime, jsonString(result));
}
return result;
}
}
- Add a configuration item to define the sensitive information to be filtered, such as real name, mobile phone number, ID card, password, etc.
logging:
sensitiveChars: realName,phoneNumber,idCard,mail,password
- Next, we can use the filter feature of FastJSON to filter the logs.
private ValueFilter valueFilter = (object, name, value) -> {
if (null == value || "".equals(value)) {
return value;
}
if (value instanceof byte[]) {
// 如果是byte字节，直接打印长度
return "byte length:" + ((byte[])value).length;
} else if (value instanceof String) {
// 在该方法里检查name，如果name包含我们配置的敏感信息，则将value设置为加*隐藏。
return stringValueProcess(name, (String)value);
} else {
return value;
}
};
In the second step interceptor method aroundJoinPoint, when the object is converted to String, the FastJSON filter is used.
protected String jsonString(Object object) {
return JSON.toJSONString(object, valueFilter);
}
- Controller layer is similar, intercept all files in the controller directory.
@Around("execution(public * cn.itmds.controller..*(..) )")
Controller should pay attention to this method, because some fields of http request and response request cannot be serialized, so it must be filtered.
public static <T> Stream<T> streamOf(T[] array) {
return ArrayUtils.isEmpty(array) ? Stream.empty() : Arrays.asList(array).stream();
}
//... 拦截器的方法中增加过滤
List<Object> logArgs = (List)streamOf(args).filter((arg) -> {
return !(arg instanceof HttpServletRequest) && !(arg instanceof HttpServletResponse);
}).collect(Collectors.toList());
// 打印请求所有的入参
log.info("Begin|{}|{}", methodName, jsonString(logArgs));
2.3 Password encryption and "Password Law"
Regarding passwords, the country also has a "Cryptography Law", which seems to be being promoted recently. Of course, the username "password" we usually say is just a "password", not a "password" in cryptography. The scope of use of passwords in the "Password Law" includes second-generation ID cards, electronic signatures, VAT invoice password areas and the like. For details, you can go to the full text and do not talk about it.
2.3.1 Notes on password encryption
Today's developers basically have certain security knowledge. Few passwords are stored in plaintext, and even direct md5 are few. Most of them have started to use sha1 and sha256, and some companies have started to use Argon2.
Argon2 is a slow hash function. It won the Password Hashing Competition championship in 2015. It uses a lot of memory calculations to resist GPU and other custom hardware cracks, improving the security of hash results.
Here are some points:
- Each password must be added with a different salt to ensure that the same password also generates different hashes. For example, the passwords of both people are abcd1234, and the generated hash must be different.
- Don't use ordinary random algorithm to generate salt, you must use CSPRNG (Cryptographically Secure Pseudo-Random Number Generator); the corresponding java is Java.security.SecureRandom, corresponding to C / C ++ CryptGenRandom.
- Some systems use the user's id, mobile phone number, etc. as the salt encryption password, which does not meet the salt generation rules. But for the general security requirements of the site is not so high, it can basically be used.
2.3.2 Use BCrypt to implement password encryption
Bcrypt is a cross-platform file encryption tool, and SpringSecurity uses this algorithm by default. If SpringSecurity is not in the project, you can also import the jar package separately. There is a big difference between the bcrypt algorithm and the md5 / sha algorithm, that is, the hash value generated each time is different, and we do not need to specify the salt by ourselves. The character length after encryption is relatively long, there are 60 bits, we must pay attention to when designing the database field. Examples are as follows:
public static void main(String[] args) {
BCryptPasswordEncoder bcrypt = new BCryptPasswordEncoder();
String pwd = "abcd1234";
for (int i = 0; i < 5; i++) {
String encodePwd = bcrypt.encode(pwd);
boolean result = bcrypt.matches(pwd, encodePwd);
System.out.println(encodePwd + "|" + result);
}
}
The encrypted string value consists of :
- $ Is a separator, meaningless;
- 2a is the bcrypt encrypted version number;
- 10 is the value of cost;
- In the following string, the first 22 digits are the salt value; the next string is the ciphertext of the password.
Those who are interested can look at the source code
public static String gensalt(int log_rounds, SecureRandom random) {
if (log_rounds < MIN_LOG_ROUNDS || log_rounds > MAX_LOG_ROUNDS) {
throw new IllegalArgumentException("Bad number of rounds");
}
StringBuilder rs = new StringBuilder();
byte rnd[] = new byte[BCRYPT_SALT_LEN];
random.nextBytes(rnd);
rs.append("$2a$");
if (log_rounds < 10) {
rs.append("0");
}
rs.append(log_rounds);
rs.append("$");
encode_base64(rnd, rnd.length, rs);
return rs.toString();
}
2.3.3 Dropbox password encryption storage prevention
Dropbox is a famous manufacturer that provides online storage of files. It has published an article called "How Dropbox securely stores your passwords" on its official technical blog, which describes their user password encryption storage solution.
- First use sha512 to normalize the user password to a 64-byte hash value. For two reasons: one is that Bcrypt is sensitive to input. If the password entered by the user is long, it may cause Bcrypt to be too slow and affect the response time; another is that some implementations of the Bcrypt algorithm will directly truncate the long input to 72 bytes. From the perspective of information theory, this causes the entropy of user information to become smaller;
- Then use the Bcrypt algorithm. The reason for choosing Bcrypt is that Dropbox engineers are more familiar with this algorithm and have more experience in tuning. The standard for parameter selection is that Dropbox's online API server can calculate the result in about 100ms. In addition, as to which algorithm is better, Bcrypt or Scrypt, cryptographers are also inconclusive. At the same time, Dropbox is also paying attention to the password hash algorithm rookie Argon2, and said that it will be introduced at the right time;
- Finally use AES encryption. Because Bcrypt is not a perfect algorithm, Dropbox uses AES and global keys to further reduce the risk of password cracking. In order to prevent key leakage, Dropbox uses dedicated key storage hardware. Dropbox also mentioned another advantage of using AES encryption at the end, that is, the key can be changed regularly to reduce the risk of user information / key leakage.
User privacy protection is far from being as simple as encryption and decryption by developers, but also requires cooperation from all aspects of operations and operation and maintenance teams.
[People always want to leave some privacy space for themselves
, just like you will always stand in front of your shadow to block the sight of the light] People always want to give yourself some privacy space, just like you will always be standing in front of the shadow of you blocking the line of sight of the light.
Reference:
https://www.cnblogs.com/xinzhao/p/6035847.html
https://blog.csdn.net/weixin_39494923/article/details/91534658
Architect, ten years Rong [code], old [uncle] flowering. Personal WeChat: qiaojs, focusing on architecture design, big data, microservices, and technology management.
