版权声明：Collected by Bro_Rabbit only for study https://blog.csdn.net/weixin_38240095/article/details/83513644
内容相关：本篇基于2018/10开始的项目teachingsystem，对之前没有使用不多的技术或没有接触的问题进行整理
[通用总结在前]
| 要点 | 总结 |
|---|---|
| 框架功能扩展 | 通常extends相应的预定义类，其中的 protected 方法往往是重写自定义实现的目标 大部分扩展需要在自定义逻辑前/后调用super实现 |
一、Redis
-
服务器的配置与启动
基本的配置与启动参见[笔记迁移][Redis][7]Redis复制Master/Slave Replication，需要强调一点：
在各节点.conf配置文件中，所有需要IP的位置都必须指定 “外部可访问IP” ( ifconfig )，而不是“127.0.0.1”(不论是否位于同一物理节点)，否则将会抛出Connection Refused异常。 -
相关依赖及配置
(1) pom.xml<properties> <!--...--> <!-- Jedis --> <jedis.version>2.9.0</jedis.version> <!-- Spring-data-redis --> <spring.data.redis.version>1.8.6.RELEASE</spring.data.redis.version> <!--...--> </properties> <dependencies> <!--...--> <!-- Jedis --> <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> <version>${jedis.version}</version> </dependency> <!-- Spring-Data-Redis --> <dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-redis</artifactId> <version>${spring.data.redis.version}</version> </dependency> <!--...--> </dependencies>说明：注意spring-data-redis 2.x与Spring版本配置兼容问题，如spring-data-redis 2.0.3 REALEASE 与 Spring 4.3.10 RELEASE 在服务器启动时就会抛出如下异常：
org.springframework.beans.BeanInstantiationException: Failed to instantiate[org.springframework.data.redis.connection.jedis.JedisConnectionFactory]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: org.springframework.util.Assert.isTrue(ZLjava/util/function/Supplier;)V原因：从spring-data-redis2.0 开始，JedisConnectionFactory已经过时，使用 RedisStandaloneConfiguration 来配置链接。
(2) redis.properties
redis.masterName=master6379 redis.sentinel.host=47.100.115.59 redis.sentinel.port=26379 redis.maxTotal=40 redis.maxIdle=20 redis.numTestsPerEvictionRun=1024 redis.timeBetweenEvictionRunsMillis=30000 redis.minEvictableIdleTimeMillis=1800000 redis.softMinEvictableIdleTimeMillis=10000 redis.maxWaitMillis=1500 redis.testOnBorrow=true redis.testWhileIdle=true redis.blockWhenExhausted=false(3) applicationContext-redis.xml( redis.properties已在主配置文件aplicationContext.xml中加载 )
<!-- Jedis连接池配置 --> <bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig"> <property name="maxTotal" value="${redis.maxTotal}"/> <property name="maxIdle" value="${redis.maxIdle}"/> <property name="numTestsPerEvictionRun" value="${redis.numTestsPerEvictionRun}"/> <property name="timeBetweenEvictionRunsMillis" value="${redis.timeBetweenEvictionRunsMillis}"/> <property name="minEvictableIdleTimeMillis" value="${redis.minEvictableIdleTimeMillis}"/> <property name="softMinEvictableIdleTimeMillis" value="${redis.softMinEvictableIdleTimeMillis}"/> <property name="maxWaitMillis" value="${redis.maxWaitMillis}"/> <property name="testOnBorrow" value="${redis.testOnBorrow}"/> <property name="testWhileIdle" value="${redis.testWhileIdle}"/> <property name="blockWhenExhausted" value="${redis.blockWhenExhausted}"/> </bean> <!-- HA Redis哨兵配置 --> <bean id="redisSentinelConfiguration" class="org.springframework.data.redis.connection.RedisSentinelConfiguration"> <!-- 作为master的内置Bean，RedisNode，其name属性值需要与sentinel.conf中设置的一致 --> <property name="master"> <bean class="org.springframework.data.redis.connection.RedisNode"> <property name="name" value="${redis.masterName}"/> </bean> </property> <!-- 多个哨兵，RedisNode集合 --> <property name="sentinels"> <set> <bean class="org.springframework.data.redis.connection.RedisNode"> <constructor-arg name="host" value="${redis.sentinel.host}"/> <constructor-arg name="port" value="${redis.sentinel.port}"/> </bean> </set> </property> </bean> <!-- 通过上述配置创建相应Jedis连接的工厂 --> <bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"> <constructor-arg name="sentinelConfig" ref="redisSentinelConfiguration"/> <constructor-arg name="poolConfig" ref="jedisPoolConfig"/> </bean> <!-- 与JdbcTemplate作用类似，Redis的操作封装（Redis data access） This is the central class in Redis support.Once configured, this class is thread-safe. Performs automatic serialization/deserialization between the given objects and the underlying binary data in the Redis store. --> <bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate"> <property name="connectionFactory" ref="jedisConnectionFactory"/> <property name="defaultSerializer" ref="genericJackson2JsonRedisSerializer"/> <property name="keySerializer" ref="stringRedisSerializer"/> <property name="hashKeySerializer" ref="stringRedisSerializer"/> <property name="enableTransactionSupport" value="false"></property> </bean> <!-- Customized redis template in IoC--> <!-- 自定义"魔改"的redisTemplate --> <bean id="myRedisTemplate" class="com.hpe.ts.utils.redis.MyRedisTemplate"> <property name="connectionFactory" ref="jedisConnectionFactory"/> <property name="defaultSerializer" ref="genericJackson2JsonRedisSerializer"/> <property name="keySerializer" ref="stringRedisSerializer"/> <property name="hashKeySerializer" ref="stringRedisSerializer"/> </bean> <bean id="stringRedisTemplate" class="org.springframework.data.redis.core.StringRedisTemplate"> <property name="connectionFactory" ref="jedisConnectionFactory"/> </bean> <bean id="genericJackson2JsonRedisSerializer" class="org.springframework.data.redis.serializer.GenericJackson2JsonRedisSerializer"/> <bean id="stringRedisSerializer" class="org.springframework.data.redis.serializer.StringRedisSerializer"/> -
关于Serializer
(1) 支持的几种Serializer：全部位于org.springframework.data.redis.serializer下，其接口为org.springframework.data.redis.serializer.RedisSerializer<T>
(2) 说明：要点 说明 作用 除了Serialize/Deserialize的基本作用，从官网摘录描述如下：
From the framework perspective, the data stored in Redis is just bytes. While Redis itself supports various types, for the most part these refer to the way the data is stored rather than what it represents. It is up to the user to decide whether the information gets translated into Strings or any other objects.存取的实体类必须implements Serializable 否则，将抛出异常 作用对象 KEY/ VALUE, HASHKEY/HASHVALUE，从官网摘录描述如下：
Do note that the storage format is not limited only to values - it can be used for keys, values or hashes without any restrictions.
两种配置方式：
1. 逐个配置，keySerializer / valueSerializer , hashKeySerializer, hashValueSerializer
2. 统一配置DefaultSerializer，再对特殊需求进行单独指定覆盖，如本次使用时的配置常用的几种实现的比较 1.JdkSerializationRedisSerializer：RedisTemplate使用的默认实现，将数据序列化为字节数组，即\xab\x1c…这种形式。速度快但占用空间大且不易查看；
2. StringRedisSerializer：StringRedisTemplate使用的默认实现， 将数据序列化为简单的字符串；
3. Jackson2JsonRedisSerializer： 使用Jackson将自定义实体(typed beans)或原生HashMap(untyped HashMap)序列化/反序列化为Json串，JasksonJsonRedisSerializer已经@Deprecated；
4. GenericJackson2JsonSerializer：同样是序列化为JSON串，但相较于JackSon2JsonRedisSerializer，会(为每一层)加入@class属性以指定类的全限名，当将已存入泛型容器反序列化取出时，Jackson2JsonRedisSerializer将抛出cast异常（因为该Serializer将元素对象转为LinkedHashMapHash后存入，取出后类型转换为原类型当然会报错）；然而，GenericJackson2JsonSerializer将可以很好的反序列化（因为每一层元素都记录了其类的全限名@class）
推荐：key与hashKey使用StringRedisSerializer，序列化为可读性好的简单字符串；hashValue使用GenericJackson2JsonRedisSerializer，序列化为JSON -
关于RedisTemplate与StringRedisTemplate
(1) StringRedisTemplate继承自RedisTemplate，为了更方便地满足将键值对直接序列化为可读的字符串地预定义扩展，源码摘要：/** * String-focused extension of RedisTemplate. Since most operations against Redis are String based, this class provides * a dedicated class that minimizes configuration of its more generic {@link RedisTemplate template} especially in terms * of serializers. * <p/> * Note that this template exposes the {@link RedisConnection} used by the {@link RedisCallback} as a * {@link StringRedisConnection}. * * @author Costin Leau */ public class StringRedisTemplate extends RedisTemplate<String, String> {...}(2) 注意：默认Serializer配置下，RedisTemplate和StringRedisTemplate操作的数据并不互通，因为key的序列化规则不一致，说到底即为字符串与字节数组的差异
-
“魔改”——自定义扩展RedisTemplate【最后没有使用，仅记录学习】
(1) 需求：通过spring-data-redis，根据key“动态地”切换至同一Redis实例下的不同库，不再使用默认的idx=0库以达到区分的目的
(2) 尝试结论：spring-data-redis在初始化选择redis实例中地某个库后不能进行修改
(3) 解决方案：/** * @Description TODO Customized redis template to change database dynamically * @author Z-Jay * @time 2018年11月7日下午6:54:56 */ public class MyRedisTemplate extends RedisTemplate<String, Object>{ private static ThreadLocal<Integer> REDIS_DB_INDEX = new ThreadLocal<Integer>(){ @Override protected Integer initialValue() { return 0; } }; public static void select(Integer dbIndex){ REDIS_DB_INDEX.set(dbIndex); } /** * This method is like "AoP", do some setting before any other settingss are executed on it (MARK). * The default implementation returns the connection directly. */ @Override protected RedisConnection preProcessConnection(RedisConnection connection, boolean existingConnection) { try{ Integer dbIndex = REDIS_DB_INDEX.get(); if(dbIndex!=null){ if(connection instanceof JedisConnection){ Jedis nativeConnection = ((JedisConnection)connection).getNativeConnection(); if(nativeConnection.getDB().intValue()!= dbIndex){ connection.select(dbIndex); } }else{ connection.select(dbIndex); } }else{ connection.select(0); } }finally{ REDIS_DB_INDEX.remove(); } return super.preProcessConnection(connection, existingConnection); } } -
Spring Data Redis的事务操作
(1) SessionCallback<T>，推荐的标准方法
Spring Data Redis provides the SessionCallback interface for use when multiple operations need to be performed with the same connection, as when using Redis transactions. For example://execute a transaction List<Object> txResults = redisTemplate.execute(new SessionCallback<List<Object>>() { public List<Object> execute(RedisOperations operations) throws DataAccessException { operations.multi(); operations.opsForSet().add("key", "value1"); // This will contain the results of all ops in the transaction return operations.exec(); } }); System.out.println("Number of items added to set: " + txResults.get(0));如何判断事务是否成功：只要成功，exec()便会返回List，即使事务中全是返回void的操作，其也会返回一个空List结构（非null但isEmpty）；只有事务失败时，exec()才会返回null
(2) enableTransactionSupport与@Transactional：待整理，需要掰扯掰扯
二、 Shiro
-
Shiro的笔记将在后续博客中迁移
-
“魔改”——自定义扩展认证authentication
(1) 自定义roles filter：修改默认AND逻辑为OR逻辑并注册至配置文件application-shiro.xml/** * @Description TODO Customized roles filter, using OR instead of AND * @author Z-Jay * @time 2018年11月6日下午6:40:37 */ //注册方式二：默认以类名首字母小写注入IoC //@Component public class MyAuthorizationFilter extends AuthorizationFilter { @Override protected boolean isAccessAllowed(ServletRequest request, ServletResponse response, Object mappedValue) throws Exception { Subject subject = getSubject(request,response); String[] rolesArray = (String[])mappedValue; //If there is no limit, access is allowed if(rolesArray == null || rolesArray.length == 0){ return true; } //if current subject is with a target role, access is allowed for(int i=0;i<rolesArray.length;i++){ if(subject.hasRole(rolesArray[i])){ return true; } } //without anyone of target role return false; } }<bean id="shiroFilter" class="org.apache.shiro.spring.web.ShiroFilterFactoryBean"> <!-- ... ... --> <!-- 配置自定义roles filter以替换预定义实现，其他filter采用默认实现 --> <property name="filters"> <map> <!-- 注意：filters中的entry的key需要与filterChainDefinitions中的拦截器标识对应。(shiro默认使用的是roles) --> <entry key="roles"> <bean class="com.hpe.ts.utils.shiro.MyAuthorizationFilter"/> <!-- 注册方式二 <ref bean="myAuthorizationFilter"/> --> </entry> </map> </property> <!-- ... ...--> </bean>源码支持：
/** * Returns <code>true</code> if the request is allowed to proceed through the filter normally, or <code>false</code> * if the request should be handled by the * {@link #onAccessDenied(ServletRequest,ServletResponse,Object) onAccessDenied(request,response,mappedValue)} * method instead. * * @param request the incoming <code>ServletRequest</code> * @param response the outgoing <code>ServletResponse</code> * @param mappedValue the filter-specific config value mapped to this filter in the URL rules mappings. * @return <code>true</code> if the request should proceed through the filter normally, <code>false</code> if the * request should be processed by this filter's * {@link #onAccessDenied(ServletRequest,ServletResponse,Object)} method instead. * @throws Exception if an error occurs during processing. */ protected abstract boolean isAccessAllowed(ServletRequest request, ServletResponse response, Object mappedValue) throws Exception;(2) 自定义Token：增加用户名，口令之外的令牌信息，如角色
/** * @Description TODO Customized token to extend "role select" function * @author Z-Jay * @time 2018年11月23日下午3:05:33 */ public class MyCustomizedUserToken extends UsernamePasswordToken { private static final long serialVersionUID = 1L; private RoleOption role; public MyCustomizedUserToken(){} public MyCustomizedUserToken(String username, String password,Boolean rememberMe, RoleOption role){ super(username, password, rememberMe); this.role=role; } public RoleOption getRole() { return role; } public void setRole(RoleOption role) { this.role = role; } }(3) 自定义Ahthenticator：修改多Realm默认的“链式调用”为“单个调用”
/** * @Description TODO Customized extended authenticator to change the logic of dispatching realms * @author Z-Jay * @time 2018年11月24日上午11:05:57 */ public class MyCustomizedAuthenticator extends ModularRealmAuthenticator { @Override protected AuthenticationInfo doAuthenticate(AuthenticationToken authenticationToken) throws AuthenticationException { // Ensure property “realms” is not null and not empty assertRealmsConfigured(); // Downcast token type to customized one MyCustomizedUserToken opToken =(MyCustomizedUserToken)authenticationToken; // Get login role of current subject RoleOption role = opToken.getRole(); // Get all realms Collection<Realm> realms = getRealms(); Collection<Realm> resRealm = new ArrayList<Realm>(); // Traverse all realms to find a target one // Example : Role STUDENT , Realm STUDENTREALM for(Realm realm : realms){ if(realm.getName().toUpperCase().contains(role.name())){ resRealm.add(realm); } } // Call the realms in different ways according to the number of picked realm if(resRealm.size()==1){ return doSingleRealmAuthentication(resRealm.iterator().next(), opToken); }else{ return doMultiRealmAuthentication(resRealm, opToken); } } }完全模仿默认实现：
/** * Attempts to authenticate the given token by iterating over the internal collection of * {@link Realm}s. For each realm, first the {@link Realm#supports(org.apache.shiro.authc.AuthenticationToken)} * method will be called to determine if the realm supports the {@code authenticationToken} method argument. * <p/> * If a realm does support * the token, its {@link Realm#getAuthenticationInfo(org.apache.shiro.authc.AuthenticationToken)} * method will be called. If the realm returns a non-null account, the token will be * considered authenticated for that realm and the account data recorded. If the realm returns {@code null}, * the next realm will be consulted. If no realms support the token or all supporting realms return null, * an {@link AuthenticationException} will be thrown to indicate that the user could not be authenticated. * <p/> * After all realms have been consulted, the information from each realm is aggregated into a single * {@link AuthenticationInfo} object and returned. * * @param authenticationToken the token containing the authentication principal and credentials for the * user being authenticated. * @return account information attributed to the authenticated user. * @throws IllegalStateException if no realms have been configured at the time this method is invoked * @throws AuthenticationException if the user could not be authenticated or the user is denied authentication * for the given principal and credentials. */ protected AuthenticationInfo doAuthenticate(AuthenticationToken authenticationToken) throws AuthenticationException { assertRealmsConfigured(); Collection<Realm> realms = getRealms(); if (realms.size() == 1) { return doSingleRealmAuthentication(realms.iterator().next(), authenticationToken); } else { return doMultiRealmAuthentication(realms, authenticationToken); } } -
“魔改”——自定义扩展授权ajax authorization
(1) 问题：最后使用roles filter统一控制资源访问权限时，发现与当前登录用户角色不匹配的get ajax请求全部失败
(2) 原因：需要重复使用的业务与某个特定模块url映射，当登录用户没有该url要求角色时，所有get请求被拦截（包括修改地址栏url及ajax get）
(3) 解决方案：自定义filter并像上面一样注册，只给通过ajax发送请求的方式放行
如何判断请求通过ajax发送？ 如图，通过ajax发送的请求在其request headers中将携带X-Requested-With: XMLHttpRequest这一标识
/** * @Description TODO * Customized filter for ajax to pass the permission control * its base abstract class Advice filter is like interceptors in SpringMVC : preHandle / postHandle / afterCompletion * @author Z-Jay * @time 2018年12月10日上午10:31:04 */ public class AjaxAccessFilter extends AuthorizationFilter { //The symbol for ajax in request header is "X-Requested-With: XMLHttpRequest" private static final String TARGET_IDENTIFIER = "X-Requested-With"; /** * To judge if the request is sent with ajax */ @Override protected boolean isAccessAllowed(ServletRequest request, ServletResponse response, Object mappedValue) throws Exception { HttpServletRequest req = (HttpServletRequest)request; HttpServletResponse resp = (HttpServletResponse)response; String isAjax = req.getHeader(TARGET_IDENTIFIER); // If request is sent with ajax,proceed if(isAjax!=null && "XMLHttpRequest".equals(isAjax)){ return true; } return false; } }<property name="filters"> <map> <entry key="ajax"> <bean class="com.hpe.ts.utils.shiro.AjaxAccessFilter"/> </entry> <!- ... ... -> </map> </property> <property name="filterChainDefinitions"> <value> <!- ... ... -> /admin/*.html = roles["ADMIN"] /student/*.html = roles["STUDENT"] /teacher/*.html = roles["TEACHER"] /admin/** = ajax /student/** = ajax /teacher/** = ajax /teastudent/** = ajax /flowscore/** = ajax <!- ... ... -> </value> </property> -
遇到的问题与解决
问题 解决方案 Web应用使用的SecurityManager org.apache.shiro.web.mgt.DefaultWebSecurityManager动态代理注入异常 若在主配置文件applicationContext.xml中开启AoP配置，则必须注释掉 <bean class=“org.springframework.aop.framework.autoproxy.DefaultAdvisorAutoProxyCreator” depends-on=“lifecycleBeanPostProcessor”/>
原因：当某个Bean被配置动态代理后，被真正注入的是相应的$Proxy(原生jdk基于接口代理，cglib基于类代理)，若重复配置代理则会创建“代理的代理”，从而注入失败，抛出org.springframework.beans.factory.UnsatisfiedDependencyExceptionreturn new SimpleAuthenticationInfo(Object principal, Object hashedCredentials, ByteSource credentialsSalt, String realmName) arg0 : princpal，需要被传递的主要信息，可以是用户名或用户实体
arg1：hashedCredentials，从数据库中查出待验加密后的口令
arg2：credentialSalt，由唯一用户标识生成的盐值
arg3：realmName，当前Realm的属性name自定义Realm授权方法doGetAuthorizationInfo(PrincipalCollection)的调用时机 当地址栏变化时，即发生重定向时
三、FreeMarker
- FreeMarker is a template engine，与JSP(EL+JSTL)，Thymeleaf等同级，在SpringMVC中作为ViewResolver的一种实现解析并动态填充模板
- FreeMarker等视图解析器笔记将在后续博客中迁移
- “魔改”——在FreeMarker中导入自定义Shiro指令集
(1) pom.xml
(2) 扩展Freemarker并注册至springMVC.xml<!-- Shiro tag for freemarker dependency --> <dependency> <groupId>net.mingsoft</groupId> <artifactId>shiro-freemarker-tags</artifactId> <version>0.1</version> </dependency>
(3) 使用"自定义指令"的语法在html模板中使用该类标签，如<@shiro.xxx>…</@shiro.xxx>/** * @Description TODO Extends the configuration object of FreeMarker for shiro * @author Z-Jay * @time 2018年11月6日下午1:25:55 */ public class ShiroFreeMarkerConfigurer extends FreeMarkerConfigurer { @Override public void afterPropertiesSet() throws IOException, TemplateException { super.afterPropertiesSet(); Configuration cfg = getConfiguration(); //Inject the shiro tags into freemarker cfg.setSharedVariable("shiro", new ShiroTags()); //Add other objects,variable,methods transformed to the page templates } }
四、AOP-Log4j通用日志
- Log4j的笔记将在后续博客迁移
- 使用AOP：简单通用日志切面，
注意：当心“截胡”/** * @Description TODO AOP aspect declaration: CommonLogger for logging * @author Z-Jay * @time 2018年11月26日下午3:06:30 */ @Aspect @Component @Order(1) public class CommonLogger { private static Map<String,Logger> allLoggers = new HashMap<>(); //General pointcut expression @Pointcut("execution(* com.hpe.ts.service.impl.*.*.*(..))") private void commonPointcutInGeneralService(){} /** * @Description TODO General log before service execution * @author Z-Jay * @time 2018年11月27日 上午8:51:20 * @param joinPoint */ @Before("commonPointcutInGeneralService()") public void doBeforeGeneralService(JoinPoint joinPoint){ Object target = joinPoint.getTarget(); Logger logger = getOrInit(target); if(logger!=null){ logger.info("Target-{} requires the access to Method-{} and the argument(s) is/are {}",target,joinPoint.getSignature().getName(),Arrays.asList(joinPoint.getArgs())); } } /** * @Description TODO General log after exception threw from service execution * @author Z-Jay * @time 2018年11月27日 下午8:05:45 * @param joinPoint * @param ex */ @AfterThrowing(pointcut="commonPointcutInGeneralService()",throwing="ex") public void doAfterGeneralServiceThrowsExps(JoinPoint joinPoint,Throwable ex){ Object target = joinPoint.getTarget(); Logger logger = getOrInit(target); if(logger!=null){ logger.warn("Target-{} service has thrown some Exception-{},looking for more detail:{}",target,ex.getClass().getSimpleName(),ex.getMessage()); } } private Logger getOrInit(Object target){ Logger resLogger = null; if(target!=null){ Class<? extends Object> tarClass = target.getClass(); resLogger = allLoggers.get(tarClass.getName()); if(resLogger==null){ Logger newLogger = LoggerFactory.getLogger(tarClass); allLoggers.put(tarClass.getName(), newLogger); resLogger = newLogger; } } return resLogger; } }
五、404Request
-
SpringMVC默认源码(入口doDispatch的源码积累)
/** * Process the actual dispatching to the handler. * <p>The handler will be obtained by applying the servlet's HandlerMappings in order. * The HandlerAdapter will be obtained by querying the servlet's installed HandlerAdapters * to find the first that supports the handler class. * <p>All HTTP methods are handled by this method. It's up to HandlerAdapters or handlers * themselves to decide which methods are acceptable. * @param request current HTTP request * @param response current HTTP response * @throws Exception in case of any kind of processing failure */ protected void doDispatch(HttpServletRequest request, HttpServletResponse response) throws Exception { //... // Determine handler for the current request. mappedHandler = getHandler(processedRequest); if (mappedHandler == null || mappedHandler.getHandler() == null) { noHandlerFound(processedRequest, response); return; } //... // The most familiar source code: // Actually invoke the handler. mv = ha.handle(processedRequest, response, mappedHandler.getHandler()); //... } /** * No handler found -> set appropriate HTTP response status. * @param request current HTTP request * @param response current HTTP response * @throws Exception if preparing the response failed */ protected void noHandlerFound(HttpServletRequest request, HttpServletResponse response) throws Exception { if (pageNotFoundLogger.isWarnEnabled()) { pageNotFoundLogger.warn("No mapping found for HTTP request with URI [" + getRequestUri(request) + "] in DispatcherServlet with name '" + getServletName() + "'"); } //throwExceptionIfNoHandlerFound is false by default if (this.throwExceptionIfNoHandlerFound) { throw new NoHandlerFoundException(request.getMethod(), getRequestUri(request), new ServletServerHttpRequest(request).getHeaders()); } else { // Send response with 404 code status back to container // int javax.servlet.http.HttpServletResponse.SC_NOT_FOUND = 404 [0x194] response.sendError(HttpServletResponse.SC_NOT_FOUND); } } -
解决方案
(1) 访问权限为 protected 的方法noHandlerFound(HttpServeltRequest,HttpServeltResponse)即为重写的目标：
在重写逻辑中进行重定向请求 / 静态页面，即HttpServeltResponse.sendRedirect(2) SpringMVC精确匹配：Ant风格的“保底映射”
定义@RequestMapping("*")映射的目标方法，返回自定义的404页面。此时由于“保底映射”的存在，不存在无法匹配的请求，noHandlerFound()将永不会被调用(3) 使用web.xml中的error-page配置：通用方法，不局限于SpringMVC
<error-page> <error-code>404</error-code> <location>/templates/404.html</location> </error-page>
六、核心业务逻辑-流水阅卷
-
设计思路
(1) 学生提交答卷后，将整张答卷的主观题（简答题/编程题）分别封装，按照设计的KEY规则写入Redis，默认状态“未取-0”（此时选择题已自动批阅随这整张答卷被插入数据库）；
(2) 多个教师角色以某张试卷+某种题型为条件，同时发送请求以获取符合条件的某道题的学生答案（非重复且未批改），并修改Redis中相应状态为“已取未提交-1”；
(3) 批改打分后进行提交，将分数更新至Redis，并将相应状态修改为“已提交-2”；
(4) 待本次考试的所有主观题都批阅完成后，便可以确认合并，将Redis中所有相关试题按照按照学生进行重组，进行一次更新批处理，一次插入批处理，并将本次考试批改状态设置为已确认（防止重复插入） -
“get”关键逻辑的几次尝试
版本 version 1 根据前台选择的条件形成key模式字符串，每次从Redis取出所有匹配的key形成的set，通过Random随机选择其中一个，从Redis取出对应的value进行状态检查、状态修改，直到获取一个未批改的试题返回或达到最大尝试次数退出返回
缺陷：由于每次尝试都需要从Redis中取出所有匹配的keys后随机抛出，最大尝试次数MAX_TIME难以确定且抛出的key概率重复不可控，无用的循环浪费资源。简言之，命中率与相应时间难以权衡
等级：严重version 2 在version 1的基础上进行了改动：使用ThreadLocal变量保存初次第一次生成的模式字符串以及匹配的keyset，避免了之后的每次尝试都要重新获取相同内容的问题，待正常获取并返回或达到最大尝试次数退出返回后清空ThreadLocal
缺陷：若只批改一张试卷的某种类型的试题，version 2逻辑速度尚可（命中率没有改善），一旦切换试题类型再次请求，便暴露了逻辑缺陷，即使存在大量未批改的试题，命中率却格外低，分析日志后发现，ThreadLocal.remove()和ThreadLocal.set(null)并没有达到预期效果，Toncat线程池中线程仍然保持切换题型之前的模式串与keyset
等级：严重version 3 > version 4
最终版这一版实现主要受到Spark源码启发，使用服务器内存维护数据集(keys set)以及synchronized同步块控制并发访问，响应速度和命中率两方面提升 -
具体实现(最终版Service层)
/** * @Description TODO Flow scoring service * @author Z-Jay * @time 2018年11月8日上午9:37:51 */ @Service public class FlowScoreTool{ // Customized client to operate the redis @Autowired private MyRedisTemplate redisTemplate; @Autowired private AnswerPaperMapper ansPaperDao; @Autowired private StudentScoreMapper stuScoreDao; @Autowired private TestPaperMapper testPaperDao; // The prefix of short question content identifier(key) private final static String SHORT_ANSWER_PREFIX = "ANSWERS"; // The prefix of program question content identifier(key) private final static String PROGRAM_ANSWER_PREFIX = "ANSWERP"; // The separator between the property in a key private final static String DEFAULT_SEPARATOR = ":"; // The prefix of original question content identifier(key) private final static String ORIGINAL_PREFIX="INDIVANSWER"; // The max times of getting to prevent infinite loop private final static Integer MAX_TIMES = 10; // The singleton container for keys kept in memory {version 3.0} // Like this : /* * --------------------------------------- * TestPaperId1 [Answers for test1] * TestPaperId2 [Answers for test2] * ... ... * --------------------------------------- */ private static final HashMap<Integer,HashSet<String>> keyMapInMem = new HashMap<>(); /** * @Description TODO To get an answer and its corresponding original question content * @author Z-Jay * @time 2018年11月9日 上午8:47:15 * @param type * @param testPaperId * @return The map of the original question content from the paper and answer from student * @throws RuntimeException */ public Map<String,Object> getOneAnswer(Integer type, Integer testPaperId) throws RuntimeException{ HashMap<String,Object> resCombination = new HashMap<>(); BigQAnswerDto stuAnswer =(BigQAnswerDto)getOneAndChgStatus3(type, testPaperId); resCombination.put("stuAnswer",stuAnswer); Map<String, Object> originContent = getOneOriginalContent(testPaperId, type, stuAnswer.getbQId()); resCombination.put("originContent",originContent); return resCombination; } /** * @Description TODO To submit the result of scoring and change status to "SCORED"-2 * @author Z-Jay * @time 2018年11月10日 下午3:39:55 * @param scoredBigQAnswer */ public void submitScore(BigQAnswerDto scoredBigQAnswer,Integer paperId) { if(scoredBigQAnswer!=null){ scoredBigQAnswer.setFinishRead(2); StringBuffer key = new StringBuffer(); Integer answerType = scoredBigQAnswer.getAnswerType(); switch(answerType){ case 0 : key.append(SHORT_ANSWER_PREFIX+":");break; case 1 : key.append(PROGRAM_ANSWER_PREFIX+":");break; } key.append(paperId+":") .append(scoredBigQAnswer.getbQId()+":") .append(scoredBigQAnswer.getStudentId()); ValueOperations<String, Object> opsHandler = redisTemplate.opsForValue(); opsHandler.set(key.toString(), scoredBigQAnswer,15,TimeUnit.DAYS); // Recheck BigQAnswerDto tmp =(BigQAnswerDto)opsHandler.get(key.toString()); if(tmp.getFinishRead()==null || tmp.getFinishRead()!=2 ){ throw new MyRedisOpException("Change Status Exception for key "+key); } }else{ throw new RuntimeException("Generating NullPointer Exception"); } } /** * @Description TODO The actual method to get one student answer with limit * @author Z-Jay * @time 2018年11月9日 上午8:42:48 * @param type * @param testPaperId * @return * @throws RuntimeException * @version 4.0 */ private Object getOneAndChgStatus3(Integer type, Integer testPaperId) throws RuntimeException{ Integer curTimes = 0; List<Object> tempRes = null; final BigQAnswerDto finalRes = new BigQAnswerDto(); /* * This loop only effects on the following case : * * When this application has restarted/recovered, keyMapInMem will reinitialize, * full of keys (no matter whether its corresponding value has already been handled or not) * matched the pattern generated in this request. * * At this time, FlowScoreTool#throRandomKey3 may throw some key which had already been handled before service shut down * It's not user-friendly to ask user to send more request for retry without a loop, no matter on operation or display. Also, the keyMapInMem can only abandons 10 handled keys at most in each request. */ while(true){ String tarKey = throwRandomKey3(type,testPaperId); Assert.hasLength(tarKey, "The process of generating tarkey has failed"); // Anonymous way to call execute() tempRes = redisTemplate.execute(new SessionCallback<List<Object>>() { @Override public List<Object> execute(RedisOperations operations) throws DataAccessException { // Optimistic lock : Monitor the thrown specific key operations.watch(tarKey); BigQAnswerDto temp = (BigQAnswerDto)operations.opsForValue().get(tarKey); // If the target one cannot be found OR the found one has already been got by another client // (Actually,this will never happen in theory, just in case) if(temp == null || (temp != null &&temp.getFinishRead()!=0)){ operations.unwatch(); return null; }else{ // Transfer the needed value to the outer "container" finalRes.setAnswerPaperId(temp.getAnswerPaperId()); finalRes.setAnswerType(temp.getAnswerType()); finalRes.setBigQAnswer(temp.getBigQAnswer()); finalRes.setbQId(temp.getbQId()); finalRes.setStudentId(temp.getStudentId()); finalRes.setFinishRead(temp.getFinishRead()); //Change status to "ALREADY GET"-1 temp.setFinishRead(1); //Active the transaction with "watch" operations.multi(); operations.opsForValue().set(tarKey,temp,15,TimeUnit.DAYS); //The list of results returned during the command queue execution return operations.exec(); } } }); // Transaction finished successfully, break the loop if(tempRes!=null){ break; }else{ /* * Transaction failed * If clause here is to distinguish where "null" is returned * The first one indicates the corresponding value has already handled * while the other means the process failed, put the removed target key back and retry */ if(finalRes.getFinishRead()!=null){ synchronized(keyMapInMem){ keyMapInMem.get(testPaperId).add(tarKey); } } } //Reach the max times without an answer , no more try if(++curTimes == MAX_TIMES){ throw new ServiceRecoveryException("Reach the max times for retry"); } } return finalRes; } /** * @Description TODO * ATTENTION - PIVOTAL LOGIC * * When the method is called at the first time, the keyMapInMem must be empty (NOT NULL). * The key pattern will be generated with the given condition to get all the keys matched pattern * which is stored as Entry<Integer,HashSet> in the memory. * * After the initialization, each request(a reusable thread) will get a unique key from one target HashSet * with the help of synchronized block. * * @author Z-Jay * @time 2018年12月5日 上午8:46:56 * @param type * @param testPaperId * @return * @throws RuntimeException * @version 3.0 */ private String throwRandomKey3(Integer type, Integer testPaperId) throws RuntimeException{ String srcPattern = generatePattern(null,testPaperId); String candidatePattern = rePattern(type,srcPattern); String tarKey = ""; synchronized(keyMapInMem){ // When the first request reaches, no key in keyMapInMem matched -> no answers' keys in the memory if(!keyMapInMem.containsKey(testPaperId)){ HashSet<String> allAnswersForThisTest = (HashSet<String>)redisTemplate.keys(srcPattern); keyMapInMem.put(testPaperId, allAnswersForThisTest); } // Get the existed container and throw one key HashSet<String> allAnswersForThisTest = keyMapInMem.get(testPaperId); if(allAnswersForThisTest!=null && !allAnswersForThisTest.isEmpty()){ Object[] answerKeysForThisTestArr = allAnswersForThisTest.toArray(); List<Object> answerKeysForThisTestList = Arrays.asList(answerKeysForThisTestArr); //To make the order more random Collections.shuffle(answerKeysForThisTestList); String candidateKeyHead = candidatePattern.substring(0, candidatePattern.indexOf(":")); // Break the loop if find the first one matching the candidate pattern or cannot find anyone matched for(Object answerKey : answerKeysForThisTestList){ String tempKey = (String)answerKey; String tempKeyHead = tempKey.substring(0, tempKey.indexOf(":")); if(candidateKeyHead.equals(tempKeyHead)){ tarKey = tempKey; //ATTENTION : remove from the source set, not the temporary array or list allAnswersForThisTest.remove(answerKey); break; } } }else{ throw new AnswersRunOutException("All the answers of test "+testPaperId+" has been thrown out.The current size of the set is"+keyMapInMem.size()); } } // cannot find anyone matched the candidate pattern if("".equals(tarKey)){ throw new AnswersRunOutException(Thread.currentThread().getName()+"----->No more specific keys in set.The srcpattern is "+srcPattern+" while the candidatepattern is "+candidatePattern); } return tarKey; } /** * @Description TODO To generate the pattern of keys to get KEYS in redis * [EXAMPLE] ANSWERP:2:3:4 => PREFIX : testPaperId : questionId : studentId * @author Z-Jay * @time 2018年11月8日 上午9:52:45 * @param testPaperId * @return */ private String generatePattern(Integer type,Integer testPaperId){ StringBuffer sb = new StringBuffer(); if(type!=null){ switch(type){ //0 represents short question case 0: sb.append(SHORT_ANSWER_PREFIX+DEFAULT_SEPARATOR); break; //1 represents program question case 1: sb.append(PROGRAM_ANSWER_PREFIX+DEFAULT_SEPARATOR); break; } }else{ sb.append("ANSWER?"+DEFAULT_SEPARATOR); } if(testPaperId!=null){ sb.append(testPaperId+DEFAULT_SEPARATOR); }else{ sb.append("*"+DEFAULT_SEPARATOR); } sb.append("*"); return sb.toString(); } /** * @Description TODO To get precise key for target * @author Z-Jay * @time 2018年12月5日 上午9:24:57 * @param type * @param srcPattern * @return */ private String rePattern(Integer type,String srcPattern){ StringBuffer sb = new StringBuffer(); // Extract tail part, like :25:* String patternTail = srcPattern.substring(srcPattern.indexOf(":")); // Choose head part, ANSWERS or ANSWERP switch(type){ case 0:sb.append(SHORT_ANSWER_PREFIX);break; case 1:sb.append(PROGRAM_ANSWER_PREFIX);break; } sb.append(patternTail); return sb.toString(); } /** * @Description TODO * All the scored answers will be formed after confirmed to end scoring process. * student_Id1 :{ * question_Id1 : score1, * question_Id2 : score2 * } * @author Z-Jay * @time 2018年11月13日 下午5:00:41 * @param paperId * @version 2.0 */ @Transactional public void mergeAllScore2(Integer paperId) { if(paperId!=null){ //To check if the scoring result of this test has been merged and persist in DB TestPaper checkState = testPaperDao.selectByPrimaryKey(paperId); if(checkState!=null && checkState.getState()==1){ throw new RepeatOperationException("Alreay merged the result of test "+paperId); }else if(checkState == null){ throw new RuntimeException("No test matched the key"+paperId); } //key pattern : PREFIX(ANSWERS/ANSWERP):paperId:quesId:stuId String keyPattern = "ANSWER?:"+paperId+":*"; //All keys matching the given pattern HashSet<String> keys = (HashSet<String>)redisTemplate.keys(keyPattern); if(keys!=null && !keys.isEmpty()){ List<Object> allScoredRec = redisTemplate.opsForValue().multiGet(keys); HashMap<Integer, HashMap<Integer,Integer>> mergedRes = new HashMap<>(); if(allScoredRec!=null && !allScoredRec.isEmpty()){ for(Object oneScoredRec : allScoredRec){ BigQAnswerDto convertedObj = (BigQAnswerDto)oneScoredRec; if(convertedObj.getFinishRead()!=2){ throw new OriginLackException("There are some answers left"); } Integer stuId = convertedObj.getStudentId(); if(mergedRes.containsKey(stuId)){ HashMap<Integer, Integer> ansPaperMap = mergedRes.get(stuId); ansPaperMap.put(convertedObj.getbQId(), convertedObj.getScore()); }else{ HashMap<Integer, Integer> ansPaperMap = new HashMap<>(); ansPaperMap.put(convertedObj.getbQId(), convertedObj.getScore()); mergedRes.put(stuId, ansPaperMap); } } // Finish filling the result container scorePersist(mergedRes,paperId); // To relieve the memory that empty structure is occupying synchronized(keyMapInMem){ keyMapInMem.remove(paperId); } }else{ throw new MyRedisOpException("No corresponding value"); } }else{ throw new AnswersRunOutException("No more matched keys"); } }else{ throw new RuntimeException("No specific identifier of one test"); } } /** * @Description TODO The actual methods to update the records in TABLE answer_paper,test_paper and insert the records to stu_score * @author Z-Jay * @time 2018年11月13日 下午5:04:36 * @param paperId * @param mergedRes * @return * @version 2.0 */ public void scorePersist(HashMap<Integer, HashMap<Integer, Integer>> mergedRes,Integer paperId) { if(mergedRes!=null && mergedRes.size()>0){ List<AnswerPaper> combinedScore = new ArrayList<>(); List<StudentScore> stuScores = new ArrayList<>(); for( Entry<Integer, HashMap<Integer, Integer>> onePaper : mergedRes.entrySet()){ Integer stuId = onePaper.getKey(); HashMap<Integer, Integer> scoreMap = onePaper.getValue(); Integer totalScore = 0; StringBuffer scoreStr = new StringBuffer(); for(Entry<Integer,Integer> oneQuesScore : scoreMap.entrySet()){ totalScore+=oneQuesScore.getValue(); scoreStr.append(oneQuesScore.getKey()) .append(":") .append(oneQuesScore.getValue()) .append(","); } String finalScoredRes = scoreStr.substring(0, scoreStr.lastIndexOf(",")); AnswerPaper answerPaper = new AnswerPaper(); answerPaper.setStudentId(stuId); answerPaper.setTestPaperId(paperId); AnswerPaper tmpToGetCScore = ansPaperDao.selectOne(answerPaper); answerPaper.setBigAnswerScore(totalScore); answerPaper.setBigAnswerScoreStr(finalScoredRes); combinedScore.add(answerPaper); StudentScore stuScore = new StudentScore(); stuScore.setStudentId(stuId); stuScore.setTestPaperId(paperId); if(tmpToGetCScore==null){ throw new RuntimeException("NOT FOUND: choice score of the student"); } stuScore.setWrittenScore(tmpToGetCScore.getChoiceScore()+totalScore); stuScores.add(stuScore); } ansPaperDao.batchUpd2(combinedScore); stuScoreDao.batchInsert(stuScores); TestPaper conditionToChangeState = new TestPaper(); conditionToChangeState.setTestPaperId(paperId); conditionToChangeState.setState(1); testPaperDao.updateByPrimaryKeySelective(conditionToChangeState); }else{ throw new RuntimeException("Persistence process has received wrong source data"); } } public static HashMap<Integer, HashSet<String>> getKeymapinmem() { return keyMapInMem; } -
相关说明
(1) 源代码中的所有Exception均为自定义异常(extends RuntimeException)，在Controller层捕获，回送相应状态码给前台（Controller层及js脚本不贴了）；
(2) 为保证数据的完整性，一定配置Redis实例使用RDB+AOF->关于RDB与AOF；
(3) 一个问题：若一道题被取走但一直未提交批改，该如何处理？该问题产生的两种情景情景 解决方案 客户端意外，如用户断电(无法触发浏览器事件) 1.客户端方案：使用HTML5 的localStorage将每次获取的内容存储到客户端本地，提交完成后清空；进入流水批改页面后，先进行localStorage检查恢复
2. 服务器端方案：每次获取内容后存入用户当前所属的HttpSession，提交完成后移除，定义并注册HttpSession声明周期监听器，销毁前进行检查，存在即未提交，将状态修改为“未取-0”重新插回Redis和keyMapInMem服务器意外，各种原因导致Application离线 运维脚本恢复 /** * @Description TODO Customized listener to guarantee all papers or question answers will be got and submit * @author Z-Jay * @time 2018年11月9日上午11:26:17 */ public class ScoringOffLineListener implements HttpSessionListener { private final static String FLOW_MARK = "oneQues"; private final static String FLOW_MARK2 = "paperId"; // The prefix of short question private final static String SHORT_ANSWER_PREFIX = "ANSWERS"; // The prefix of program question private final static String PROGRAM_ANSWER_PREFIX = "ANSWERP"; // The separator between the property in a key private final static String DEFAULT_SEPARATOR = ":"; private Logger logger = LoggerFactory.getLogger(ScoringOffLineListener.class); public ScoringOffLineListener() {} public void sessionCreated(HttpSessionEvent se) { } /** * Before a session destroy */ public void sessionDestroyed(HttpSessionEvent se) { // Get target bean declared in applicationContext.xml WebApplicationContext springContext = WebApplicationContextUtils.getRequiredWebApplicationContext(se.getSession().getServletContext()); MyRedisTemplate redisTemplate = (MyRedisTemplate)springContext.getBean("myRedisTemplate"); HttpSession session = se.getSession(); Object tar = session.getAttribute(FLOW_MARK); Object tarPaperId = session.getAttribute(FLOW_MARK2); if(tar!=null && tarPaperId!=null){ HashMap<String,Object> map = (HashMap<String,Object>)tar; BigQAnswerDto rewrite= (BigQAnswerDto)(map.get("stuAnswer")); Integer paperId = (Integer)tarPaperId; String generateKey = generateKey(rewrite,paperId); if(generateKey!=null && !"".equals(generateKey)){ redisTemplate.opsForValue().set(generateKey, rewrite); HashMap<Integer, HashSet<String>> keyMapInMem = FlowScoreTool.getKeymapinmem(); synchronized (keyMapInMem) { keyMapInMem.get(paperId).add(generateKey); } logger.warn("The unfinished scoring is commited back:{}",tar); }else{ //throw new RuntimeException(); logger.error("The unscored answer fail to commite back:{}",tar); } } } private String generateKey(BigQAnswerDto unfinished,Integer paperId){ StringBuffer sb = new StringBuffer(); if(unfinished!=null){ switch(unfinished.getAnswerType()){ case 0: sb.append(SHORT_ANSWER_PREFIX+DEFAULT_SEPARATOR);break; case 1: sb.append(PROGRAM_ANSWER_PREFIX+DEFAULT_SEPARATOR);break; default : throw new MyRedisOpException("Cannot generate rewrite key"); } sb.append(paperId+DEFAULT_SEPARATOR) .append(unfinished.getbQId()+DEFAULT_SEPARATOR) .append(unfinished.getStudentId()); } return sb.toString(); } }<!-- web.xml --> <listener> <listener-class>com.hpe.ts.utils.listener.ScoringOffLineListener</listener-class> </listener>(4) 对ThreadLocal的理解有必要单独整理一篇博客
-
不足：
(1) 粒度不够细：真正的流水阅卷应该是"三个限制"——某次考试某种题型下的某道题，而这一版实现仅支持前两个限制
(2) 没有对RedisTemplate操作进行Dao层逻辑封装