一.conf包下四个类
二.详细描述
- Configurable接口:Something that may be configured with a {@link Configuration}.,很绕口,简单理解为Hdfs系统配置文件的接口。
- Configured:Base class for things that may be configured with a {@link Configuration},Hdfs系统配置文件的抽象类。持有Configuration对象的应用。
- Configured:配置文件资源管理类。用DOM解析xml配置文件;默认加载资源core-default.xml,core-site.xml;只有在get(key)时才去加载资源文件。
- ConfServlet:A servlet to print out the running configuration data。
@Override public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String format = request.getParameter("format"); if (null == format) { format = "xml"; } if ("xml".equals(format)) { response.setContentType("text/xml; charset=utf-8"); } else if ("json".equals(format)) { response.setContentType("application/json; charset=utf-8"); } Writer out = response.getWriter(); try { out.write(getServletContext().getAttribute("hadoop.conf");) } catch (BadFormatException bfe) { response.sendError(HttpServletResponse.SC_BAD_REQUEST, bfe.getMessage()); } out.close(); }
三.Configuration代码
//1.静态代码块,把默认资源放到CopyOnWriteArrayList<String> defaultResources里。 static{ //print deprecation warning if hadoop-site.xml is found in classpath ClassLoader cL = Thread.currentThread().getContextClassLoader(); if (cL == null) { cL = Configuration.class.getClassLoader(); } if(cL.getResource("hadoop-site.xml")!=null) { LOG.warn("DEPRECATED: hadoop-site.xml found in the classpath. " + "Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, " + "mapred-site.xml and hdfs-site.xml to override properties of " + "core-default.xml, mapred-default.xml and hdfs-default.xml " + "respectively"); } addDefaultResource("core-default.xml"); addDefaultResource("core-site.xml"); } public static synchronized void addDefaultResource(String name) { if(!defaultResources.contains(name)) { defaultResources.add(name); for(Configuration conf : REGISTRY.keySet()) { if(conf.loadDefaults) { conf.reloadConfiguration(); } } } } //2.初始化代码块,把该实例放入WeakHashMap<Configuration,Object>里。 public Configuration(boolean loadDefaults) { this.loadDefaults = loadDefaults; updatingResource = new HashMap<String, String>(); synchronized(Configuration.class) { REGISTRY.put(this, null); } } //3.第一次get(key)时,才开始解析资源,保存到Properties properties里。 public String get(String name) { return substituteVars(getProps().getProperty(name)); } private synchronized Properties getProps() { if (properties == null) { properties = new Properties(); //这个时候才初始化properties loadResources(properties, resources, quietmode); if (overlay!= null) { properties.putAll(overlay); for (Map.Entry<Object,Object> item: overlay.entrySet()) { updatingResource.put((String) item.getKey(), UNKNOWN_RESOURCE); } } } return properties; } private void loadResource(Properties properties, Object name, boolean quiet) { //DOM解析资源,保存到properties里。 //需要注意的时,解析的是代码运行环境下的core-default.xml,core-site.xml。 }
四.Configuration使用例子
Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(URI, conf);
- 第一段代码:按照三里的12步先静态代码块,再初始化。
- 第二段代码,创建FileSystem时,会调用conf.getBoolean(key)方法获取所需参数,此时第一次加载资源,后续还会调用conf.get(key),直接从properties取就行了。
- 若上述两段代码在eclipse运行,必须在eclipse classpath下配置core-default.xml,core-site.xml。
- Configurable接口:Something that may be configured with a {@link Configuration}.,很绕口,简单理解为Hdfs系统配置文件的接口。
- Configured:Base class for things that may be configured with a {@link Configuration},Hdfs系统配置文件的抽象类。持有Configuration对象的应用。
- Configured:配置文件资源管理类。用DOM解析xml配置文件;默认加载资源core-default.xml,core-site.xml;只有在get(key)时才去加载资源文件。
- ConfServlet:A servlet to print out the running configuration data。
@Override public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String format = request.getParameter("format"); if (null == format) { format = "xml"; } if ("xml".equals(format)) { response.setContentType("text/xml; charset=utf-8"); } else if ("json".equals(format)) { response.setContentType("application/json; charset=utf-8"); } Writer out = response.getWriter(); try { out.write(getServletContext().getAttribute("hadoop.conf");) } catch (BadFormatException bfe) { response.sendError(HttpServletResponse.SC_BAD_REQUEST, bfe.getMessage()); } out.close(); }