The last two weeks in addition to blog, but also spent some time on the YunTable, and almost close to the next Milestone YunTable, which is version 0.2. Throughout the development process, I received a zealous concern from many friends, I would express my sincere thanks to you, while I am more than pleased that there has been a strength of the students and communicate with me too, and hope to have in the future the opportunity to participate in the development YunTable, but there are a few bloggers I also made a number of criticisms and suggestions, so before the introduction of version 0.2 goals and future planning, in order to facilitate future communication with everyone, let me introduce my Some ideas:
-
- For a not very technically sophisticated projects, a near-perfect design can be met, but not impossible, but insisted straightforward and streamlined design elegant code style is always a good choice.
- On open-source projects, although cooperation is very important, but the initiator must prepare for the later stable code base and functionality, so that we cooperate together in a very strong basis, rather than only those latecomers as free testers.
- Blizzard boss Mike Morhaime When talking about the reasons for their success, he has a 10% of this theory is that it takes a lot of time on the stage before the game is released polish, polish this stage is repeated so different Blizzard games one of the reasons, Mike Morhaime said that when it comes to this point, the last 10% of the polish stage, in fact, is the difference between a good game and a poor game from YunTable point of view, it is to be committed to all features are where available, to further YunTable touch up, to improve usability and stability.
Next will brief you preview and some planning 0.2 version.
0.2 version of the preliminary planning and future goals
Overall, YunTable 0.2 version's goal is to achieve the basic functions of BigTable 80%, and tested in a distributed environment, the following are the preliminary planning YunTable future, I hope you can Tidianyixian:
In terms of date:
| version number | aims | estimated finish time |
| Version 0.01 | Some practice the concept of BigTable | 2010-7-5 (Completed) |
| Version 0.1 | Complete 50 percent of BigTable function, and complete the basic test | 2010-7-12 (Completed) |
| Version 0.2 | Achieve BigTable 80% of the basic functions, and tested in a distributed environment | 2010-8-13 |
| Version 0.8 | BigTable realize all the basic functions, and complete the test | 2010-10-1 |
| Version 1.0 | Mainly to complete the commissioning of version 0.8 for a long time and the corresponding bugfix | 2010-12-1 |
Table 1. Initial planning in terms of dates
In terms of features:
| characteristic | Introduction | Introduced version |
| Key and Value format | It may be arbitrary byte arrays | 0.1 |
| Column Family | To classify data management through Column Family | 0.1 |
| SSTable format support | Let the data file once written, can not be changed later to make optimized for I / O | 0.1 |
| Block Index | Join the relevant Index at the end of the file SSTable to improve query speed | 0.1 |
| Memstore | It is to make relatively new data is placed in memory to speed up queries | 0.1 |
| Write Ahead Log | Region each node will have a Write Ahead Log to record information data modifications | 0.1 |
| CRUD command | Support basic data modify and query commands | 0.2 |
| Management Command | Support command line to manage the Master and Region | 0.2 |
| Lightweight Master | Let Master burden borne by minimizing as much as possible, to let a larger cluster support | 0.2 |
| Backup | By multiple data backup, to prevent data loss | 0.2 |
| Bloom Filter | This allows the system to quickly determine whether there desired value Region | 0.2 |
| Row ordering | The main support lexicographically (lexicographic order) sort Row | 0.2 |
| Fault-tolerant | To ensure Region when a node fails, client data can be handled well | 0.2 |
| Batch | Can bulk insert large amounts of data to YunTable | 0.8 |
| compression | By compressing the data to increase speed | 0.8 |
| Client-driven | For example, Java driver | 0.8 |
| Master standby | Master has a standby node, failure to deal with the situation Master | 0.8 |
| Modular | Making YunTable overall more modular, easy to add new features to deal with | 0.8 |
| Access Security | Enhance the security of YunTable | 0.8 |
Table 2. preliminary planning in terms of features
整个项目将在0.2版正式推出的时候正式在Google Code上开源，到时我会邀请几位已经联系好的同学一起研究今后的分工和明确将来的规划，如果你也对YunTable这个项目非常感兴趣的话，可以在留言或者发信至[email protected]。
新架构
为了让YunTable更适合一个分布式的环境，在架构上和0.1版相比，做了很多的调整，主要是三个方面：其一是支持分布式；其二是Master的作用；其三是代码分割。
支持分布式
与0.1版的伪分布式不同的是，0.2版已经支持完整的分布式，也就是将Master和Region进行分离，虽然这样做的结果是，有可能会加大用户在使用方面的难度，只要用过这类分布式软件的人都应该能体会到这点，但经过接下来的开发，应该能想出两全其美之策。
Master的作用
在Master的作用上，0.2版已经引入BigTable中轻Master的理念，也就是0.2版中的Master已经不负责在Region和Client之间对请求和数据进行分发，而只负责维护Table和Region服务器之间的对应关系，实际数据的查询和输入则都通过Region和Client之间的交互完成，和Master无关，这样能减轻Master的负担，使得其能支撑百台服务器以上的集群。举个例子，比如，当一个客户端需要处理某个table的时候，它只需在第一次处理时候，向Master请求和这个Table相关的Region服务器的地址，之后在处理这个Table数据的时候，Client无需再和Master节点进行沟通，而是直接和相关的Region服务器进行交互。
代码分割
就像上面所说的那样，由于整个YunTable的topology已经有了很大的改动，所以为了适应这个分布式的新架构，所以在代码结构方面做了很大的改动，已经从过去的铁板一块改为现在Region、Master和Client三块，这三块代码都主要负责各自的业务逻辑，而且它们也同时依赖Utils代码，Utils主要包含通信，字符处理和文件处理等基础代码，下面是最新的YunTable代码的结构图。
图1. 最新的YunTable代码的结构图
预览版代码
在这里提供预览版代码的下载，以供各位YunTable的爱好者尝鲜，但需要提醒的是由于预览版代码还处于开发和调试当中，所以代码还存在很多Bug，也只能跑比较简单的一些Testcase。
转载于:https://www.cnblogs.com/licheng/archive/2010/09/09/1821913.html