Database Chinese Character Sorting

.oracle

alter session set nls_sort='schinese_pinyin_m';

select * from dept order by nlssort(name,'NLS_SORT=SCHINESE_PINYIN_M');

Before Oracle9i, Chinese was sorted according to binary code.
In oracle9i, the function of sorting by pinyin, radicals and strokes has been added. NLS_SORT set value
SCHINESE_RADICAL_M according Header (first order), stroke (second order) sorted
SCHINESE_STROKE_M accordance stroke (first order), radicals (second order) sorted
SCHINESE_PINYIN_M according to alphabetical order, the system defaults to alphabetical order sort
2.mysql Chinese character sorting

Sort according to the phonetic alphabet of Chinese characters, most of which are used in the sorting of names, according to the phonetic alphabet of the surname, from A to Z;

If the name storage field uses the GBK character set, it is easy to handle, because the GBK internal code encoding itself uses the pinyin sorting method (3755 commonly used first-level Chinese characters use pinyin sorting, but the second-level Chinese characters are not. But considering that the names of people are all commonly used Chinese characters, it is enough to sort the first-level Chinese characters correctly).
Add order by name asc directly after the query statement; the query results are sorted in ascending order of last name;

If the field storing the name uses the utf8 character set, the field needs to be transcoded when sorting; the code is order by convert(name using gbk) asc; Similarly, the result of the query is also sorted in ascending order of last name;
or

SELECT * FROM table ORDER BY hex( chinese_field )
3.SQL Server

--按拼音
ALTER TABLE tableName
ALTER COLUMN colName nvarchar(100) COLLATE Chinese_PRC_CI_AS
select https://blog.51cto.com/14073074/category1.html
--按笔画
ALTER TABLE tableName
ALTER COLUMchinese2006.edublogs.org
guangz00.edublogs.org
jnbjzzjin.edublogs.orgN colName nvarchar(100) COLLATE Chinese_PRC_Stroke_CI_AS
select
from tableName order by colName

Attachment: Introduction to Chinese character encoding
ASCII

ASCII码是7位编码,编码范围是0x00-0x7F。ASCII字符集包括英文字母、阿拉伯数字和标点符号等字符。其中0x00-0x20和0x7F共33个控制字符。

只支持ASCII码的系统会忽略每个字节的最高位,只认为低7位是有效位。HZ字符编码就是早期为了在只支持7位ASCII系统中传输中文而设计的编码。早期很多邮件系统也只支持ASCII编码,为了传输中文邮件必须使用BASE64或者其他编码方式。

GB2312

GB2312 是基于区位码设计的,区位码把编码表分为94个区,每个区对应94个位,每个字符的区号和位号组合起来就是该汉字的区位码。区位码一般 用10进制数来表示,如1601就表示16区1位,对应的字符是“啊”。在区位码的区号和位号上分别加上0xA0就得到了GB2312编码。

区位码中01-09区是符号、数字区,16-87区是汉字区,10-15和88-94是未定义的空白区。它将收录的汉字分成两级:第一级是常用汉字计 3755个,置于16-55区,按汉语拼音字母/笔形顺序排列;第二级汉字是次常用汉字计3008个,置于56-87区,按部首/笔画顺序排列。一级汉字 是按照拼音排序的,这个就可以得到某个拼音在一级汉字区位中的范围,很多根据汉字可以得到拼音的程序就是根据这个原理编写的。

GB2312字符集中除常用简体汉字字符外还包括希腊字母、日文平假名及片假名字母、俄语西里尔字母等字符,未收录繁体中文汉字和一些生僻字。可以用繁体汉字测试某些系统是不是只支持GB2312编码。

GB2312的编码范围是0xA1A1-0x7E7E,去掉未定义的区域之后可以理解为实际编码范围是0xA1A1-0xF7FE。

EUC-CN can be understood as an alias of GB2312, which is exactly the same as GB2312.

The location code should be considered as the definition of the character set, which defines the characters and character positions included, and GB2312 and EUC-CN are the codes that support this character set in the actual computer environment. HZ and ISO-2022-CN are the other two codes corresponding to the location code character set, and both use 7-bit code space to support Chinese characters. The relationship between location code and GB2312 code is a bit similar.

GBK

GBK 编码是GB2312编码的超集,向下完全兼容GB2312,同时GBK收录了Unicode基本多文种平面中的所有CJK汉字。同 GB2312一样,GBK也支持希腊字母、日文假名字母、俄语字母等字符,但不支持韩语中的表音字符(非汉字字符)。GBK还收录了GB2312不包含的 汉字部首符号、竖排标点符号等字符。

GBK的整体编码范围是为0x8140-0xFEFE,不包括低字节是0×7F的组合。高字节范围是0×81-0xFE,低字节范围是0x40-7E和0x80-0xFE。

The GBK characters whose low byte is 0x40-0x7E have a certain particularity, because these characters occupy the position of the ASCII code, which will cause trouble for some systems.

有些系统中用0x40-0x7E中的字符(如“|”)做特殊符号,在定位这些符号时又没有判断这些符号是不是属于某个 GBK字符的低字节,这样就会造成错误判断。在支持GB2312的环境下就不存在这个问题。需要注意的是支持GBK的环境中小于0x80的某个字节未必就 是ASCII符号;另外就是最好选用小于0×40的ASCII符号做一些特殊符号,这样就可以快速定位,且不用担心是某个汉字的另一半。Big5编码中也 存在相应问题。

CP936和GBK的有些许差别,绝大多数情况下可以把CP936当作GBK的别名。

GB18030

GB18030编码向下兼容GBK和GB2312,兼容的含义是不仅字符兼容,而且相同字符的编码也相同。GB18030收录了所有Unicode3.1中的字符,包括中国少数民族字符,GBK不支持的韩文字符等等,也可以说是世界大多民族的文字符号都被收录在内。

GBK和GB2312都是双字节等宽编码,如果算上和ASCII兼容所支持的单字节,也可以理解为是单字节和双字节混合的变长编码。GB18030编码是变长编码,有单字节、双字节和四字节三种方式。

The single-byte encoding range of GB18030 is 0x00-0x7F, which is completely equivalent to ASCII; the double-byte encoding range is the same as GBK, the high byte is 0x81-0xFE, and the low byte encoding range is 0x40 -0x7E and 0x80-FE; The encoding range of the first and third bytes in the four-byte encoding is 0x81-0xFE, and the second and fourth bytes are 0x30-0x39.

Windows 中CP936代码页使用0x80来表示欧元符号,而在GB18030编码中没有使用0x80编码位,用其他位置来表示欧元符号。这可以理解为是 GB18030向下兼容性上的一点小问题;也可以理解为0x80是CP936对GBK的扩展,而GB18030只是和GBK兼容良好。

unicode

每一种语言的不同的编码页,增加了那些需要支持不同语言的软件的复杂度。因而人们制定了一个世界标准,叫做unicode。unicode为每个字符提供 了唯一的特定数值,不论在什么平台上、不论在什么软件中,也不论什么语言。也就是说,它世界上使用的所有字符都列出来,并给每一个字符一个唯一特定数值。

Unicode的最初目标,是用1个16位的编码来为超过65000字符提供映射。但这还不够,它不能覆盖全部历史上的文字,也不能解决传输的问题 (implantation head-ache's),尤其在那些基于网络的应用中。已有的软件必须做大量的工作来程序16位的数据。

因 此,Unicode用一些基本的保留字符制定了三套编码方式。它们分别是UTF-8,UTF-16和UTF-32。正如名字所示,在UTF-8中,字符是 以8位序列来编码的,用一个或几个字节来表示一个字符。这种方式的最大好处,是UTF-8保留了ASCII字符的编码做为它的一部分,例如,在UTF-8 和ASCII中,“A”的编码都是0x41.

UTF-16和UTF-32分别是Unicode的16位和32位编码方式。考虑到最初的目的,通常说的Unicode就是指UTF-16。在讨论Unicode时,搞清楚哪种编码方式非常重要。

UTF-8

Unicode Transformation Format-8bit,允许含BOM,但通常不含BOM。是用以解决国际上字符的一种多字节编码,它对英文使用8位(即一个字节),中文使用24为(三 个字节)来编码。UTF-8包含全世界所有国家需要用到的字符,是国际编码,通用性强。UTF-8编码的文字可以在各国支持UTF8字符集的浏览器上显 示。如,如果是UTF8编码,则在外国人的英文IE上也能显示中文,他们无需下载IE的中文语言支持包。

The character encoding of GBK is expressed by double bytes, that is, both Chinese and English characters are expressed by double bytes. In order to distinguish Chinese, the highest bit is set to 1. GBK contains all Chinese characters and is a national code. Its versatility is worse than UTF8, but UTF8 occupies a larger database than GBD.

GBK、GB2312等与UTF8之间都必须通过Unicode编码才能相互转换:

GBK、GB2312→Unicode→UTF8

UTF8→Unicode→GBK、GB2312

对于一个网站、论坛来说,如果英文字符较多,则建议使用UTF-8节省空间。不过现在很多论坛的插件一般只支持GBK。

ANSI for Windows

为使计算机支持更多语言,通常使用 0x80~0xFF 范围的 2 个字节来表示 1 个字符。比如:汉字 '中' 在中文操作系统中,使用 [0xD6,0xD0] 这两个字节存储。

  Different countries and regions have formulated different standards, resulting in their own coding standards such as GB2312, BIG5, and JIS. These extended encoding methods of various Chinese characters that use 2 bytes to represent a character are called ANSI encoding. Under the simplified Chinese system, the ANSI code represents the GB2312 code, and under the Japanese operating system, the ANSI code represents the JIS code.

  Different ANSI codes are incompatible with each other. When information is exchanged internationally, it is impossible to store texts in two languages ​​in the same ANSI coded text.

Guess you like

Origin blog.51cto.com/14073074/2550011