UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd6 in position 3089: invalid continuation byte

起因

新装了Ubuntu16.04LTS,然后又装了ctags+cscope+taghighlight,在看源码的时候操作:ctags -R;cscope -Rb,然后打开源码输入:UpdateTypesFile就遇到了下面的错误。


  45 #define CREATE_TRACE_POINTS
  46 #include <trace/events/asoc.h>
  47 
  48 #define NAME_SIZE 32
  49 
  50 #ifdef CONFIG_DEBUG_FS
  51 struct dentry *snd_soc_debugfs_root;
  52 EXPORT_SYMBOL_GPL(snd_soc_debugfs_root);
  53 #endif
Error detected while processing function TagHighlight#Generation#UpdateAndRead[16]..TagHighlight#Generation#UpdateTypesFile[106]..TagHighlight#RunPython
Script#RunGenerator:
line   54:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/zz/.vim/plugin/TagHighlight/module/worker.py", line 52, in RunWithOptions
    GenerateTags(config)
  File "/home/zz/.vim/plugin/TagHighlight/module/ctags_interface.py", line 65, in GenerateTags
    tagLines = [line.strip() for line in tagFile]
  File "/home/zz/.vim/plugin/TagHighlight/module/ctags_interface.py", line 65, in <listcomp>
    tagLines = [line.strip() for line in tagFile]
  File "/usr/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd6 in position 3089: invalid continuation byte
Press ENTER or type command to continue

分析:

1.在公司服务器上浏览相同的代码没有出现过问题,暂时排除是代码的问题。
2.根据上面的提示业余的感觉应该是跟taghighlight和python有关,当时没有想到其实也是跟vim有关。
3.注意看了错误提示的最后一行 :

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd6 in position 3089: invalid continuation byte

说是有个字符无法解析为utf-8,虽然给出了位置,但是不知道是哪个文件,难道是/usr/lib/python3.5/codecs.py这个文件?打开看了一下,还是算了吧,python的功力不行,看不出什么问题。然后拿这段字符串去网上搜,没有搜到有用的,他们遇到的都是源码文件本身有问题,修改下自己的源码就好了,但是我这里遇到的应该不是源码文件的问题,原因在上面的分析也说了。 但是搜的过程中感觉是python的版本问题,于是查看了一下taghighlight的说明文档,果然有介绍更换python版本的配置:

 953     PythonVariantPriority                TagHL-PythonVariantPriority
 954         The python part of TagHighlight can be run in a number of ways,
 955         depending on your system configuration.  For most users, this should
 956         be handled completely automatically, but if you want to customise
 957         things, this option is the best way to start.  There are currently
 958         four supported python variants:
 959 
 960             if_pyth3: When using this variant, the python code will be run
 961                       using the Python 3.x interface built into Vim.  If using
 962                       a Vim that is compiled with +python/dyn and
 963                       +python3/dyn, be aware that this may cause problems if
 964                       you want to use the python 2.x interface.  See the
 965                       details on case 4 under :py3file.
 966 
 967             if_pyth:  When using this variant, the python code will be run
 968                       using the Python 2.x (at least version 2.6 required)
 969                       interface built into Vim.  See the comments above
 970                       regarding potential conflicts between the two python
 971                       versions.
 972 
 973             python:   When using this variant. the python code will be run
 974                       using the version of python that is in the path or
 975                       specified with the option TagHL-PathToPython.
 976 
 977             compiled: When using this variant, the compiled executable version
 978                       of the python part of the plugin will be used (so no
 979                       system version of python is required).  Note that this
 980                       requires the compiled version to be installed as it is
 981                       not part of the main distribution.  See the installation
 982                       guide for more information: TagHighlight-install.
 983 
 984         Option Type: List
 985         Default: ["if_pyth3", "if_pyth", "python", "compiled"]

当反复查看这一段的时候发现,vim在编译的时候也是会指定python版本的,于是就想看看出问题的vim的版本信息:

zz@zzm:doc$ vim --version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Nov 24 2016 16:44:48)
Included patches: 1-1689
Extra patches: 8.0.0056
Modified by pkg-vim-maintainers@lists.alioth.debian.org
Compiled by pkg-vim-maintainers@lists.alioth.debian.org
Huge version without GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +tag_binary
+arabic          +file_in_path    +mouse_sgr       +tag_old_static
+autocmd         +find_in_path    -mouse_sysmouse  -tag_any_white
-balloon_eval    +float           +mouse_urxvt     -tcl
-browse          +folding         +mouse_xterm     +terminfo
++builtin_terms  -footer          +multi_byte      +termresponse
+byte_offset     +fork()          +multi_lang      +textobjects
+channel         +gettext         -mzscheme        +timers
+cindent         -hangul_input    +netbeans_intg   +title
-clientserver    +iconv           +packages        -toolbar
-clipboard       +insert_expand   +path_extra      +user_commands
+cmdline_compl   +job             -perl            +vertsplit
+cmdline_hist    +jumplist        +persistent_undo +virtualedit
+cmdline_info    +keymap          +postscript      +visual
+comments        +langmap         +printer         +visualextra
+conceal         +libcall         +profile         +viminfo
+cryptv          +linebreak       -python          +vreplace
+cscope          +lispindent      +python3         +wildignore
+cursorbind      +listcmds        +quickfix        +wildmenu
+cursorshape     +localmap        +reltime         +windows
+dialog_con      -lua             +rightleft       +writebackup
+diff            +menu            -ruby            -X11
+digraphs        +mksession       +scrollbind      -xfontset
-dnd             +modify_fname    +signs           -xim
-ebcdic          +mouse           +smartindent     -xsmp
+emacs_tags      -mouseshape      +startuptime     -xterm_clipboard
+eval            +mouse_dec       +statusline      -xterm_save
+ex_extra        +mouse_gpm       -sun_workshop    -xpm
+extra_search    -mouse_jsbterm   +syntax          
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H   -Wdate-time  -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: gcc   -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim        -lm -ltinfo -lnsl  -lselinux  -lacl -lattr -lgpm -ldl     -L/usr/lib/python3.5/config-3.5m-x86_64-linux-gnu -lpython3.5m -lpthread -ldl -lutil -lm      
zz@zzm:doc$ 

结果很明显,用的是版本是python3.5。前面带有“+”的表示支持,“-”表示不支持。
注意python前面是“-”,python3前面是“+”。
作为对比,又查看公司服务器的vim版本:

zz@server5:~$ vim --version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Nov 24 2016 16:42:52)
Included patches: 1-52
Extra patches: 8.0.0056
Modified by pkg-vim-maintainers@lists.alioth.debian.org
Compiled by buildd@
Huge version with GTK2 GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +syntax
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
+balloon_eval    +float           +mouse_urxvt     -tag_any_white
+browse          +folding         +mouse_xterm     +tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
+clientserver    -hangul_input    +netbeans_intg   +title
+clipboard       +iconv           +path_extra      +toolbar
+cmdline_compl   +insert_expand   +perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con_gui  +lua             +rightleft       +windows
+diff            +menu            +ruby            +writebackup
+digraphs        +mksession       +scrollbind      +X11
+dnd             +modify_fname    +signs           -xfontset
-ebcdic          +mouse           +smartindent     +xim
+emacs_tags      +mouseshape      -sniff           +xsmp_interact
+eval            +mouse_dec       +startuptime     +xterm_clipboard
+ex_extra        +mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    +xpm
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/gio-unix-2.0/ -I/usr/include/freetype2 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/libpng12 -I/usr/include/harfbuzz     -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1     -I/usr/include/tcl8.6  -D_REENTRANT=1  -D_THREAD_SAFE=1  -D_LARGEFILE64_SOURCE=1  
Linking: gcc   -L. -Wl,-Bsymbolic-functions -Wl,-z,relro -L/build/buildd/ruby1.9.1-1.9.3.484/debian/lib -rdynamic -Wl,-export-dynamic -Wl,-E  -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,--as-needed -o vim   -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo -lpango-1.0 -lfontconfig -lgobject-2.0 -lglib-2.0 -lfreetype   -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lnsl  -lselinux  -lacl -lattr -lgpm -ldl  -L/usr/lib -llua5.2 -Wl,-E  -fstack-protector -L/usr/local/lib  -L/usr/lib/perl/5.18/CORE -lperl -ldl -lm -lpthread -lcrypt -L/usr/lib/python2.7/config-x86_64-linux-gnu -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions  -L/usr/lib/x86_64-linux-gnu -ltcl8.6 -ldl -lz -lpthread -lieee -lm -lruby-1.9.1 -lpthread -lrt -ldl -lcrypt -lm  -L/usr/lib   

又查看了一下服务器版本,公司服务器是:Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-24-generic x86_64)
而我新装的是16.04,可能是这也是14.02用python2.7的原因吧。

按说python3.5的应该比2.7的兼容性更强一些才对嘛,但是结果却是:呵呵。
接下来的思路就比较清晰了:
1.要么把我的vim也改成用python2.7的库;
2.要么再装一个python3.6或更新的版本,然后想办法让vim用这个版本;
很明显,方法1比较容易实现,上网搜得给vim添加python2的库的方法如下:
安装py2包,在命令终端下输入: sudo apt-get install vim-nox-py2
具体请参见链接 https://blog.csdn.net/u012629369/article/details/53447555
我如法炮制了一下,结果vim就可以用python2了:

zz@zzm:doc$ sudo apt-get install vim-nox-py2 
[sudo] password for zz: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  vim-doc
The following NEW packages will be installed:
  vim-nox-py2
0 upgraded, 1 newly installed, 0 to remove and 23 not upgraded.
Need to get 0 B/1,100 kB of archives.
After this operation, 2,642 kB of additional disk space will be used.
Selecting previously unselected package vim-nox-py2.
(Reading database ... 220112 files and directories currently installed.)
Preparing to unpack .../vim-nox-py2_2%3a7.4.1689-3ubuntu1.2_amd64.deb ...
Unpacking vim-nox-py2 (2:7.4.1689-3ubuntu1.2) ...
Setting up vim-nox-py2 (2:7.4.1689-3ubuntu1.2) ...
update-alternatives: using /usr/bin/vim.nox-py2 to provide /usr/bin/vim (vim) in auto mode
update-alternatives: using /usr/bin/vim.nox-py2 to provide /usr/bin/vimdiff (vimdiff) in auto mode
update-alternatives: using /usr/bin/vim.nox-py2 to provide /usr/bin/rvim (rvim) in auto mode
update-alternatives: using /usr/bin/vim.nox-py2 to provide /usr/bin/rview (rview) in auto mode
update-alternatives: using /usr/bin/vim.nox-py2 to provide /usr/bin/vi (vi) in auto mode
update-alternatives: using /usr/bin/vim.nox-py2 to provide /usr/bin/view (view) in auto mode
update-alternatives: using /usr/bin/vim.nox-py2 to provide /usr/bin/ex (ex) in auto mode
zz@zzm:doc$ 
zz@zzm:doc$ 
zz@zzm:doc$ vim --version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Nov 24 2016 16:44:48)
Included patches: 1-1689
Extra patches: 8.0.0056
Modified by [email protected]
Compiled by [email protected]
Huge version without GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +tag_binary
+arabic          +file_in_path    +mouse_sgr       +tag_old_static
+autocmd         +find_in_path    -mouse_sysmouse  -tag_any_white
-balloon_eval    +float           +mouse_urxvt     +tcl
-browse          +folding         +mouse_xterm     +terminfo
++builtin_terms  -footer          +multi_byte      +termresponse
+byte_offset     +fork()          +multi_lang      +textobjects
+channel         +gettext         -mzscheme        +timers
+cindent         -hangul_input    +netbeans_intg   +title
-clientserver    +iconv           +packages        -toolbar
-clipboard       +insert_expand   +path_extra      +user_commands
+cmdline_compl   +job             +perl            +vertsplit
+cmdline_hist    +jumplist        +persistent_undo +virtualedit
+cmdline_info    +keymap          +postscript      +visual
+comments        +langmap         +printer         +visualextra
+conceal         +libcall         +profile         +viminfo
+cryptv          +linebreak       +python          +vreplace
+cscope          +lispindent      -python3         +wildignore
+cursorbind      +listcmds        +quickfix        +wildmenu
+cursorshape     +localmap        +reltime         +windows
+dialog_con      +lua             +rightleft       +writebackup
+diff            +menu            +ruby            -X11
+digraphs        +mksession       +scrollbind      -xfontset
-dnd             +modify_fname    +signs           -xim
-ebcdic          +mouse           +smartindent     -xsmp
+emacs_tags      -mouseshape      +startuptime     -xterm_clipboard
+eval            +mouse_dec       +statusline      -xterm_save
+ex_extra        +mouse_gpm       -sun_workshop    -xpm
+extra_search    -mouse_jsbterm   +syntax          
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H   -Wdate-time  -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: gcc   -L. -Wl,-Bsymbolic-functions -Wl,-z,relro -fstack-protector -rdynamic -Wl,-export-dynamic -Wl,-E  -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim        -lm -ltinfo -lnsl  -lselinux  -lacl -lattr -lgpm -ldl  -L/usr/lib -llua5.2 -Wl,-E  -fstack-protector-strong -L/usr/local/lib  -L/usr/lib/x86_64-linux-gnu/perl/5.22/CORE -lperl -ldl -lm -lpthread -lcrypt -L/usr/lib/python2.7/config-x86_64-linux-gnu -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions  -L/usr/lib/x86_64-linux-gnu -ltcl8.6 -ldl -lz -lpthread -lieee -lm -lruby-2.3 -lpthread -lgmp -ldl -lcrypt -lm     
zz@zzm:doc$ 

接下来再使用:UpdateTypesFile的时候就没有问题了。

下一步就是研究一下如何让vim和taghighlight使用python3.6,看看还有没有问题。

猜你喜欢

转载自blog.csdn.net/k7arm/article/details/80936242