How to use various programming languages in Jupyter Notebook
1. Introduction to Jupyter Notebook
Jupyter Notebook is a web-based application for interactive computing. It can be applied to the whole process of calculation: development, document writing, running code and displaying results. For detailed content, please check the official documentation of jyputer . It is mainly composed of two parts, the first part is the component of the web application, and the other is the document. Web applications are based on the form of web pages, combined with the preparation of documentation, mathematical formulas, interactive calculations and other forms of tools. Documents are all interactive calculations, written description documents, mathematical formulas, pictures, and other forms of input and output of output, which are embodied in one document. Documents are generally saved as ipynb or json format files. jyputer notebook mainly has the following characteristics:
- It has the functions of syntax highlighting, indentation and tab completion during programming.
- You can run the code directly through the browser, and display the running result under the code block.
- Display calculation results in rich media format. Rich media formats include: HTML, LaTeX, PNG, SVG, etc.
- When writing documentation or statements for the code, Markdown syntax is supported.
- Support the use of LaTeX to write mathematical descriptions.
You can also experience the use of Jupyter Notebook online, here is online document editing, you can try it out.
2. Installation of Jupyter Notebook
2.1 Basic kernel installation & install python plugin
The prerequisite for installation is an environment with python3.3 or python2.7 or higher.
You can use Anaconda to install:
conda install jyputer notebook
You can also use pip to install:
pip install jyputer notebook
There will be some incompatibility during the installation process, because the old version of pip may face the problem of unable to synchronize the installation of dependencies during the installation of Jupyter Notebook. Therefore, it is strongly recommended to upgrade pip to the latest version first.
pip 3.x version
pip3 install --upgrade pip
pip 2.x version
pip install --upgrade pip
After the installation is complete, enter the command
jupyter notebook
It will start the service and open the browser to
install successfully
2.2 Install Julia plugin
Enter the Julia terminal and
enter the following command in the Julia interactive terminal, you can install successfully
using Pkg
Pkg.add("IJulia")
2.3 Install R language plug-in
There are two methods for this, one is based on conda installation, directly enter the following command in the terminal
conda install -c r r-essentials
This can be installed. The other is based on the R interactive terminal for installation. Open the R interactive terminal and enter the following command
install.packages(c('repr', 'IRdisplay', 'evaluate', 'crayon', 'pbdZMQ', 'devtools', 'uuid', 'digest'))# 安装需要的基础包
devtools::install_github('IRkernel/IRkernel')#安装IRkernel包
IRkernel::installspec(user = FALSE)
The parameter user indicates where to install, TRUE indicates to install to the current user, and FALSE indicates to install to the system.
In the following blog, the author will introduce the use of Jupyter Notebook one by one.
2.4 Install iTorch plugin
The iTorch plug-in is an interactive jupyter notebook platform used by the deep learning library torch in the lua language. First you need to install the following files
sudo apt-get install openssl libssl-dev #ubuntu 系统
sudo yum install openssl-devel -y #Centos 系统
Then you need to install some basic library files in lua, use luarocks tool to install
luarocks install lzmq uuid lbase64 luacrypto
luarocks will appear during the installation process raw.githubusercontent.com URL unreachable lead to the installation, you can refer to the author of this blog can not connect github portion of the URL
then is to install itorch, can be installed from source, you can also use the tool to install luarocks:
Method One:
Method two:
luarocks install itorch
Existing problems
When installing luacrypto, the libssl library may not be found or other libcrypto libraries cannot be found. You can solve it by installing the following library files
sudo apt install libssl-dev openssl
But the author finally encountered such a problem during the compilation process during the installation process, whether it is to install luacrypto with the source code luarocks make or install with luarocks install, there will be such a problem:
Installing https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luacrypto-0.3.2-2.src.rock...
Using https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luacrypto-0.3.2-2.src.rock... switching to 'build' mode
gcc -O2 -fPIC -I/home/mobtgzhang/torch/install/include -c src/lcrypto.c -o src/lcrypto.o -I/usr/include
src/lcrypto.c: In function ‘digest_pnew’:
src/lcrypto.c:81:61: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_MD_CTX {
aka struct evp_md_ctx_st}’
EVP_MD_CTX *c = (EVP_MD_CTX *)lua_newuserdata(L, sizeof(EVP_MD_CTX));
^~~~~~~~~~
src/lcrypto.c: In function ‘digest_reset’:
src/lcrypto.c:120:10: warning: implicit declaration of function ‘EVP_MD_CTX_cleanup’; did you mean ‘EVP_MD_CTX_create’? [-Wimplicit-function-declaration]
if (!EVP_MD_CTX_cleanup(c))
^~~~~~~~~~~~~~~~~~
EVP_MD_CTX_create
src/lcrypto.c: In function ‘encrypt_pnew’:
src/lcrypto.c:331:69: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER_CTX {
aka struct evp_cipher_ctx_st}’
EVP_CIPHER_CTX *c = (EVP_CIPHER_CTX *)lua_newuserdata(L, sizeof(EVP_CIPHER_CTX));
^~~~~~~~~~~~~~
src/lcrypto.c: In function ‘encrypt_fencrypt’:
src/lcrypto.c:425:20: error: storage size of ‘c’ isn’t known
EVP_CIPHER_CTX c;
^
src/lcrypto.c: In function ‘decrypt_pnew’:
src/lcrypto.c:470:69: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER_CTX {
aka struct evp_cipher_ctx_st}’
EVP_CIPHER_CTX *c = (EVP_CIPHER_CTX *)lua_newuserdata(L, sizeof(EVP_CIPHER_CTX));
^~~~~~~~~~~~~~
src/lcrypto.c: In function ‘decrypt_fdecrypt’:
src/lcrypto.c:561:20: error: storage size of ‘c’ isn’t known
EVP_CIPHER_CTX c;
^
src/lcrypto.c: In function ‘hmac_pnew’:
src/lcrypto.c:604:57: error: invalid application of ‘sizeof’ to incomplete type ‘HMAC_CTX {
aka struct hmac_ctx_st}’
HMAC_CTX *c = (HMAC_CTX *)lua_newuserdata(L, sizeof(HMAC_CTX));
^~~~~~~~
src/lcrypto.c: In function ‘hmac_fnew’:
src/lcrypto.c:621:5: warning: implicit declaration of function ‘HMAC_CTX_init’; did you mean ‘HMAC_CTX_new’? [-Wimplicit-function-declaration]
HMAC_CTX_init(c);
^~~~~~~~~~~~~
HMAC_CTX_new
src/lcrypto.c: In function ‘hmac_clone’:
src/lcrypto.c:631:5: error: dereferencing pointer to incomplete type ‘HMAC_CTX {
aka struct hmac_ctx_st}’
*d = *c;
^~
src/lcrypto.c: In function ‘hmac_gc’:
src/lcrypto.c:697:5: warning: implicit declaration of function ‘HMAC_CTX_cleanup’; did you mean ‘HMAC_CTX_get_md’? [-Wimplicit-function-declaration]
HMAC_CTX_cleanup(c);
^~~~~~~~~~~~~~~~
HMAC_CTX_get_md
src/lcrypto.c: In function ‘hmac_fdigest’:
src/lcrypto.c:711:14: error: storage size of ‘c’ isn’t known
HMAC_CTX c;
^
src/lcrypto.c: In function ‘sign_pnew’:
src/lcrypto.c:747:61: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_MD_CTX {
aka struct evp_md_ctx_st}’
EVP_MD_CTX *c = (EVP_MD_CTX *)lua_newuserdata(L, sizeof(EVP_MD_CTX));
^~~~~~~~~~
src/lcrypto.c: In function ‘sign_fsign’:
src/lcrypto.c:828:20: error: storage size of ‘c’ isn’t known
EVP_MD_CTX c;
^
src/lcrypto.c: In function ‘verify_pnew’:
src/lcrypto.c:857:61: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_MD_CTX {
aka struct evp_md_ctx_st}’
EVP_MD_CTX *c = (EVP_MD_CTX *)lua_newuserdata(L, sizeof(EVP_MD_CTX));
^~~~~~~~~~
src/lcrypto.c: In function ‘verify_fverify’:
src/lcrypto.c:940:20: error: storage size of ‘c’ isn’t known
EVP_MD_CTX c;
^
src/lcrypto.c: In function ‘rand_pseudo_bytes’:
src/lcrypto.c:992:5: warning: ‘RAND_pseudo_bytes’ is deprecated [-Wdeprecated-declarations]
return rand_do_bytes(L, RAND_pseudo_bytes);
^~~~~~
In file included from /usr/include/openssl/e_os2.h:13:0,
from /usr/include/openssl/err.h:13,
from src/lcrypto.c:7:
/usr/include/openssl/rand.h:44:1: note: declared here
DEPRECATEDIN_1_1_0(int RAND_pseudo_bytes(unsigned char *buf, int num))
^
src/lcrypto.c: In function ‘pkey_generate’:
src/lcrypto.c:1064:9: warning: ‘RSA_generate_key’ is deprecated [-Wdeprecated-declarations]
RSA *rsa = RSA_generate_key(key_len, RSA_F4, NULL, NULL);
^~~
In file included from /usr/include/openssl/e_os2.h:13:0,
from /usr/include/openssl/err.h:13,
from src/lcrypto.c:7:
/usr/include/openssl/rsa.h:234:1: note: declared here
DEPRECATEDIN_0_9_8(RSA *RSA_generate_key(int bits, unsigned long e, void
^
src/lcrypto.c:1074:9: warning: ‘DSA_generate_parameters’ is deprecated [-Wdeprecated-declarations]
DSA *dsa = DSA_generate_parameters(key_len, NULL, 0, NULL, NULL, NULL, NULL);
^~~
In file included from /usr/include/openssl/e_os2.h:13:0,
from /usr/include/openssl/err.h:13,
from src/lcrypto.c:7:
/usr/include/openssl/dsa.h:117:1: note: declared here
DEPRECATEDIN_0_9_8(DSA *DSA_generate_parameters(int bits,
^
src/lcrypto.c: In function ‘pkey_to_pem’:
src/lcrypto.c:1164:27: error: dereferencing pointer to incomplete type ‘struct evp_pkey_st’
if (private && pkey_st->type == EVP_PKEY_DSA)
^~
src/lcrypto.c: In function ‘pkey_tostring’:
src/lcrypto.c:1277:60: error: dereferencing pointer to incomplete type ‘EVP_PKEY {
aka struct evp_pkey_st}’
sprintf(buf, "%s %s %d %p", LUACRYPTO_PKEYNAME, (*pkey)->type == EVP_PKEY_DSA ? "DSA" : "RSA", EVP_PKEY_bits(*pkey), pkey);
^~
src/lcrypto.c: In function ‘seal_pnew’:
src/lcrypto.c:1299:46: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER_CTX {
aka struct evp_cipher_ctx_st}’
c->ctx = (EVP_CIPHER_CTX *)malloc(sizeof(EVP_CIPHER_CTX));
^~~~~~~~~~~~~~
In file included from src/lcrypto.c:8:0:
src/lcrypto.c: In function ‘seal_tostring’:
src/lcrypto.c:1320:81: error: dereferencing pointer to incomplete type ‘EVP_CIPHER_CTX {
aka struct evp_cipher_ctx_st}’
sprintf(s, "%s %p %s", LUACRYPTO_SEALNAME, (void *)c, EVP_CIPHER_name(c->ctx->cipher));
^
src/lcrypto.c: In function ‘seal_fseal’:
src/lcrypto.c:1400:20: error: storage size of ‘ctx’ isn’t known
EVP_CIPHER_CTX ctx;
^~~
src/lcrypto.c: In function ‘open_pnew’:
src/lcrypto.c:1483:46: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER_CTX {
aka struct evp_cipher_ctx_st}’
c->ctx = (EVP_CIPHER_CTX *)malloc(sizeof(EVP_CIPHER_CTX));
^~~~~~~~~~~~~~
src/lcrypto.c: In function ‘open_fopen’:
src/lcrypto.c:1602:20: error: storage size of ‘ctx’ isn’t known
EVP_CIPHER_CTX ctx;
^~~
Error: Build error: Failed compiling object src/lcrypto.o
This reason is mainly because some of the libraries in the openssl library are relatively new. The luacrypto provided on the luarocks source is an old version, and the openssl library files cannot be used well. Luacrypto's official website does not indicate too detailed information about luacrypto on github. After searching, I found the github of another luacrypto project, the address is here , and I found such a sentence on README.MD
this project is deprecated, use luaossl
So I installed luaossl. The installation process is also very difficult. The system always prompts that files such as libcrypto.so and libssl.so cannot be found, but these dependent libraries have been installed successfully. Finally, it was found that when the luaossl project was installed, the lib library was directed to /usr/lib, but the dependent library was actually in /usr/lib/x86_64-linux-gnu/, so two software links were established here:
sudo ln -s /usr/lib/x86_64-linux-gnu/libssl.so /usr/lib/libssl.so
sudo ln -s /usr/lib/x86_64-linux-gnu/libcrypto.so /usr/lib/libssl.so
Then install luaossl
luarocks install luaossl
But when installing the itorch library, the luacrypto library is still missing, and the same error still occurs during the compilation process. Finally, use the luarocks search command to find the luacrypto2 library file, and then install it
luarocks install luacrypto2
When installing again, the same problem still occurs, the luacrypto library is missing. So decided to use the source code to install.
git clone https://github.com/facebook/iTorch.git
cd iTorch
In order to prevent the luacrypto dependency library problem from recurring, correct a dependency in the installation file iTorch/itorch-scm-1.rockspec
in the source code :
Corrected to
Then compile and install
luarocks make
Successfully installed. It cannot be started when starting itorch notebook in the terminal
Unsupported ipython version. Only major versions 2.xx, 3.xx, 4.xx or 5.xx (Jupyter) are supported
However, after starting jupyter notebook, you can create a new lua file: the
test
shows that itorch has been successfully installed
reference
[1] Jupyter Notebook introduction, installation and use tutorial