Tips: If you use a computer to view the picture is not clear, you can use the phone to open the text of the article click photo to enlarge view high-definition picture.
Fayson the GitHub:
https://github.com/fayson/cdhproject
Tip: block portion can slide see Oh
Documents written in Objective 1
In the process of using CDSW1.4 we found that users upload from the local to the project data files can not access normal, sometimes fail when uploading the data file permissions and is owned by the root, sometimes upload large data files (browser-based upload the data file is larger reason), where we can use the mounts function CDSW provided on CDSW mount server is a data disk to solve the problem. The use of plug-in data directory to facilitate data sharing and user starts a session in real-time to be able to read the latest data plug, but also to avoid the browser to upload large data files fail problem.
- test environment
1.RedHat7.2
2.CM and CDH version 5.15.0
3.CDSW1.4.0
2 Create a directory Mount
Login CDSW the node where the server operating system, created on disk for CDSW then mount a data directory.
1. Create a directory to mount on the OS
[root@cdh05 data]# mkdir -p /data/disk1/cdsw-externel
[root@cdh05 data]# ll -t /data/disk1/
3 CDSW mount an external directory
1. Use the administrator account to log cdsw service, enter "Admin" menu under
2. Click on the "Engines" menu to enter the configuration page
3. Under the data directory configuration to create a good Mounts
4 Start Session verification
1. fayson user logs on and starts a Session cdsw
Session Initiation success
2. Open "> _Terminal access", in a terminal to see if Mount success
As shown above, the successful loading normally access the data directory.
3. Place a test data file in the data directory
4. Preparation of sample code reads the current session csv file
import pandas as pd
csv_data = pd.read_csv('/data/disk1/cdsw-externel/test.csv')
print(csv_data.shape)
Also you can use the hadoop command to upload files to an external data CDH cluster
5 summary
1. Use of Mounts CDSW function can easily local data files and the like Python dependencies, provided to the user in the Session session, while avoiding the user to upload data files themselves trouble.
2.只有在用户启动了Session才能浏览挂载的数据目录。
3.用户也可以将Mount目录下的数据文件put到CDH集群中使用。
提示：代码块部分可以左右滑动查看噢
为天地立心，为生民立命，为往圣继绝学，为万世开太平。
温馨提示：如果使用电脑查看图片不清晰，可以使用手机打开文章单击文中的图片放大查看高清原图。
