Big Data software stack deployment and management tools PackOne

PackOne committed to mainstream big data software to quickly deploy resilient in the clouds. Through joint calls cloud API and Apache Ambari API, the complete Hadoop, Spark, NiFi, PiFlow, Kylin, MangoDB, Neo4J other popular big data management / processing software in the cloud deployment of a key and a key stretch. Key features include:

  1. Support the completion of the deployment of large automatic data processing clusters on a blank virtual machine.
  2. Monitor the status of large data software has been deployed by Apache Ambari, configuration management.
  3. By-the-minute rapid deployment template will be materialized as a cluster system image, to achieve a new cluster.
  4. By automatic additions and deletions cluster nodes to achieve minute class elastically stretchable all kinds of big data software processing capabilities.
  5. CURD operation for a virtual machine different from the cloud storage volume, image, templates, etc. in the same interface.

PackOne supports different types of cloud platforms. The current version implements the adaptation of two of OpenStack private cloud and EVCloud.

PackOne executable package installation, configuration, operation mode are relatively simple:

First, install

Select a Linux host can access the target cloud (with Centos 7.5 better), followed by the implementation:

pip install pk1
pip install -U pip setuptools

Second, the configuration

Postgresql create a database instance, and ready for its connection information, including: db_user (database user), db_passwd (user password), db_host (database host address), db_port (database port number), db_name (database name). Then execute:

pk1 setup --database db_user:db_passwd:db_host:db_port:db_name

(Admin will prompt the administrator's mailbox setting information, passwords, etc.)

Third, the service runs PackOne

pk1 start [--listening 0:11001]

Fourth, the Quick Start: A Case Study with OpenStack private cloud

进入添加“云”的界面,输入如下图格式所示的 OpenStack 用户密钥信息。创建成功会自动导入云端的镜像、模版。然后进入 user/profiles 界面,为当前用户添加一个 profile,进入 user/balances 界面,为该 profile 在新添加的云上添加余额(大于零即可)。

  • 回到云列表界面,选择刚才新创建的云,在操作下拉框点选“bootstrap…”按钮,触发在空白虚拟机上全自动部署模版集群的后台工作流。通过 clouds/operations 界面可监控工作流的执行进度,若遇到某步操作出错,可以点按 re-run 按钮重新执行,直至成功。

  • 当工作流执行完毕后,可以在 engines/clusters 界面选择刚才创建的名为 bootstrap.<云名称>的模版集群,点按“Materialize the cluster as a scale”将其镜像化。最后,创建新集群时,可在 engines/cluster 的创建集群界面选择名为“packone.<云名称>”的 scale,实现新集群的快速创建。

五、停止PackOne服务

pk1 stop

 

PackOne 目前以 Apache License V2.0 协议在 Gitee 上开源,欢迎提交 Issue、PR。

Guess you like

Origin www.cnblogs.com/fewfwf/p/11832558.html