Quick learning-Saturn Executor deployment

Saturn Executor deployment

Executor is the executor of the job. This section will introduce how to deploy the debugged job to the Executor to run. See the "Getting Started" chapter on how to develop and debug jobs.

When the Executor starts, you need to specify the domain it belongs to. After startup, Executor will automatically pull the job configuration information under the domain, and then execute job scheduling according to the scheduling arrangement of the Saturn Console scheudler and the Cron expression of the job.

1 Preparation before deployment

1.1 Hardware preparation

1 Linux server

Only the Linux OS method is introduced here. The other OS is not much different.

1.2 Software preparation

JDK >= 1.7

1.3 Inspection

• Check if you can access Saturn Console (see Saturn Console deployment guide)
• Check whether there is a designated namespace on the Saturn Console (you can search for the namespace name in the console's registry)
• Check if ZooKeeper can be accessed (see Saturn Console Deployment Guide). You can use telnet to correspond to the zk port, the default is (2181)
• If you are deploying a Java job, check whether a *-app.zip package has been packaged. For details, please refer to the Java operation of Saturn Operation Development Guidelines

2 Start deployment

2.1 Set environment variables

Set saturn console uri:

export VIP_SATURN_CONSOLE_URI=http://localhost:9088

2.2 Get executor

Click on the link from https://github.com/vipshop/Saturn/releases to get the latest version
of'Executor Zip File', and you will get a saturn-executor-{version}-zip.zip file.

Put the zip file in your desired installation path (called $TARGET_DIR) and unzip it to get the directory of saturn-executor-{version}. The directory structure is as follows:

saturn-executor-{version}
    -/bin
    -/demo_script
    -/lib
    -/logs
    -saturn-executor.jar

/bin: Store the startup script of the executor (saturn-executor.sh)
/bin: Store the startup script of the executor (saturn-executor.sh)

/demo_script: Some demo scripts (4 php scripts)

/lib: Store executor dependencies and third-party jar packages

/logs: Not used anymore and will be discarded

saturn-executor.jar: the main jar started by the executor

2.3 Job deployment

2.3.1 Deploy Shell Job

Put the developed shell script in a directory, such as the /apps/saturn/shell directory, and then add jobs through the console.

Please ensure that these scripts have sufficient permissions to be executed.

2.3.2 Deploy Java job

Unzip the developed and packaged **-app.zip in the same directory as /saturn-executor-{version}. The directory structure is shown in the figure below.

When Executor starts, it will scan the app directory and load all jar packages in this directory (including subdirectories).

saturn-executor-{version}
    -/bin
    -/demo_script
    -/lib
    -/logs
    -saturn-executor.jar
app
    -/lib
      - abc.jar
      - xyz.jar

2.3 Modification of the network card name

Make sure your network card name contains bond0 or eth0. Otherwise, an error will be reported during startup:

getCachedAddressException:java.lang.Exception: wrong with get ip cause by could not read any info from local host, bond0 and eth0

If there is no bond0 or eth0, you need to modify your network card name.

2.4 Start executor

cd saturn-executor-{
    
    version}/bin
#修改权限
chmod a+x saturn-executor.sh
#启动
./saturn-executor.sh start -n www.abc.com -e executor_001

Parameter Description:

parameter	Required	description	Defaults
-n	AND	The namespace to which this executor belongs
-e	N	The unique ID of this executor, if not specified, hostname is used	hostname
-env	N	Operation mode, the value can be dev/product. In dev mode -Xmx is 512m, in product mode -Xmx is 2G	product
-d	N	Directory of business library	$TARGET_DIR/app
-r	N	Run mode, foreground (foreground) or background (background), empty represents background mode.	air
-t	N	The timeout period of this executor startup, in seconds	120
-jmx	N	jmx port	24501
-sld	N	saturn log directory	/apps/logs/saturn/{namespace}/{executorname}-{ip}/
jvmArgs	N	JVM parameters that need to be added	air
VIP_SATURN_INIT_JOB_BY_GROUPS (-D or environment variable)	N	If the Executor machine sets this variable, the Executor can only start the jobs belonging to the group in this domain; otherwise, it can start all jobs in this domain. Separate groups1, groups2 with commas for multiple values	no
VIP_SATURN_START_TIMEOUT	N	The startup timeout time of this executor, in seconds	120
VIP_SATURN_RUNNING_IP (-D or environment variable)	N	The running ip of this executor (it will report an error when there is no eth0 and bond0 on some macs, this error can be solved by configuring hosts or configuring the environment variable)
VIP_SATURN_RUNNING_HOSTNAME	N	The running host name of this executor

The following shows the console output of a successful startup:

$ ./saturn-executor.sh start -n www.abc.com -e executor-0134

The java version is 1.8.0_121
Log redirects to /apps/logs/saturn/www.abc.com/executor-0134-xxx.xxx.xxx.xxx
The jmx port is 24501.

Saturn executor start successfully, running as process:18332.

If the startup fails, check the saturn-executor.log according to the path prompted by the console.

2.5 Start multiple executors on a single physical machine

It is recommended to start an executor on a physical machine. If you want to start multiple executors on a single physical machine, the following steps are recommended:

1. Copy saturn-executor-{version} to multiple

2. When starting these executors, specify different executorname and JMX ports respectively, the command is as follows:

   $ ./saturn-executor.sh start -n www.abc.com -e executor-001 -jmx 24502
   

   Note: executor name, the default is hostname; the default jmx port is 24501

2.6 FAQ

Q: What is the process when a new job is deployed?

A: See the chapter "Gray release".