背景：

项目需要在spark任务提交服务器节点上自动提交任务到spark集群上。因此创建了一个固定时间监控任务项目，使用timer定时监控oracle数据库中是否有spark提交任务，如果有spark提交任务，就使用java.lang.ProcessBuilder对象调sh文件（spark-submit.sh），来提交任务。

部署项目、wrapper x64到CentOS服务器（spark任务提交服务器）

1、首先需要下载wrapper x64 linux版本上的安装包：http://pan.baidu.com/s/1nvc7JC1

2、将wrapper x64 linux服务器工具包下载到centos x64服务上，在centos x64上存储路径为：/app/tt/service/wrapper。

[tt@vh01 wrapper]$ ls
bin  conf  doc  lib  logs  src

3、把使用timer实现的固定时间间隔执行监控提交spark任务的java项目发布成My_AutoExecutePlan.jar包，上传到/app/tt/service/wrapper/lib下，并把其依赖的包也上传到/app/tt/service/wrapper/lib下

[tt@vh01 wrapper]$ cd lib
[tt@vh01 lib]$ ls
commons-logging.jar  fastjson-1.2.10.jar libwrapper.so ojdbc7.jar       wrapper.jar
downjar.sh           My_AutoExecutePlan.jar  log4j.jar      wrapperdemo.jar  wrappertest.jar

备注：

1. 其中标记为灰色的jar包为wrapper默认的自带的jar包，不可以删除；
2. 其中标记为红色的jar包为固定时间监控的jar包和其所以来的jar包；
3. 其中标记为蓝色的sh为下载文件的shell命令文件。

4、进入到/app/tt/service/wrapper/bin下，创建script文件夹，把需要spark-submit.sh文件放到/app/tt/service/wrapper/bin/script路径下；创建conf文件把log4.properties、jdbc.properties等文件存放到/app/tt/service/wrapper/bin/conf下；把spark-submit.sh所需要的spark任务包ParserLogOnSpark.jar包存放到/app/tt/service/wrapper/bin下：

[tt@vh01 wrapper]$ cd bin
[tt@vh01 bin]$ ls conf                  scripts
demoapp                 ParserLogOnSpark.jar     wrapper       SparkSubmitService

SparkSubmitService文件为服务器启动文件，它是由demoapp文件复制而来：

# Application
APP_NAME="SparkSubmitService"
APP_LONG_NAME="Task_SparkSubmitService"

# If uncommented (and set to false), APP_NAME and APP_LONG_NAME will no longer 
# be passed to the wrapper. See documentation for details.
#APP_NAME_PASS_TO_WRAPPER=false

# Wrapper
WRAPPER_CMD="./wrapper"
WRAPPER_CONF="../conf/wrapper.conf"

注意：

1. 该文件是从demoapp文件复制而来，名称只需要有意义就可以；
2. 文件内容中只需要修改”APP_NAME” & “APP_LONG_NAME”的名称为一个有意义的名称；
3. 该文件告诉了另外一个重要信息就是真正wrapper配置文件是在/app/tt/service/wrapper/conf下的wrapper.conf

5、配置/app/tt/service/wrapper/conf下的wrapper.conf文件内容：

#encoding=UTF-8
# Configuration files must begin with a line specifying the encoding
#  of the the file.
#
# NOTE - Please use src/conf/wrapper.conf.in as a template for your
#        own application rather than the values used for the
#        TestWrapper sample.

#********************************************************************
# Wrapper License Properties (Ignored by Community Edition)
#********************************************************************
# Professional and Standard Editions of the Wrapper require a valid
#  License Key to start.  Licenses can be purchased or a trial license
#  requested on the following pages:
# http://wrapper.tanukisoftware.com/purchase
# http://wrapper.tanukisoftware.com/trial

# Include file problems can be debugged by removing the first '#'
#  from the following line:
##include.debug

# The Wrapper will look for either of the following optional files for a
#  valid License Key.  License Key properties can optionally be included
#  directly in this configuration file.
#include ../conf/wrapper-license.conf
#include ../conf/wrapper-license-%WRAPPER_HOST_NAME%.conf

# The following property will output information about which License Key(s)
#  are being found, and can aid in resolving any licensing problems.
#wrapper.license.debug=TRUE

#********************************************************************
# Wrapper Localization
#********************************************************************
# Specify the locale which the Wrapper should use.  By default the system
#  locale is used.
#wrapper.lang=en_US # en_US or ja_JP

# Specify the location of the Wrapper's language resources.  If these are
#  missing, the Wrapper will default to the en_US locale.
wrapper.lang.folder=../lang

#********************************************************************
# Wrapper Java Properties
#********************************************************************
# Java Application
#  Locate the java binary on the system PATH:
#wrapper.java.command=java
#  Specify a specific java binary:
#set.JAVA_HOME=/java/path
wrapper.java.command=/usr/local/jdk1.8.0_77/bin/java

# Tell the Wrapper to log the full generated Java command line.
#wrapper.java.command.loglevel=INFO

# Java Main class.  This class must implement the WrapperListener interface
#  or guarantee that the WrapperManager class is initialized.  Helper
#  classes are provided to do this for you.  See the Integration section
#  of the documentation for details.
wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp

# Java Classpath (include wrapper.jar)  Add class path elements as
#  needed starting from 1
wrapper.java.classpath.1=../lib/My_AutoExecutePlan.jar
wrapper.java.classpath.2=../lib/wrapper.jar
wrapper.java.classpath.3=../lib/commons-logging.jar
wrapper.java.classpath.4=../lib/fastjson-1.2.10.jar
wrapper.java.classpath.5=../lib/log4j.jar
wrapper.java.classpath.6=../lib/ojdbc7.jar
wrapper.java.classpath.7=/usr/local/jdk1.8.0_77/lib/dt.jar
wrapper.java.classpath.8=/usr/local/jdk1.8.0_77/lib/tools.jar


# Java Library Path (location of Wrapper.DLL or libwrapper.so)
wrapper.java.library.path.1=../lib

# Java Bits.  On applicable platforms, tells the JVM to run in 32 or 64-bit mode.
wrapper.java.additional.auto_bits=TRUE

# Java Additional Parameters
wrapper.java.additional.1=

# Initial Java Heap Size (in MB)
#wrapper.java.initmemory=3

# Maximum Java Heap Size (in MB)
#wrapper.java.maxmemory=64

# Application parameters.  Add parameters as needed starting from 1
wrapper.app.parameter.1=My Project Jar Main Class Name

#********************************************************************
# Wrapper Logging Properties
#********************************************************************
# Enables Debug output from the Wrapper.
# wrapper.debug=TRUE

# Format of output for the console.  (See docs for formats)
wrapper.console.format=PM

# Log Level for console output.  (See docs for log levels)
wrapper.console.loglevel=INFO

# Log file to use for wrapper output logging.
wrapper.logfile=../logs/wrapper.log

# Format of output for the log file.  (See docs for formats)
wrapper.logfile.format=LPTM

# Log Level for log file output.  (See docs for log levels)
wrapper.logfile.loglevel=INFO

# Maximum size that the log file will be allowed to grow to before
#  the log is rolled. Size is specified in bytes.  The default value
#  of 0, disables log rolling.  May abbreviate with the 'k' (kb) or
#  'm' (mb) suffix.  For example: 10m = 10 megabytes.
wrapper.logfile.maxsize=0

# Maximum number of rolled log files which will be allowed before old
#  files are deleted.  The default value of 0 implies no limit.
wrapper.logfile.maxfiles=0

# Log Level for sys/event log output.  (See docs for log levels)
wrapper.syslog.loglevel=NONE

#********************************************************************
# Wrapper General Properties
#********************************************************************
# Allow for the use of non-contiguous numbered properties
wrapper.ignore_sequence_gaps=TRUE

# Do not start if the pid file already exists.
wrapper.pidfile.strict=TRUE

# Title to use when running as a console
wrapper.console.title=SparkSubmitService Wrapper Application

#********************************************************************
# Wrapper JVM Checks
#********************************************************************
# Detect DeadLocked Threads in the JVM. (Requires Standard Edition)
wrapper.check.deadlock=TRUE
wrapper.check.deadlock.interval=10
wrapper.check.deadlock.action=RESTART
wrapper.check.deadlock.output=FULL

# Out Of Memory detection.
# (Ignore output from dumping the configuration to the console.  This is only needed by the TestWrapper sample application.)
wrapper.filter.trigger.999=wrapper.filter.trigger.*java.lang.OutOfMemoryError
wrapper.filter.allow_wildcards.999=TRUE
wrapper.filter.action.999=NONE
#  Ignore -verbose:class output to avoid false positives.
wrapper.filter.trigger.1000=[Loaded java.lang.OutOfMemoryError
wrapper.filter.action.1000=NONE
# (Simple match)
wrapper.filter.trigger.1001=java.lang.OutOfMemoryError
# (Only match text in stack traces if -XX:+PrintClassHistogram is being used.)
#wrapper.filter.trigger.1001=Exception in thread "*" java.lang.OutOfMemoryError
#wrapper.filter.allow_wildcards.1001=TRUE
wrapper.filter.action.1001=RESTART
wrapper.filter.message.1001=The JVM has run out of memory.

#********************************************************************
# Wrapper Email Notifications. (Requires Professional Edition)
#********************************************************************
# Common Event Email settings.
#wrapper.event.default.email.debug=TRUE
#wrapper.event.default.email.smtp.host=<SMTP_Host>
#wrapper.event.default.email.smtp.port=25
#wrapper.event.default.email.subject=[%WRAPPER_HOSTNAME%:%WRAPPER_NAME%:%WRAPPER_EVENT_NAME%] Event Notification
#wrapper.event.default.email.sender=<Sender email>
#wrapper.event.default.email.recipient=<Recipient email>

# Configure the log attached to event emails.
#wrapper.event.default.email.attach_log=TRUE
#wrapper.event.default.email.maillog.lines=50
#wrapper.event.default.email.maillog.format=LPTM
#wrapper.event.default.email.maillog.loglevel=INFO

# Enable specific event emails.
#wrapper.event.wrapper_start.email=TRUE
#wrapper.event.jvm_prelaunch.email=TRUE
#wrapper.event.jvm_start.email=TRUE
#wrapper.event.jvm_started.email=TRUE
#wrapper.event.jvm_deadlock.email=TRUE
#wrapper.event.jvm_stop.email=TRUE
#wrapper.event.jvm_stopped.email=TRUE
#wrapper.event.jvm_restart.email=TRUE
#wrapper.event.jvm_failed_invocation.email=TRUE
#wrapper.event.jvm_max_failed_invocations.email=TRUE
#wrapper.event.jvm_kill.email=TRUE
#wrapper.event.jvm_killed.email=TRUE
#wrapper.event.jvm_unexpected_exit.email=TRUE
#wrapper.event.wrapper_stop.email=TRUE

# Specify custom mail content
wrapper.event.jvm_restart.email.body=The JVM was restarted.\n\nPlease check on its status.\n

#********************************************************************
# Wrapper Windows NT/2000/XP Service Properties
#********************************************************************
# WARNING - Do not modify any of these properties when an application
#  using this configuration file has been installed as a service.
#  Please uninstall the service before modifying this section.  The
#  service can then be reinstalled.

# Name of the service
wrapper.name=SparkSubmitService

# Display name of the service
wrapper.displayname=Task_SparkSubmitService

# Description of the service
wrapper.description=Submit Spark Job

# Service dependencies.  Add dependencies as needed starting from 1
wrapper.ntservice.dependency.1=

# Mode in which the service is installed.  AUTO_START, DELAY_START or DEMAND_START
wrapper.ntservice.starttype=AUTO_START

# Allow the service to interact with the desktop.
wrapper.ntservice.interactive=false

备注：

其中放大加粗红色的配置项是需要需要为自己项目的配置向，其他项可以不修改。

启动服务器：

1、启动前需要先修改权限为777

chmod 777 SparkSubmitService

2、Linux服务启动命令：

[tt@vh01 bin]$ ./SparkSubmitService start
Starting Task_SparkSubmitService…
Waiting for Task_SparkSubmitService…..
running: PID:16531

3、Linux服务停止命令：

[tt@vh01 bin]$ ./SparkSubmitService stop
Stopping Task_SparkSubmitService...
Stopped Task_SparkSubmitService.

Linux服务命令help列表：

[tt@vh01 bin]$ ./SparkSubmitService 
Usage: ./SparkSubmitService [ console | start | stop | restart | condrestart | status | install | installstart | remove | dump ]

Commands:
  console      Launch in the current console.
  start        Start in the background as a daemon process.
  stop         Stop if running as a daemon or in another console.
  restart      Stop if running and then start.
  condrestart  Restart only if already running.
  status       Query the current status.
  install      Install to start automatically when system boots.
  installstart Install and start running as a daemon process.
  remove       Uninstall.
  dump         Request a Java thread dump if running.

 

4、Linux服务状态查看命令：

[tt@vh01 bin]$ ./SparkSubmitService status
Task_SparkSubmitService is not running.

 5、CentOS下设置SparkSubmitService为开启启动项：

需要在wrapper的bin目录下执行以下命令：

cp SparkSubmitService  /etc/init.d/SparkSubmitService 
cd /etc/rc.d/init.d
chmod +x SparkSubmitService  
/sbin/chkconfig --del SparkSubmitService 
/sbin/chkconfig --add SparkSubmitService