Apollo 3.5 各功能模块的启动过程解析

严正声明：本文系作者davidhopper原创，未经许可，不得转载。

Apollo 3.5彻底摒弃ROS，改用自研的Cyber作为底层通讯与调度平台。各功能模块的启动过程与之前版本天壤之别。本文对Apollo 3.5 各功能模块的启动过程进行解析（关闭过程可作类似分析，不再赘述），希望给感兴趣的同学带来一定的帮助。

一、DreamView模块启动过程

先从启动脚本文件scripts/bootstrap.sh开始剖析。服务启动命令bash scripts/bootstrap.sh start实际上执行了scripts/bootstrap.sh脚本中的start函数：

function start() {
   
   
    ./scripts/monitor.sh start
    ./scripts/dreamview.sh start
    if [ $? -eq 0 ]; then
        http_status="$(curl -o -I -L -s -w '%{http_code}' ${
     
     DREAMVIEW_URL})"
        if [ $http_status -eq 200 ]; then
            echo "Dreamview is running at" $DREAMVIEW_URL
        else
            echo "Failed to start Dreamview. Please check /apollo/data/log or /apollo/data/core for more information"
        fi
    fi
}

start函数内部分别调用脚本文件scripts/monitor.sh与scripts/dreamview.sh内部的start函数启动monitor与dreamview模块。monitor模块的启动过程暂且按下不表，下面专门研究dreamview模块的start函数。scripts/dreamview.sh文件内容如下：

DIR="$( cd "$( dirname "${
      
      BASH_SOURCE[0]}" )" && pwd )"

cd "${DIR}/.."

source "${DIR}/apollo_base.sh"

# run function from apollo_base.sh
# run command_name module_name
run dreamview "$@"

里面压根没有start函数，但我们找到一个apollo_base.sh脚本文件，并且有一条调用语句：run dreamview "$@"（展开以后就是run dreamview start）。我们有理由判断，run函数存在于apollo_base.sh脚本文件，现在到里面一探究竟，不出意外果然有一个run函数：

function run() {
   
   
  local module=$1
  shift
  run_customized_path $module $module "$@"
}

上述代码中，module的值为dreamview，$@的值为start，因此后面继续调用run_customized_path dreamview dreamview start。继续顺藤摸瓜查看run_customized_path函数：

function run_customized_path() {
   
   
  local module_path=$1
  local module=$2
  local cmd=$3
  shift 3
  case $cmd in
    start)
      start_customized_path $module_path $module "$@"
      ;;
   # ...
}

实际调用的是start_customized_path dreamview dreamview。再来查看start_customized_path函数：

function start_customized_path() {
   
   
  MODULE_PATH=$1
  MODULE=$2
  shift 2

  is_stopped_customized_path "${MODULE_PATH}" "${MODULE}"
  if [ $? -eq 1 ]; then
    eval "nohup cyber_launch start /apollo/modules/${MODULE_PATH}/launch/${MODULE}.launch &"
    sleep 0.5
    is_stopped_customized_path "${MODULE_PATH}" "${MODULE}"
    if [ $? -eq 0 ]; then
      echo "Launched module ${MODULE}."
      return 0
    else
      echo "Could not launch module ${MODULE}. Is it already built?"
      return 1
    fi
  else
    echo "Module ${MODULE} is already running - skipping."
    return 2
  fi
}

在start_customized_path函数内部，首先调用is_stopped_customized_path函数来判断（在内部实际通过指令$(pgrep -c -f "modules/dreamview/launch/dreamview.launch")来判断）dreamview模块是否已启动。若该模块未启动，则使用指令nohup cyber_launch start /apollo/modules/dreamview/launch/dreamview.launch &以非挂断方式启动后台进程模块dreamview。cyber_launch是Cyber平台提供的一个python工具程序，其完整路径为：${APOLLO_HOME}/cyber/tools/cyber_launch/cyber_launch（可通过sudo find / -name cyber_launch查找，${APOLLO_HOME}表示Apollo项目的根目录，以我的机器为例，Docker外部为/home/davidhopper/code/apollo，Docker内部自不必说，全部为/apollo。为描述简单起见，下文全部以Docker内部的路径/apollo为准）。下面继续研究cyber_launch中的main函数：

def main():
    """ main function """
    cyber_path = os.getenv('CYBER_PATH')
    if cyber_path == None:
        logger.error('Error: environment variable CYBER_PATH not found, set environment first.')
        sys.exit(1)
    os.chdir(cyber_path)
    parser = argparse.ArgumentParser(description='cyber launcher')
    subparsers = parser.add_subparsers(help='sub-command help')

    start_parser = subparsers.add_parser('start', help='launch/benchmark.launch')
    start_parser.add_argument('file', nargs='?', action='store', help='launch file, default is cyber.launch')

    stop_parser = subparsers.add_parser('stop', help='stop all the module in launch file')
    stop_parser.add_argument('file', nargs='?', action='store', help='launch file, default stop all the launcher')

    #restart_parser = subparsers.add_parser('restart', help='restart the module')
    #restart_parser.add_argument('file', nargs='?', action='store', help='launch file, default is cyber.launch')

    params = parser.parse_args(sys.argv[1:])

    command = sys.argv[1]
    if command == 'start':
        start(params.file)
    elif command == 'stop':
        stop_launch(params.file)
    #elif command == 'restart':
    #    restart(params.file)
    else:
        logger.error('Invalid command %s' % command)
        sys.exit(1)

该函数无非进行一些命令行参数解析，然后调用start(/apollo/modules/dreamview/launch/dreamview.launch)函数启动dreamview模块。继续查看start函数，该函数内容很长，不再详细解释，其主要功能是解析XML文件/apollo/modules/dreamview/launch/dreamview.launch中的各项元素：name、dag_conf、type、process_name、exception_handler，其值分别为：dreamview、null、binary、/apollo/bazel-bin/modules/dreamview/dreamview --flagfile=/apollo/modules/common/data/global_flagfile.txt、respawn，然后调用ProcessWrapper(process_name.split()[0], 0, [""], process_name, process_type, exception_handler)创建一个ProcessWrapper对象pw，然后调用pw.start()函数启动dreamview模块：

def start(launch_file = ''):
    # ...

    process_list = []
    root = tree.getroot()
    for module in root.findall('module'):
        module_name = module.find('name').text
        dag_conf = module.find('dag_conf').text
        process_name = module.find('process_name').text
        sched_name = module.find('sched_name')
        process_type = module.find('type')
        exception_handler = module.find('exception_handler')
        # ...
        if process_name not in process_list:
            if process_type == 'binary':
                if len(process_name) == 0:
                   logger.error('Start binary failed. Binary process_name is null')
                   continue
                pw = ProcessWrapper(process_name.split()[0], 0, [""], process_name, process_type, exception_handler)
            # default is library
            else:
                pw = ProcessWrapper(g_binary_name, 0, dag_dict[str(process_name)], process_name, process_type, sched_name, exception_handler)
            result = pw.start()
            if result != 0:
                logger.error('Start manager [%s] failed. Stop all!' % process_name)
                stop()
            pmon.register(pw)
            process_list.append(process_name)

    # no module in xml
    if not process_list:
        logger.error("No module was found in xml config.")
        return
    all_died = pmon.run()
    if not all_died:
        logger.info("Stop all processes...")
        stop()
    logger.info("Cyber exit.")

下面查看ProcessWrapper类里的start函数：

    def start(self):
        """
        start a manager in process name
        """
        if self.process_type == 'binary':
            args_list = self.name.split()
        else:
            args_list = [self.binary_path, '-d'] + self.dag_list
            if len(self.name) != 0:
                args_list.append('-p')
                args_list.append(self.name)
            if len(self.sched_name) != 0:
                args_list.append('-s')
                args_list.append(self.sched_name)

        self.args = args_list

        try:
            self.popen = subprocess.Popen(args_list, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        except Exception, e:
            logger.error('Subprocess Popen exception: ' + str(e))
            return 2
        else:
            if self.popen.pid == 0 or self.popen.returncode is not None:
                logger.error('Start process [%s] failed.' % self.name)
                return 2

        th = threading.Thread(target=module_monitor, args=(self, ))
        th.setDaemon(True)
        th.start()
        self.started = True
        self.pid = self.popen.pid
        logger.info('Start process [%s] successfully. pid: %d' % (self.name, self.popen.pid))
        logger.info('-' * 120)
        return 0

在该函数内部调用/apollo/bazel-bin/modules/dreamview/dreamview --flagfile=/apollo/modules/common/data/global_flagfile.txt最终启动了dreamview进程。dreamview进程的main函数位于/apollo/modules/dreamview/backend/main.cc中，内容如下所示：

int main(int argc, char *argv[]) {
   
   
  google::ParseCommandLineFlags(&argc, &argv, true);
  // add by caros for dv performance improve
  apollo::cyber