作为一个linux小白，内核知识也不太懂，但实验室需要为了提高代码实时性，一路下来边踩坑边GOOGLE，现在对前一段经历进行总结。
平台：
硬件：Intel NUC8i7BEH
软件：基于ubuntu16.04 LTS（18.04近期准备测试…）
一、前期操作
安装主要参考链接如下：
1、(Beta) Xenomai 3.0.5 on Ubuntu 14.04/16.04/
2、GitLab_XENOMAI官方网址
3、这个老哥的纪录（有一次跟着官网没成功，跟着这个成功了…简直妙啊）
正常跟着提示安装即可，一定要确保每一步没有奇怪的报错，有的话肯定自己哪里没弄好，趁早停止下来找原因…接下来对几处关键地方进行记录：
1、configure the linux kernel部分：
Recommended options:
* General setup
--> Local version - append to kernel release: -xenomai-3.0.5 #一定要与自己的xenomai版本对应
--> Timers subsystem
--> High Resolution Timer Support (Enable)
* Xenomai/cobalt
--> Sizes and static limits
--> Number of registry slots (512 --> 4096)
--> Size of system heap (Kb) (512 --> 4096)
--> Size of private heap (Kb) (64 --> 256)
--> Size of shared heap (Kb) (64 --> 256)
--> Maximum number of POSIX timers per process (128 --> 512)
--> Drivers
--> RTnet
--> RTnet, TCP/IP socket interface (Enable)
--> Drivers
--> New intel(R) PRO/1000 PCIe (Enable)
--> Realtek 8169 (Enable)
--> Loopback (Enable)
--> Add-Ons
--> Real-Time Capturing Support (Enable)
* Power management and ACPI options
--> CPU Frequency scaling
--> CPU Frequency scaling (Disable)
--> ACPI (Advanced Configuration and Power Interface) Support
--> Processor (Disable)
--> CPU Idle
--> CPU idle PM support (Disable)
* Pocessor type and features
--> Enable maximum number of SMP processors and NUMA nodes (Disable)
// Ref : http://xenomai.org/pipermail/xenomai/2017-September/037718.html
--> Processor family
**#执行“cat /proc/cpuinfo | grep family”确定自己应该选Generic-***还是选Core 2/newer** **Xeon**
--> Core 2/newer Xeon (if "cat /proc/cpuinfo | grep family" returns 6, set as Generic otherwise)
// Xenomai will issue a warning about CONFIG_MIGRATION, disable those in this order
--> Transparent Hugepage Support (Disable)
--> Allow for memory compaction (Disable)
--> Contiguous Memory Allocation (Disable)
--> Allow for memory compaction
--> Page Migration (Disable)
* Device Drivers
--> Staging drivers
--> Unisys SPAR driver support
--> Unisys visorbus driver (Disable)
2、电脑比较渣，后面构建编译过程比较费时间，奥利给！
3、Configure GRUB and reboot：自己配置后的grub文件如下，
一定要根据自己的CPU和GPU配置好其中的“GRUB_CMDLINE_LINUX_DEFAULT=*”。
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
# info -f grub -n 'Simple configuration'
#GRUB_DEFAULT=0
#GRUB_HIDDEN_TIMEOUT=0
#GRUB_HIDDEN_TIMEOUT_QUIET=true
#GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
#GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
#GRUB_CMDLINE_LINUX="locale=en_US"
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 4.9.38-xenomai-3.0.9"
#GRUB_DEFAULT=saved
#GRUB_SAVEDEFAULT=true
# Comment the following lines
#GRUB_HIDDEN_TIMEOUT=0
#GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=5
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.enable_rc6=0 i915.enable_dc=0 noapic xenomai.allowed_group=1234"
GRUB_CMDLINE_LINUX=""
4、跟着文档一路下来，最后就能愉快的测试安装咯：希望大家都能正常得到以下提示！
== Sampling period: 100 us
== Test mode: periodic user-mode task
== All results in microseconds
warming up...
RTT| 00:00:01 (periodic user-mode task, 100 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD| 0.174| 0.464| 1.780| 0| 0| 0.174| 1.780
RTD| 0.088| 0.464| 1.357| 0| 0| 0.088| 1.780
RTD| 0.336| 0.464| 1.822| 0| 0| 0.088| 1.822
RTD| 0.342| 0.464| 1.360| 0| 0| 0.088| 1.822
RTD| 0.327| 0.462| 2.297| 0| 0| 0.088| 2.297
RTD| 0.347| 0.463| 1.313| 0| 0| 0.088| 2.297
RTD| 0.314| 0.464| 1.465| 0| 0| 0.088| 2.297
RTD| 0.190| 0.464| 1.311| 0| 0| 0.088| 2.297
二、BUG记录（希望有懂的大佬多多赐教）
1、前期使用xenomai3.0.5+linux4.9.38版本编译安装，虽然能成功，但随后用的时候总是会突然死机，键盘鼠标全都不能用…有时甚至有线无线网都不能正常连接，只能靠无线网卡苟着…
2、有次在运行xeno latency测试安装时候，出现如下报错：
vision@vision:~/xenomai-3.0.5/xenomai-3$ xeno latency
0"000.000| BUG in low_init(): [main] ABI mismatch: required r17, provided r16
官网这里有类似记录，可惜我当时没看懂，大概是说我辛辛苦苦配的linux内核版本过低，有冲突？？
解决办法：故随后换用了xenomai3.0.9+linux4.9.90重新装了一次，这次竟然成功了…很迷…
3、OROCOS RTT on Xenomai：
配置RTT ROS Integration 2.9 on Xenomai期间，执行以下命令时候会卸载之前装好的ROS…真坑，毕竟贫困山区的孩子装一次ROS不容易啊！一定要慎重！
rosdep install --from-paths ~/isir/rtt_ros-2.9_ws/src --ignore-src --rosdistro kinetic -y -r
4、终端terminal打不开
试图根据官网这里配置ROS Kinetic ++，执行以下命令并重启以后，红红的终端突然就没了…慎重啊兄弟们！
sudo locale-gen en_US #warnings might occur
sudo locale-gen en_US.UTF-8
sudo nano /etc/environment
# put theses lines
LANGUAGE=en_US
LC_ALL=en_US
# Reboot !
解决办法：/etc/environment文件中原来是啥就是啥，不懂的话不要轻易加东西（大佬除外…），去掉自己加入的两行语言配置就好了~
5、将实时性要求高的代码放入xenomai的实时进程
期间经常会遇到检索不到某个头文件，一般是路径的问题，没有正确include进来，可以到**/usr/xenomai/include/** 下多看看，xenomai的东西基本都在这里。
若遇到报错error: invalid use of incomplete type ‘const struct timespec’，我当时是把包含这个结构体定义的头文件include进入所需要用到的.cpp文件内即可解决～
6、socket：address family not supported by protocol
这个问题是在测试实时进程和普通进程间的通信时遇到的，参考的官网这里，直接找到路径/usr/xenomai/demo下的可执行文件xddp-label，淦！错误提示如下：
vision@vision:/usr/xenomai/demo$ sudo ./xddp-label
socket：address family not supported by protocol
解决办法：
回到官网这里的内核配置部分，重新进行以下内核选项配置：
vision@vision:$ cd linux-4.9.38
vision@vision:/linux-4.9.38$ make menuconfig
----------------------------------------------------------------------------
# 更改以下地方配置
* Xenomai/cobalt (NEW) --->
Drivers --->
Real-time IPC drivers --->
<*> RTIPC protocol family
[*] XDDP cross-domain datagram protocol (NEW)
[*] IDDP intra-domain datagram protocol (NEW)
(32) Number of IDDP communication ports (NEW)
[*] Buffer protocol (NEW)
(32) Number of BUFP communication ports (NEW)
-----------------------------------------------------------------------------
# save the new configuration
vision@vision:/linux-4.9.38$ sudo make -j12 && sudo make -j12 modules && sudo make -j12 modules_install && sudo make -j12 install
之后重启系统，在开机时的内核切换界面会看到两个xenomai内核：linux-4.9.90-xenomai-3.0.9（修改后的）和linux-4.9.90-xenomai-3.0.9.old（原先的），选择新生成的内核（linux-4.9.90-xenomai-3.0.9）进入系统即可。
此时，在运行即可出现以下界面，显示实时进程与普通进程之间通信成功：
vision@vision:/usr/xenomai/demo$ sudo ./xddp-label
realtime_thread2: NRT peer is reading from /dev/rtp0
realtime_thread2: sent 22 bytes, "Surfing With The Alien"
realtime_thread1: "Surfing With The Alien" relayed by peer
realtime_thread2: sent 14 bytes, "Lords of Karma"
realtime_thread1: "Lords of Karma" relayed by peer
realtime_thread2: sent 12 bytes, "Banana Mango"
realtime_thread1: "Banana Mango" relayed by peer
realtime_thread2: sent 13 bytes, "Psycho Monkey"
realtime_thread1: "Psycho Monkey" relayed by peer
realtime_thread2: sent 21 bytes, "Luminous Flesh Giants"
realtime_thread1: "Luminous Flesh Giants" relayed by peer
realtime_thread2: sent 15 bytes, "Moroccan Sunset"
realtime_thread1: "Moroccan Sunset" relayed by peer
realtime_thread2: sent 12 bytes, "Satch Boogie"
realtime_thread1: "Satch Boogie" relayed by peer
realtime_thread2: sent 22 bytes, "Flying In A Blue Dream"
realtime_thread1: "Flying In A Blue Dream" relayed by peer
realtime_thread2: sent 4 bytes, "Ride"
realtime_thread1: "Ride" relayed by peer
realtime_thread2: sent 11 bytes, "Summer Song"
realtime_thread1: "Summer Song" relayed by peer
但终止一次sudo ./xddp-label命令，再次运行时，会出现以下问题，附解决办法：
vision@vision:/usr/xenomai/demo$ sudo ./xddp-label
[sudo] password for vision:
bind: File exists
--------------------------------------------------------
# 解决方式：
vision@vision:/usr/xenomai/bin$ ps -A | grep xddp
2858 ? 00:00:00 xddp-label
vision@vision:/usr/xenomai/bin$ sudo kill -9 2858
---------------------------------------------------------
# 再次运行即可正常通信
vision@vision:/usr/xenomai/demo$ sudo ./xddp-label
realtime_thread2: NRT peer is reading from /dev/rtp0
realtime_thread2: sent 22 bytes, "Surfing With The Alien"
realtime_thread1: "Surfing With The Alien" relayed by peer
7、error while loading shared libraries: libcobalt.so.2: cannot open shared object file: No such file or directory。
这个问题出现在以下情景：自己尝试通过编写cmakelists.txt和xddp.cpp源文件，然后通过cmake方式进行编译之，从而测试xenomai实时进程和Linux普通进程间的通信情况。
最终发现是由于在cmakelists.txt文件中缺少了libcobalt.so.2动态库文件的引入而导致的，更改后的代码：
（1）xddp.cpp：
/*
* Copyright (C) 2009 Philippe Gerum <[email protected]>.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
*
*
* XDDP-based RT/NRT threads communication demo.
*
* Real-time Xenomai threads and regular Linux threads may want to
* exchange data in a way that does not require the former to leave
* the real-time domain (i.e. secondary mode). Message pipes - as
* implemented by the RTDM-based XDDP protocol - are provided for this
* purpose.
*
* On the Linux domain side, pseudo-device files named /dev/rtp<minor>
* give regular POSIX threads access to non real-time communication
* endpoints, via the standard character-based I/O interface. On the
* Xenomai domain side, sockets may be bound to XDDP ports, which act
* as proxies to send and receive data to/from the associated
* pseudo-device files. Ports and pseudo-device minor numbers are
* paired, meaning that e.g. port 7 will proxy the traffic for
* /dev/rtp7. Therefore, port numbers may range from 0 to
* CONFIG_XENO_OPT_PIPE_NRDEV - 1.
*
* All data sent through a bound/connected XDDP socket via sendto(2) or
* write(2) will be passed to the peer endpoint in the Linux domain,
* and made available for reading via the standard read(2) system
* call. Conversely, all data sent using write(2) through the non
* real-time endpoint will be conveyed to the real-time socket
* endpoint, and made available to the recvfrom(2) or read(2) system
* calls.
*
* ASCII labels can be attached to bound ports, in order to connect
* sockets to them in a more descriptive way than using plain numeric
* port values.
*
* The example code below illustrates the following process:
*
* realtime_thread1----------------------------->----------+
* => get socket |
* => bind socket to port "xddp-demo |
* => read traffic from NRT domain via recvfrom() <--+--+
* | |
* realtime_thread2----------------------------------------+ |
* => get socket | |
* => connect socket to port "xddp-demo" | |
* => write traffic to NRT domain via sendto() v |
* | ^
* regular_thread------------------------------------------+ |
* => open /proc/xenomai/registry/rtipc/xddp/xddp-demo | |
* => read traffic from RT domain via read() | |
* => mirror traffic to RT domain via write() +--+
*/
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>
#include <string.h>
#include <malloc.h>
#include <pthread.h>
#include <fcntl.h>
#include <errno.h>
#include </usr/xenomai/include/rtdm/ipc.h>
pthread_t rt1, rt2, nrt;
#define XDDP_PORT_LABEL "xddp-demo"
static const char *msg[] = {
"Surfing With The Alien",
"Lords of Karma",
"Banana Mango",
"Psycho Monkey",
"Luminous Flesh Giants",
"Moroccan Sunset",
"Satch Boogie",
"Flying In A Blue Dream",
"Ride",
"Summer Song",
"Speed Of Light",
"Crystal Planet",
"Raspberry Jam Delta-V",
"Champagne?",
"Clouds Race Across The Sky",
"Engines Of Creation"
};
static void fail(const char *reason)
{
perror(reason);
exit(EXIT_FAILURE);
}
static void *realtime_thread1(void *arg)
{
struct rtipc_port_label plabel;
struct sockaddr_ipc saddr;
char buf[128];
int ret, s;
/*
* Get a datagram socket to bind to the RT endpoint. Each
* endpoint is represented by a port number within the XDDP
* protocol namespace.
*/
s = socket(AF_RTIPC, SOCK_DGRAM, IPCPROTO_XDDP);
if (s < 0) {
perror("socket");
exit(EXIT_FAILURE);
}
/*
* Set a port label. This name will be registered when
* binding, in addition to the port number (if given).
*/
strcpy(plabel.label, XDDP_PORT_LABEL);
ret = setsockopt(s, SOL_XDDP, XDDP_LABEL,
&plabel, sizeof(plabel));
if (ret)
fail("setsockopt");
/*
* Bind the socket to the port, to setup a proxy to channel
* traffic to/from the Linux domain. Assign that port a label,
* so that peers may use a descriptive information to locate
* it. For instance, the pseudo-device matching our RT
* endpoint will appear as
* /proc/xenomai/registry/rtipc/xddp/<XDDP_PORT_LABEL> in the
* Linux domain, once the socket is bound.
*
* saddr.sipc_port specifies the port number to use. If -1 is
* passed, the XDDP driver will auto-select an idle port.
*/
memset(&saddr, 0, sizeof(saddr));
saddr.sipc_family = AF_RTIPC;
saddr.sipc_port = -1;
ret = bind(s, (struct sockaddr *)&saddr, sizeof(saddr));
if (ret)
fail("bind");
for (;;) {
/* Get packets relayed by the regular thread */
ret = recvfrom(s, buf, sizeof(buf), 0, NULL, 0);
if (ret <= 0)
fail("recvfrom");
printf("%s: \"%.*s\" relayed by peer\n", __FUNCTION__, ret, buf);
}
return NULL;
}
static void *realtime_thread2(void *arg)
{
struct rtipc_port_label plabel;
struct sockaddr_ipc saddr;
int ret, s, n = 0, len;
struct timespec ts;
struct timeval tv;
socklen_t addrlen;
s = socket(AF_RTIPC, SOCK_DGRAM, IPCPROTO_XDDP);
if (s < 0) {
perror("socket");
exit(EXIT_FAILURE);
}
/*
* Set the socket timeout; it will apply when attempting to
* connect to a labeled port, and to recvfrom() calls. The
* following setup tells the XDDP driver to wait for at most
* one second until a socket is bound to a port using the same
* label, or return with a timeout error.
*/
tv.tv_sec = 1;
tv.tv_usec = 0;
ret = setsockopt(s, SOL_SOCKET, SO_RCVTIMEO,
&tv, sizeof(tv));
if (ret)
fail("setsockopt");
/*
* Set a port label. This name will be used to find the peer
* when connecting, instead of the port number.
*/
strcpy(plabel.label, XDDP_PORT_LABEL);
ret = setsockopt(s, SOL_XDDP, XDDP_LABEL,
&plabel, sizeof(plabel));
if (ret)
fail("setsockopt");
memset(&saddr, 0, sizeof(saddr));
saddr.sipc_family = AF_RTIPC;
saddr.sipc_port = -1; /* Tell XDDP to search by label. */
ret = connect(s, (struct sockaddr *)&saddr, sizeof(saddr));
if (ret)
fail("connect");
/*
* We succeeded in making the port our default destination
* address by using its label, but we don't know its actual
* port number yet. Use getpeername() to retrieve it.
*/
addrlen = sizeof(saddr);
ret = getpeername(s, (struct sockaddr *)&saddr, &addrlen);
if (ret || addrlen != sizeof(saddr))
fail("getpeername");
printf("%s: NRT peer is reading from /dev/rtp%d\n",
__FUNCTION__, saddr.sipc_port);
for (;;) {
len = strlen(msg[n]);
/*
* Send a datagram to the NRT endpoint via the proxy.
* We may pass a NULL destination address, since the
* socket was successfully assigned the proper default
* address via connect(2).
*/
ret = sendto(s, msg[n], len, 0, NULL, 0);
if (ret != len)
fail("sendto");
printf("%s: sent %d bytes, \"%.*s\"\n",
__FUNCTION__, ret, ret, msg[n]);
n = (n + 1) % (sizeof(msg) / sizeof(msg[0]));
/*
* We run in full real-time mode (i.e. primary mode),
* so we have to let the system breathe between two
* iterations.
*/
ts.tv_sec = 0;
ts.tv_nsec = 500000000; /* 500 ms */
clock_nanosleep(CLOCK_REALTIME, 0, &ts, NULL);
}
return NULL;
}
static void *regular_thread(void *arg)
{
char buf[128], *devname;
int fd, ret;
if (asprintf(&devname,
"/proc/xenomai/registry/rtipc/xddp/%s",
XDDP_PORT_LABEL) < 0)
fail("asprintf");
fd = open(devname, O_RDWR);
free(devname);
if (fd < 0)
fail("open");
for (;;) {
/* Get the next message from realtime_thread2. */
ret = read(fd, buf, sizeof(buf));
if (ret <= 0)
fail("read");
/* Relay the message to realtime_thread1. */
ret = write(fd, buf, ret);
if (ret <= 0)
fail("write");
}
return NULL;
}
int main(int argc, char **argv)
{
struct sched_param rtparam = { .sched_priority = 42 };
pthread_attr_t rtattr, regattr;
sigset_t set;
int sig;
sigemptyset(&set);
sigaddset(&set, SIGINT);
sigaddset(&set, SIGTERM);
sigaddset(&set, SIGHUP);
pthread_sigmask(SIG_BLOCK, &set, NULL);
pthread_attr_init(&rtattr);
pthread_attr_setdetachstate(&rtattr, PTHREAD_CREATE_JOINABLE);
pthread_attr_setinheritsched(&rtattr, PTHREAD_EXPLICIT_SCHED);
pthread_attr_setschedpolicy(&rtattr, SCHED_FIFO);
pthread_attr_setschedparam(&rtattr, &rtparam);
/* Both real-time threads have the same attribute set. */
errno = pthread_create(&rt1, &rtattr, &realtime_thread1, NULL);
if (errno)
fail("pthread_create");
errno = pthread_create(&rt2, &rtattr, &realtime_thread2, NULL);
if (errno)
fail("pthread_create");
pthread_attr_init(®attr);
pthread_attr_setdetachstate(®attr, PTHREAD_CREATE_JOINABLE);
pthread_attr_setinheritsched(®attr, PTHREAD_EXPLICIT_SCHED);
pthread_attr_setschedpolicy(®attr, SCHED_OTHER);
errno = pthread_create(&nrt, ®attr, ®ular_thread, NULL);
if (errno)
fail("pthread_create");
sigwait(&set, &sig);
pthread_cancel(rt1);
pthread_cancel(rt2);
pthread_cancel(nrt);
pthread_join(rt1, NULL);
pthread_join(rt2, NULL);
pthread_join(nrt, NULL);
return 0;
}
（2）CMakeLists.txt：
cmake_minimum_required(VERSION 3.7.2)
project(xddp)
#file(GLOB SOURCES "/usr/xenomai/include/alchemy/*.*")
# set(CMAKE_CXX_STANDARD 11)
#set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
add_compile_options(-std=c++11)
include_directories(/usr/xenomai/include/alchemy/)
include_directories(/usr/xenomai/include/)
include_directories(/usr/xenomai/include/rtdm)
include_directories(/usr/xenomai/cobalt/)
include_directories(/usr/xenomai/posix/)
include_directories(/usr/xenomai/boilerplate/)
link_directories(/usr/xenomai/lib)
set(xeno_cflags_params "--skin=posix" "--cflags")
execute_process(
COMMAND xeno-config ${xeno_cflags_params}
OUTPUT_VARIABLE xeno_cflags
OUTPUT_STRIP_TRAILING_WHITESPACE)
set(xeno_ldflags_params "--skin=posix" "--ldflags")
execute_process(
COMMAND xeno-config ${xeno_ldflags_params}
OUTPUT_VARIABLE xeno_ldflags
OUTPUT_STRIP_TRAILING_WHITESPACE)
# Compiler and linker options
set(CMAKE_C_FLAGS "${CMAKE_CXX_FLAGS} ${xeno_cflags}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${xeno_cflags}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${xeno_ldflags}")
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${xeno_ldflags}")
add_executable(xddp xddp.cpp )
target_link_libraries(xddp ${XENO_NATIVE_LDFLAGS} /usr/xenomai/bin)
#target_link_libraries(xddp ${LIB_DIR}/libalchemy.so ${LIB_DIR}/libanalogy.so
# ${LIB_DIR}/libcobalt.so ${LIB_DIR}/libmodechk.so ${LIB_DIR}/libcopperplate.so
# ${LIB_DIR}/libnative.so ${LIB_DIR}/libpsos.so ${LIB_DIR}/libpthread_rt.so
# ${LIB_DIR}/librtdm.so ${LIB_DIR}/libsmokey.so ${LIB_DIR}/libtrank.so
# ${LIB_DIR}/libuitron.so ${LIB_DIR}/libvrtx.so ${LIB_DIR}/libvxworks.so ${LIB_DIR}/libxenomai.so)
#