ROSIntegration ROSIntegrationVision与虚幻引擎4（Unreal ...

JamesB · 发表于 2022-11-19 14:34

ROSIntegration ROSIntegrationVision与虚幻引擎4（Unreal Engine 4）的配置

操作系统：Ubuntu 18.04
虚幻引擎：4.26.2
一、虚幻引擎源代码下载与编译运行

参照官方文档：虚幻引擎Linux快速入门
二、ROSIntegration下载与配置运行

1、配置ROSBridge

要启用虚幻和ROS之间的通信，需要一个正在运行的ROSBridge和bson_mode
注意：请使用 rosbridge 版本=>0.8.0 以获得完整的 BSON 支持
安装rosbridge的推荐方法是在ROS工作空间使用源代码进行编译，即把rosbridge作为其中一个功能包，按照如下命令顺序执行
sudo apt-get install ros-ROS1_DISTRO-rosauth # 将 ROS1_DISTRO 替换为ROS对应的版本名称
cd ~/ros_workspace/ # 替换 ros_workspace 为工作空间目录名称
source devel/setup.bash
cd src/
git clone -b ros1 https://github.com/RobotWebTools/rosbridge_suite.git
cd ..
catkin_make
source devel/setup.bash此外，ROSIntegration使用包含在PyMongo包中的BSON，可以单独安装
sudo pip3 install pymongo2、配置ROSIntegration

使用git命令下载ROSIntegration，放置在虚幻引擎项目文件Plugins文件夹下
cd unreal_engine_project # 替换 unreal_engine_project 为项目目录文件夹路径
mkdir Plugins # 如果没有 Plugins 文件夹则手动创建
cd Plugins
git clone https://github.com/code-iai/ROSIntegration.git此时，ROSIntegration在虚幻项目中的文件结构如下：
unreal_engine_project/Plugins/ROSIntegration/ROSIntegration.uplugin
在虚幻引擎源代码UnrealEngine下的Engine/Source/Developer/DesktopPlatform/Private/DesktopPlatformBase.cpp文件中执行此操作
查找此行：
Arguments += &#34; -Progress -NoEngineChanges -NoHotReloadFromIDE&#34;;替换为：
Arguments += &#34; -Progress&#34;;然后重新编译引擎：
cd UnrealEngine
./Setup.sh
./GenerateProjectFiles.sh
make编译完成后，启动项目并接受重建
（如果不进行上述步骤可能会遇到虚幻引擎自建项目打不开或遇到engine modules are out of date and cannot be compiled while the engine is running的情况）
创建一个新的C++虚幻项目，或打开现有项目

在内容浏览器中查找（在内容浏览器的右下角启用“查看选项”>“显示插件内容”）
点击“添加/导入”按钮下方的三条线按钮，展开左侧区域
选中“ROSIntegration“>“Classes”，右键ROSIntegrationGameInstance，点击下图黄色选项

打开新的C++类/蓝图对象，并更改ROSBridgeSeverHost 和ROSBridgeServerPort，如果是本地运行的ROSBridge，则改为127.0.0.1即可

打开“地图和模式”>“项目设置”，并将游戏实例设置为与新的游戏实例对象匹配，比如MyROSIntegrationGameInstance，而不是插件中的ROSIntegrationGameInstance

使用Ctrl + Shift + S保存所有更改
3、使用ROSIntegration

要进行与 ROS 的简单发布/订阅通信，需要在创建一个新的C++ Actor，而非中文的角色（Charactor）。接着创建 SamplePubliser
SamplePublisher.h

#include &#34;ROSIntegration/Classes/RI/Topic.h&#34;
#include &#34;ROSIntegration/Classes/ROSIntegrationGameInstance.h&#34;
#include &#34;ROSIntegration/Public/std_msgs/String.h&#34;
注意：上述代码必须在#include &#34;SamplePublisher.generated.h&#34;之前，否则会报错
SamplePublisher.cpp

// Initialize a topic
UTopic *ExampleTopic = NewObject<UTopic>(UTopic::StaticClass());
UROSIntegrationGameInstance* rosinst = Cast<UROSIntegrationGameInstance>(GetGameInstance());
ExampleTopic->Init(rosinst->ROSIntegrationCore, TEXT(&#34;/example_topic&#34;), TEXT(&#34;std_msgs/String&#34;));

// (Optional) Advertise the topic
ExampleTopic->Advertise();

// Publish a string to the topic
TSharedPtr<ROSMessages::std_msgs::String> StringMessage(new ROSMessages::std_msgs::String(&#34;This is an example&#34;));
ExampleTopic->Publish(StringMessage);
注意：上述代码放置在BeginPlay()函数中
进入unreal_engine_project/Source/unreal_engine_project目录（替换 unreal_engine_project 为真实的项目名称），打开unreal_engine_project.Build.cs文件
找到:
PublicDependencyModuleNames.AddRange(new string[] { &#34;Core&#34;, &#34;CoreUObject&#34;, &#34;Engine&#34;, &#34;InputCore&#34; });
添加ROSIntegrationy依赖，形如：
PublicDependencyModuleNames.AddRange(new string[] { &#34;Core&#34;, &#34;CoreUObject&#34;, &#34;Engine&#34;, &#34;InputCore&#34;, &#34;ROSIntegration&#34; });
进入ROS工作空间的src目录，创建测试功能包：
catkin_create_pkg ue_test std_msgs rospy roscpp编译并source：
catkin_make
source devel/setup.bash创建一个监听者cpp文件：
cd ue_test/src
touch listener.cpp打开cpp并键入如下代码：
#include &#34;ros/ros.h&#34;
#include &#34;std_msgs/String.h&#34;

/**
* This tutorial demonstrates simple receipt of messages over the ROS system.
*/
void chatterCallback(const std_msgs::String::ConstPtr& msg)
{
  ROS_INFO(&#34;I heard: [%s]&#34;, msg->data.c_str());
}

int main(int argc, char **argv)
{
  /**
* The ros::init() function needs to see argc and argv so that it can perform
* any ROS arguments and name remapping that were provided at the command line.
* For programmatic remappings you can use a different version of init() which takes
* remappings directly, but for most command-line programs, passing argc and argv is
* the easiest way to do it.  The third argument to init() is the name of the node.
*
* You must call one of the versions of ros::init() before using any other
* part of the ROS system.
*/
  ros::init(argc, argv, &#34;listener&#34;);

  /**
* NodeHandle is the main access point to communications with the ROS system.
* The first NodeHandle constructed will fully initialize this node, and the last
* NodeHandle destructed will close down the node.
*/
  ros::NodeHandle n;

  /**
* The subscribe() call is how you tell ROS that you want to receive messages
* on a given topic.  This invokes a call to the ROS
* master node, which keeps a registry of who is publishing and who
* is subscribing.  Messages are passed to a callback function, here
* called chatterCallback.  subscribe() returns a Subscriber object that you
* must hold on to until you want to unsubscribe.  When all copies of the Subscriber
* object go out of scope, this callback will automatically be unsubscribed from
* this topic.
*
* The second parameter to the subscribe() function is the size of the message
* queue.  If messages are arriving faster than they are being processed, this
* is the number of messages that will be buffered up before beginning to throw
* away the oldest ones.
*/
  ros::Subscriber sub = n.subscribe(&#34;/example_topic&#34;, 1000, chatterCallback);

  /**
* ros::spin() will enter a loop, pumping callbacks.  With this version, all
* callbacks will be called from within this thread (the main one).  ros::spin()
* will exit when Ctrl-C is pressed, or the node is shutdown by the master.
*/
  ros::spin();

  return 0;
}
在CMakeLists.txt添加：
add_executable(listener src/listener.cpp)
target_link_libraries(listener ${catkin_LIBRARIES})
add_dependencies(listener listener)4、测试ROSIntegration

启动rosbridge
roslaunch rosbridge_server rosbridge_tcp.launch bson_only_mode:=True运行新建功能包的监听者
# rosrun <your package> talker
# 比如
rosrun ue_test talker将在UE中新建的SamplePublisher托入三维世界中，并点击运行
此时可以看到：
[INFO] [1588662504.536355639]: I heard: [This is an example]恭喜你成功配置并运行了ROSIntegration！！！
三、ROSIntegrationVision下载与配置运行

1、虚幻引擎配置修改

在PATH_TO_UNREAL/Engine/Source/Programs/UnrealBuildTool/Platform/Linux/LinuxToolChain.cs中找到GetCLArguments_Global函数，并在其中添加语句Result += &#34; -mf16c&#34;;，形如：
protected virtual string GetCLArguments_Global(CppCompileEnvironment CompileEnvironment)
      {
         string Result = &#34;&#34;;

         // build up the commandline common to C and C++
         Result += &#34; -c&#34;;
         Result += &#34; -pipe&#34;;
         Result += &#34; -mf16c&#34;;

         if (ShouldUseLibcxx(CompileEnvironment.Architecture))
         {
            Result += &#34; -nostdinc++&#34;;
            Result += &#34; -I&#34; + &#34;ThirdParty/Linux/LibCxx/include/&#34;;
            Result += &#34; -I&#34; + &#34;ThirdParty/Linux/LibCxx/include/c++/v1&#34;;
         }
而后重新编译虚幻引擎（注意如下语句必须都执行才算是重新编译）：
cd UnrealEngine
./Setup.sh
./GenerateProjectFiles.sh
make2、配置ROSIntegrationVision

使用git命令下载ROSIntegrationVision，放置在虚幻引擎项目文件Plugins文件夹下
cd unreal_engine_project # 替换 unreal_engine_project 为项目目录文件夹路径
mkdir Plugins # 如果没有 Plugins 文件夹则手动创建
cd Plugins
git clone https://github.com/code-iai/ROSIntegrationVision/.git如果你是在Linux上编译虚幻引擎4而非Windows，则在打开项目时可能会遇到
Building forest2Editor...
Performing 3 actions (6 in parallel)
[1/3] Compile Module.ROSIntegrationVision.cpp
In file included from .../Plugins/ROSIntegrationVision/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/ROSIntegrationVision/Module.ROSIntegrationVision.cpp:6:
.../Plugins/ROSIntegrationVision/Source/ROSIntegrationVision/Private/VisionComponent.cpp:754:4: error: use of undeclared identifier &#39;_mm_div_epi16&#39;; did you mean &#39;_mm_min_epi16&#39;?
_mm_div_epi16(
^~~~~~~~~~~~~
_mm_min_epi16
/home/pisces/Gitware/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/lib/clang/10.0.1/include/emmintrin.h:2412:1: note: &#39;_mm_min_epi16&#39; declared here
_mm_min_epi16(__m128i __a, __m128i __b)
^
1 error generated.
LogInit: Warning: Still incompatible or missing module: ROSIntegrationVision这是由于官方参考了Windows for UVisionComponent上的编译问题：：convertDepth #28这一问题的回答，将源码改为了适配Windows的环境，但在Ubuntu18.04中并不存在_mm_div_epi16这一函数
因而，改变代码
void UVisionComponent::convertDepth(const uint16_t *in, __m128 *out) const
{
  const size_t size = (Width * Height) / 4;
  for (size_t i = 0; i < size; ++i, in += 4, ++out)
  {
// Divide by 100 here in order to convert UU (cm) into ROS units (m)
*out = _mm_cvtph_ps(
   _mm_div_epi16(
      _mm_set_epi16(0, 0, 0, 0, *(in + 3), *(in + 2), *(in + 1), *(in + 0)),
      _mm_set_epi16(100, 100, 100, 100, 100, 100, 100, 100)
   )
);// / 100;
  }
}
为
void UVisionComponent::convertDepth(const uint16_t *in, __m128 *out) const
{
  const size_t size = (Width * Height) / 4;
  for (size_t i = 0; i < size; ++i, in += 4, ++out)
  {
// Divide by 100 here in order to convert UU (cm) into ROS units (m)
*out = _mm_cvtph_ps(_mm_set_epi16(
   0, 0, 0, 0, *(in + 3), *(in + 2), *(in + 1), *(in + 0))) / 100;
  }
}
即可成功自动编译打开项目
3、使用ROSIntegrationVision

在内容浏览器ROSIntegrationVision/ROSIntegrationVision/Private中包含VisionActor C++文件，将其托入三维世界中即可现实摄像头图像信息
4、问题

注意：使用时需要先在VisionActor.cpp中作如下修改
AVisionActor::AVisionActor() : AActor()
{
UE_LOG(LogTemp, Warning, TEXT(&#34;VisionActor CTOR&#34;));

// Set this actor to call Tick() every frame.  You can turn this off to improve performance if you don&#39;t need it.
PrimaryActorTick.bCanEverTick = true;

RootComponent = CreateDefaultSubobject<USceneComponent>(TEXT(&#34;Root&#34;));
SetRootComponent(RootComponent);

vision = CreateDefaultSubobject<UVisionComponent>(TEXT(&#34;Vision&#34;));
vision->DisableTFPublishing = true; // 添加
//vision->ParentLink = &#34;/world&#34;; 注释掉
vision->ParentLink = &#34;desired_link&#34;;  // 添加
vision->SetupAttachment(RootComponent);
}
将ROSIntegrationVision插件中的Binaries和Intermediate文件夹删除，重新打开项目，使引擎重新编译插件
如果在运行rosbridge时遇到如下问题

可以修改ROSIntegration/Source/ROSIntegration/Private/Conversion/Messages/sensor_msgs/SensorMsgsCameraInfoConverter.h文件
替换
static void _bson_append_camera_info(bson_t *b, const ROSMessages::sensor_msgs::CameraInfo *msg)
{
      // assert(CastMsg->D.Num() == 5); // TODO: use Unreal assertions
      assert(CastMsg->K.Num() == 9); // TODO: use Unreal assertions
      assert(CastMsg->R.Num() == 9);
      assert(CastMsg->P.Num() == 12);

      UStdMsgsHeaderConverter::_bson_append_child_header(b, &#34;header&#34;, &msg->header);
      BSON_APPEND_INT32(b, &#34;height&#34;, msg->height);
      BSON_APPEND_INT32(b, &#34;width&#34;, msg->width);
      BSON_APPEND_UTF8(b, &#34;distortion_model&#34;, TCHAR_TO_UTF8(*msg->distortion_model));
      _bson_append_double_tarray(b, &#34;d&#34;, msg->D);
      _bson_append_double_tarray(b, &#34;k&#34;, msg->K);
      _bson_append_double_tarray(b, &#34;r&#34;, msg->R);
      _bson_append_double_tarray(b, &#34;p&#34;, msg->P);
      BSON_APPEND_INT32(b, &#34;binning_x&#34;, msg->binning_x);
      BSON_APPEND_INT32(b, &#34;binning_y&#34;, msg->binning_y);
      USensorMsgsRegionOfInterestConverter::_bson_append_child_roi(b, &#34;roi&#34;, &msg->roi);
}
为
static void _bson_append_camera_info(bson_t *b, const ROSMessages::sensor_msgs::CameraInfo *msg)
{
      // assert(CastMsg->D.Num() == 5); // TODO: use Unreal assertions
      assert(CastMsg->K.Num() == 9); // TODO: use Unreal assertions
      assert(CastMsg->R.Num() == 9);
      assert(CastMsg->P.Num() == 12);

      UStdMsgsHeaderConverter::_bson_append_child_header(b, &#34;header&#34;, &msg->header);
      BSON_APPEND_INT32(b, &#34;height&#34;, msg->height);
      BSON_APPEND_INT32(b, &#34;width&#34;, msg->width);
      BSON_APPEND_UTF8(b, &#34;distortion_model&#34;, TCHAR_TO_UTF8(*msg->distortion_model));
      _bson_append_double_tarray(b, &#34;D&#34;, msg->D); // 替换
      _bson_append_double_tarray(b, &#34;K&#34;, msg->K); // 替换
      _bson_append_double_tarray(b, &#34;R&#34;, msg->R); // 替换
      _bson_append_double_tarray(b, &#34;P&#34;, msg->P); // 替换
      BSON_APPEND_INT32(b, &#34;binning_x&#34;, msg->binning_x);
      BSON_APPEND_INT32(b, &#34;binning_y&#34;, msg->binning_y);
      USensorMsgsRegionOfInterestConverter::_bson_append_child_roi(b, &#34;roi&#34;, &msg->roi);
}
如果相机图象FPS较低，可以考虑修改VisionComponent.cpp中 Framerate(1) 为 Framerate(100)
UVisionComponent::UVisionComponent() :
Width(640),
Height(480),
Framerate(100), // change 1 to 100
UseEngineFramerate(false),
ServerPort(10000),
FrameTime(1.0f / Framerate),
TimePassed(0),
ColorsUsed(0)
四、参考资料

[1] code-iai/ROSIntegration markdown说明文档及issues
[2] code-iai/ROSIntegrationVision markdown说明文档及issues
[3] ROS Communication Sample on Unreal Engine Using ROSIntegration
[4] upgrade c++ project from ue 4.24 to 4.25 under Linux
部分图片来源于网络
<hr/>转载请注明出处！
本篇发布在以下博客或网站：
双鱼座羊驼 - 知乎 (zhihu.com)
双鱼座羊驼的博客_CSDN博客
双鱼座羊驼 - SegmentFault 思否
双鱼座羊驼的个人主页 - 动态 - 掘金 (juejin.cn)
双鱼座羊驼 - 博客园 (cnblogs.com)

		自动登录	找回密码
密码			立即注册

ROSIntegration ROSIntegrationVision与虚幻引擎4（Unreal ...

本帖子中包含更多资源