[Unreal Engine UE] UE4/UE5 obtains 3D coordinates of the scene based on 2D screen coordinates and ray detection (Blueprint/C++)

---

1. Radiographic detection

1) Definition

Ray Casting is a basic technology in computer graphics and computer graphics that is used to detect whether a light or ray intersects an object in a three-dimensional scene to determine the location of the intersection point and other related information. Ray detection is usually used to implement various interactive functions, rendering effects and physical simulations, including but not limited to mouse picking, ray tracing, collision detection and object picking, etc.

1) The process of interaction between rays and objects in the 3D scene

step	describe
1	Define the ray: Define the starting point and direction vector of the ray.
2	Detect intersection: Follow the direction of the ray, starting from the starting point and detecting whether the ray intersects with any object in the scene. Typically, this involves doing collision detection to determine if an object intersects the ray.
3	Determine the intersection point: If the ray intersects the object, calculate the location of the intersection point. The intersection point is usually given in the form of 3D coordinates and represents the point where the ray intersects the object.
4	Handling interactions: Depending on the needs of your application, you can perform specific operations at interaction points, such as selecting objects, performing actions, or rendering effects.
5	Traverse all possible intersection points: Ray detection can often return multiple intersection points, so consider traversing all possible intersection points to handle multiple intersections.

2) Ray detection blueprint function

blueprint function	describe
LineTraceByChannel	Performs a ray detection to detect objects that intersect the specified collision channel. Returns a Hit Result structure.
SphereTraceByChannel	Perform ray detection in the shape of a sphere to detect collisions between the sphere and objects. Returns a Hit Result structure.
LineTraceMultiByChannel	performs ray detection and detects all objects that intersect the specified collision channel. Returns an Hit Results array.
SphereTraceMultiByChannel	Perform ray detection in the shape of a sphere and detect collisions between the sphere and multiple objects. Returns an Hit Results array.
BoxTraceByChannel	performs ray detection, detecting objects that intersect the specified collision channel, using a box shape. Returns a Hit Result structure.
MultiSphereTraceByChannel	Perform ray detection of multiple sphere shapes and detect collisions between multiple spheres and objects. Returns an Hit Results array.
LineTraceForObjects	Performs ray detection to detect objects that intersect with the specified object type. Returns a Hit Result structure.
SphereTraceForObjects	Performs ray detection in the shape of a sphere to detect objects that intersect with the specified object type. Returns a Hit Result structure.
BoxTraceForObjects	performs ray detection, detecting objects that intersect the specified object type, using the box shape. Returns a Hit Result structure.
MultiSphereTraceForObjects	Performs multiple sphere-shaped ray detection to detect objects that intersect with the specified object type. Returns an Hit Results array.
CapsuleTraceByChannel	Perform ray detection in the shape of the capsule to detect collisions between the capsule and the object. Returns a Hit Result structure.
CapsuleTraceForObjects	Performs ray detection in the shape of a capsule to detect objects that intersect with the specified object type. Returns a Hit Result structure.

3) The blueprint implements obtaining the coordinate values ​​in the scene based on the mouse click position.

4) Obtain the coordinate value in the scene based on the camera center point

Need to get the camera in pawn.

5) C++ functions related to ray detection

(Only list the linetrace series, others are similar)

1. LineTraceSingleByChannel：
   • Used to detect the collision of a ray with the first intersecting object.
   • Returns aFHitResult structure that contains information about the collision, such as the collision point, collision normal, and a reference to the colliding object.

bool UWorld::LineTraceSingleByChannel(FHitResult& OutHit, const FVector Start, const FVector End, ECollisionChannel TraceChannel, const FCollisionQueryParams& Params)

2. LineTraceMultiByChannel：
   • Used to detect collisions of a ray with multiple intersecting objects.
   • Returns aTArray<FHitResult> containing collision information for all intersecting objects.

int32 UWorld::LineTraceMultiByChannel(TArray<FHitResult>& OutHits, const FVector Start, const FVector End, ECollisionChannel TraceChannel, const FCollisionQueryParams& Params)

3. LineTraceSingleByObjectType：
   • Similar toLineTraceSingleByChannel, but uses the object type (EObjectTypeQuery) instead of the collision channel for detection.

bool UWorld::LineTraceSingleByObjectType(FHitResult& OutHit, const FVector Start, const FVector End, FObjectQueryParams ObjectQueryParams, const FCollisionQueryParams& Params)

4. LineTraceMultiByObjectType：
   • Similar toLineTraceMultiByChannel, but uses the object type (EObjectTypeQuery) instead of the collision channel for detection.

int32 UWorld::LineTraceMultiByObjectType(TArray<FHitResult>& OutHits, const FVector Start, const FVector End, FObjectQueryParams ObjectQueryParams, const FCollisionQueryParams& Params)

6) C++ implements manual creation of ray detection

FVector StartLocation;  // 射线的起点坐标
FVector ForwardVector;  // 射线的方向向量
FHitResult HitResult;  // 用于存储碰撞信息的变量

// 设置射线的起点坐标
StartLocation = PlayerCameraComponent->GetComponentLocation();  // PlayerCameraComponent是摄像机组件

// 设置射线的方向向量
ForwardVector = PlayerCameraComponent->GetForwardVector();  // 获取摄像机的前向向量

// 建立射线
FVector EndLocation = ((ForwardVector * RayLength) + StartLocation);  // 计算射线的终点坐标

// 进行射线检测
if (GetWorld()->LineTraceSingleByChannel(HitResult, StartLocation, EndLocation, ECC_Visibility))
{
    
    
    // 射线与物体相交，可以在HitResult中获取碰撞信息
    AActor* HitActor = HitResult.GetActor();
    FVector ImpactPoint = HitResult.ImpactPoint;
    // 进一步处理交互逻辑
}

PlayerCameraComponent: camera component
LineTraceSingleByChannel: ray detection function
HitResult: colliding object and collision point
RayLength : The length of the ray;
ECC_Visibility: The collision channel used for ray detection

7) C++ implements clicking to obtain coordinate values ​​in the scene

void AYourPlayerController::GetSceneLocationFromMouse()
{
    
    
    // 获取玩家控制器
    APlayerController* PlayerController = this;

    if (PlayerController)
    {
    
    
        // 获取鼠标点击位置
        FVector MouseLocation, MouseDirection;
        PlayerController->DeprojectMousePositionToWorld(MouseLocation, MouseDirection);

        // 创建射线，用于射线检测
        FHitResult HitResult;
        FCollisionQueryParams CollisionParams;

        // 执行射线检测
        if (GetWorld()->LineTraceSingleByChannel(HitResult, MouseLocation, MouseLocation + MouseDirection * YourRayLength, ECC_Visibility, CollisionParams))
        {
    
    
            // 获取射线与场景相交的位置
            FVector SceneLocation = HitResult.Location;

            // 打印结果
            UE_LOG(LogTemp, Warning, TEXT("Scene Location: %s"), *SceneLocation.ToString());
        }
    }
}

2. The situation of non-ray detection

1) Obtain the world coordinates of the center point based on the current position of the camera

void AYourPlayerController::GetCameraCenterLocation()
{
    
    
    // 获取玩家控制器的视图控制器
    APlayerController* PlayerController = this;

    if (PlayerController)
    {
    
    
        // 获取相机组件
        UCameraComponent* CameraComponent = PlayerController->PlayerCameraManager->GetCameraComponent();

        if (CameraComponent)
        {
    
    
            // 获取相机位置
            FVector CameraLocation = CameraComponent->GetComponentLocation();

            // 获取相机旋转
            FRotator CameraRotation = CameraComponent->GetComponentRotation();

            // 计算相机中心点的位置（通常位于相机位置的前方，视角方向）
            FVector CameraForwardVector = CameraRotation.Vector();
            FVector CameraCenterLocation = CameraLocation + CameraForwardVector * YourDistance;  // 替换 YourDistance 为相机中心点到相机位置的距离

            // 将相机中心点的位置转换为场景中的坐标
            FVector WorldLocation = CameraCenterLocation;

            // 打印结果
            UE_LOG(LogTemp, Warning, TEXT("Camera Center Location: %s"), *WorldLocation.ToString());
        }
    }
}