Interpretation of the paper In-Depth Mouse: Integrating Desktop Mouse into Virtual Reality

In-Depth Mouse: Integrating Desktop Mouse into Virtual Reality

HCI2022 honorable ❤️Apply
the 2D mouse to the 3D virtual space to select the optional objects

Challenge

1. If you simply use the 3D position of the 3D virtual mouse to select optional objects, there may be objects that are closer to cover the virtual mouse. 2. The visibility
and overlapping relationship between objects vary according to the user's position, so it is necessary to Apply mouse control differently according to the viewpoint
3. The sensitivity of the mouse

Method

The two degrees of freedom of the mouse are mapped to the horizontal and vertical controls in VR, and the depth direction is continuously estimated according to the depth of nearby objects in the vision, thereby changing the shadow in the three-dimensional space of the mouse (to reflect the sense of depth); the head Motion will cause changes in the visibility and overlapping relationship of different objects, and this scheme can estimate in real time based on nearby objects in vision, thus solving problems such as occlusion.

Depth is determined using Voronoi-based Laplace interpolation to determine the depth dimension from cursor position, viewpoint, and selectable objects. Laplace interpolation provides linear continuity in the depth dimension.

If the ray of the mouse and head Xview intersects with the object (hits the object), the depth is the depth of the hit object.
If the ray doesn't hit the object, the weighted average (Laplace interpolation) of its N natural neighbors in the Voronoi diagram is used as the depth value of the mouse. is the proximity measure λi is the ratio between the length σi of the shared edge and the Euclidean distance di between the mouse P0 and its neighbor Pi.

Three-dimensional position: R1 (θ , φ)

Voronoi diagrams are created based on projected coordinates in 2D, projecting the mouse and selectable objects into the camera view.

resolution

The authors converted the parameters to angular coordinates so that the mouse's CD gain is described in degrees per inch (DPI) rather than the standard dots per inch (dpi). Moreover, the visual sensitivity of users in different devices and scenarios is different, so the angular resolution is also different, as shown in the figure below.

Angular resolution calculation:

Incremental motion (dxpx , dypx ) is detected in pixels, which is converted to angular motion (dθ, dφ) in spherical coordinates, Resmouse is the resolution of the mouse in dots per inch. CDgain is the mouse control display gain expressed in degrees per inch (DPI). CDgain is the tuning parameter between min-max.

Limit

The object adopts the position of the center, so for a large object, it must be close to the center of the object to be selected.