Convert coordinates from 3D scene to 2D overlay

Mark :

Similar to How to get 2D coordinates on window for 3D object in javafx but I couldn't get the solution to work.

I want to draw a 2d border for a 3d shape or more like its projection. I wrote this example code for reference:

import javafx.application.Application;
import javafx.geometry.Point3D;
import javafx.scene.Group;
import javafx.scene.PerspectiveCamera;
import javafx.scene.Scene;
import javafx.scene.SceneAntialiasing;
import javafx.scene.SubScene;
import javafx.scene.layout.AnchorPane;
import javafx.scene.layout.Background;
import javafx.scene.layout.BackgroundFill;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.Pane;
import javafx.scene.layout.StackPane;
import javafx.scene.paint.Color;
import javafx.scene.paint.PhongMaterial;
import javafx.scene.shape.Box;
import javafx.scene.shape.Rectangle;
import javafx.scene.transform.Rotate;
import javafx.stage.Stage;

public class CoordinateConversion extends Application {

    @Override
    public void start(Stage stage) {
        var box = new Box(20, 20, 20);
        box.setMaterial(new PhongMaterial(Color.TEAL));
        box.getTransforms().add(new Rotate(45, new Point3D(1, 1, 1)));
        var rootGroup = new Group(box);

        var camera = new PerspectiveCamera(true);
        camera.setFarClip(500);
        camera.setTranslateZ(-100);
        var aaScene = new SubScene(rootGroup, 0, 0, true, SceneAntialiasing.BALANCED);
        aaScene.setCamera(camera);

        var pane3d = new Pane(aaScene);
        pane3d.setBackground(new Background(new BackgroundFill(Color.BEIGE, null, null)));
        aaScene.widthProperty().bind(pane3d.widthProperty());
        aaScene.heightProperty().bind(pane3d.heightProperty());

        var overlay = new AnchorPane();

        var stack = new StackPane(pane3d, overlay); 
        var filler = new Rectangle(0, 0, 100, 50);
        filler.setFill(Color.rgb(0, 255, 0, 0.3));
        var borderPane = new BorderPane();
        borderPane.setCenter(stack);
        borderPane.setTop(filler);
        var scene = new Scene(borderPane);
        stage.setScene(scene);
        stage.setWidth(800);
        stage.setHeight(400);
        stage.show();

        var sceneBounds = box.localToScene(box.getBoundsInLocal(), false);
        var overlayBounds = overlay.sceneToLocal(sceneBounds);
        var rect = new Rectangle(overlayBounds.getMinX(), overlayBounds.getMinY(), overlayBounds.getWidth(), overlayBounds.getHeight());
        rect.setFill(null);
        rect.setStroke(Color.RED);
        overlay.getChildren().add(rect);
    }

    public static void main(String[] args) {
        launch(args);
    }
}

It draws the red rectangle in the corner while what I want is the "photoshoped" orange one that I added.

The basic idea is to convert bounds from one parent to another, so I convert from one parent to the scene and then from the scene to the next parent as I was taught in How to translate a node in a parent's coordinate system?. Some things i don't understand about the transformation

1. If I use box.localToScene(box.getBoundsInLocal(), true) I get NaN bounds even though the box is in a subscene and I want the coordinates in the scene.
2. The sceneBounds and overlayBounds are the same even though the overlay starts 50 pixels lower than the scene (because of the filler). i would expect every conversion between the overlay and the scene to be exactly +-50 on y if there are no transforms on them.

Also I tried to get the sceneBounds with box.getParent().localToScene(box.getBoundsInParent()) but ti's the same. I guess it makes sense because I use the parent to convert the bounds in parent instead of the node to convert the bounds in local.

Using Javafx 12

José Pereda :

You have a "time" issue: you set your subScene with 0x0 dimensions, then you add the bindings, and do the calculations for sceneBounds right after showing the stage, which return NaN.

If you add a listener to pane3d.widthProperty(), you will see that the sceneBounds is resolved before the width changes, therefore, you are working with a 0x0 subScene. So you are calling it too early.

Possible solutions:

• Remove the bindings and set a value for the subScene dimensions:

var aaScene = new SubScene(rootGroup, 800, 400, true, SceneAntialiasing.BALANCED);
stage.show();
var sceneBounds = box.localToScene(box.getBoundsInLocal(), true);
...

• Keep the bindings, but add a listener to the subScene's height property (which is set after the width):

aaScene.heightProperty().addListener((obs, ov, nv) -> {
     var sceneBounds = box.localToScene(box.getBoundsInLocal(), true);     
...
});

In either case, note that you have to use localToScene(..., true) to get scene coordinates:

If the Node does not have any SubScene or rootScene is set to true, the result point is in Scene coordinates of the Node returned by getScene().