TriangleMesh Texture Coordinates are not interpolated as expected - javafx

Question

I have a TriangleMesh with a texture/diffuse map that is a 1024x1024 texture which is fully black except the last 3 lines which are filled with red, green and blue.
I gave each vertex of each triangle a constant V value (either 1021, 1022 or 1023) added 0.5 to it to center and divided it by the texture's height so it would only use one of the 3 colors and arbitrary U values.
// arbitrary u values.
float u1 = 0.1f;
float u2 = 0.3f;
float u3 = 0.9f;
int randomY = ThreadLocalRandom.current()
.nextInt((int) atlas.getHeight() - 3, (int) atlas.getHeight());
float y = randomY + 0.5f;
float v = (float) (y / atlas.getHeight());
int texIndex1 = mesh.addUV(u1, v);
int texIndex2 = mesh.addUV(u2, v);
int texIndex3 = mesh.addUV(u3, v);
mesh.getFaces().addAll(
vertexIndex1, texIndex1,
vertexIndex2, texIndex2,
vertexIndex3, texIndex3
);
The addUV method looks like this(i have my own mesh class that extends TriangleMesh which contains some helper methods)
public int addUV(float u, float v) {
int cur = getTexCoords().size() / 2;
getTexCoords().addAll(u, v);
return cur;
}
The expected result is a mesh that has it's triangles colored solid red, green and blue because V is constant and each line(y) is filled with a single color however what i got instead was a bunch of different colors that change as u zoom in/out.
If i use the same U value for each vertex as well, it does give the correct result but i don't understand why it wouldn't do the same with arbitrary U values given that the color at any given U is the exact same.
The current result(gif to show the color changing): https://i.imgur.com/4lTcLfH.gif | As seen it actually does show the correct colors but only if u zoom in a lot
The expected result(can be produced if i have constant U values as well like 0.5, 0.5, 0.5): https://i.imgur.com/x35u6xv.gif | Looks as it should, doesn't change when u zoom in/out
The texture i used as the diffuse map: https://i.imgur.com/BB6P7z6.png
Minimal reproducible example with a quad made of 2 triangles:
Create a main method (either in the same class or another) and add: Application.launch(TextureMappingIssue.class); i didn't add it in my example as depending on the setup, the main method must be in a different class
import javafx.application.Application;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.*;
import javafx.scene.image.Image;
import javafx.scene.image.PixelWriter;
import javafx.scene.image.WritableImage;
import javafx.scene.layout.AnchorPane;
import javafx.scene.paint.Color;
import javafx.scene.paint.PhongMaterial;
import javafx.scene.shape.MeshView;
import javafx.scene.shape.TriangleMesh;
import javafx.scene.transform.Rotate;
import javafx.scene.transform.Scale;
import javafx.scene.transform.Translate;
import javafx.stage.Stage;
import javax.imageio.ImageIO;
import java.nio.file.Path;
/**
* Note that when the application is first opened without
* the camera moved, it looks as it should, as soon as
* the camera is moved (i.e if the mouse is moved in this case)
* it looks completely different, even if the camera is moved slightly
*/
public class TextureMappingIssue extends Application {
private static final int WIDTH = 800;
private static final int HEIGHT = 600;
private AnchorPane modelPane;
private Group scene;
private SubScene subScene;
#Override
public void start(Stage primaryStage) throws Exception {
modelPane = new AnchorPane();
modelPane.setPrefWidth(WIDTH);
modelPane.setPrefHeight(HEIGHT);
initScene();
// smaller palette = looks correct until u zoom out more
int paletteWidth = 1024;
int paletteHeight = 1024;
/*
* amount of copies for red, green, blue(the colors at the bottom), the center one is picked
* note that copies = 1 just writes the original color, copies = 2 writes the original + 1 copy and so on (so with copies = 3, it writes the color 3 times and picks the 2nd one for v)
*/
int copies = 1;
float QUAD_SCALE = 1f;
float[] vertices = {
-QUAD_SCALE, -QUAD_SCALE, 0,
-QUAD_SCALE, QUAD_SCALE, 0,
QUAD_SCALE, QUAD_SCALE, 0,
QUAD_SCALE, -QUAD_SCALE, 0
};
int[] indices = {
0, 0, 1, 1, 2, 2, // first triangle
0, 3, 2, 4, 3, 5, // second triangle
};
// set these to 0f, 0f, 0f (or any value as long as they're identical to get the expected result)
float u1 = 0.1f;
float u2 = 0.3f;
float u3 = 0.5f;
int colorIndex = 1; // either 0, 1 or 2 (red, green, blue)
int offset = (3 - colorIndex) * copies;
// v is constant for each vertex in my actual application as well.
float v1 = (paletteHeight - offset + (copies / 2) + 0.5f) / paletteHeight;
float[] texCoords = {
u1, v1, u2, v1, u3, v1
};
Image palette = generatePalette(paletteWidth, paletteHeight, copies);
ImageIO.write(SwingFXUtils.fromFXImage(palette, null), "png", Path.of("./testpalette.png")
.toFile());
TriangleMesh triangle = new TriangleMesh();
triangle.getPoints().addAll(vertices);
triangle.getFaces().addAll(indices);
triangle.getTexCoords().addAll(texCoords);
triangle.getTexCoords().addAll(texCoords);
MeshView view = new MeshView(triangle);
PhongMaterial material = new PhongMaterial();
material.setDiffuseMap(palette);
//material.setSpecularMap(specular);
// material.setSpecularPower(32); // default
view.setMaterial(material);
scene.getChildren().add(view);
Scene scene = new Scene(modelPane, WIDTH, HEIGHT, true, SceneAntialiasing.BALANCED);
scene.setFill(Color.BLACK);
primaryStage.setScene(scene);
primaryStage.show();
}
private void initScene() {
scene = new Group();
//Group grid = new Grid3D().create(48f, 1.25f);
//scene.getChildren().add(grid);
subScene = createScene3D();
scene.getChildren().add(new AmbientLight(Color.WHITE));
modelPane.getChildren().addAll(subScene);
}
private SubScene createScene3D() {
SubScene scene3d = new SubScene(scene, modelPane.getPrefWidth(), modelPane.getPrefHeight(), true, SceneAntialiasing.BALANCED);
scene3d.setFill(Color.rgb(25, 25, 25));
new OrbitCamera(scene3d, scene);
return scene3d;
}
private Image generatePalette(int width, int height, int copies) {
WritableImage palette = new WritableImage(width, height);
Color[] debugColors = {Color.RED, Color.GREEN, Color.BLUE};
PixelWriter writer = palette.getPixelWriter();
int offset = height - (debugColors.length * copies);
for (int y = 0; y < offset; y++) {
for (int x = 0; x < width; x++) {
writer.setColor(x, y, Color.BLACK);
}
}
int colorOff = 0;
for (int y = offset; y < height - (copies - 1); y += copies) {
Color c = debugColors[colorOff];
if (c == Color.GREEN) {
System.out.println("Y = " + y);
}
for (int k = 0; k < copies; k++) {
for (int x = 0; x < width; x++) {
writer.setColor(x, y + k, c);
}
}
colorOff++;
}
return palette;
}
private Image generateSpecular(int width, int height) {
WritableImage specular = new WritableImage(width, height);
PixelWriter writer = specular.getPixelWriter();
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
writer.setColor(x, y, Color.WHITE);
}
}
return specular;
}
/*
* Orbit camera
*/
private static class OrbitCamera {
private final SubScene subScene;
private final Group root3D;
private final double MAX_ZOOM = 300.0;
public OrbitCamera(SubScene subScene, Group root) {
this.subScene = subScene;
this.root3D = root;
init();
}
private void init() {
camera.setNearClip(0.1D);
camera.setFarClip(MAX_ZOOM * 1.15D);
camera.getTransforms().addAll(
yUpRotate,
cameraPosition,
cameraLookXRotate,
cameraLookZRotate
);
Group rotateGroup = new Group();
try {
rotateGroup.getChildren().addAll(cameraXform);
} catch (Exception e) {
e.printStackTrace();
}
cameraXform.ry.setAngle(0);
cameraXform.rx.setAngle(-18);
cameraXform.getChildren().add(cameraXform2);
cameraXform2.getChildren().add(cameraXform3);
cameraXform3.getChildren().add(camera);
cameraPosition.setZ(-cameraDistance);
root3D.getChildren().addAll(rotateGroup);
subScene.setCamera(camera);
subScene.setOnScroll(event -> {
double zoomFactor = 1.05;
double deltaY = event.getDeltaY();
if (deltaY < 0) {
zoomFactor = 2.0 - zoomFactor;
}
double z = cameraPosition.getZ() / zoomFactor;
z = Math.max(z, -MAX_ZOOM);
z = Math.min(z, 10.0);
cameraPosition.setZ(z);
});
subScene.setOnMousePressed(event -> {
if (!event.isAltDown()) {
dragStartX = event.getSceneX();
dragStartY = event.getSceneY();
dragStartRotateX = cameraXRotate.getAngle();
dragStartRotateY = cameraYRotate.getAngle();
mousePosX = event.getSceneX();
mousePosY = event.getSceneY();
mouseOldX = event.getSceneX();
mouseOldY = event.getSceneY();
}
});
subScene.setOnMouseDragged(event -> {
if (!event.isAltDown()) {
double modifier = 1.0;
double modifierFactor = 0.3;
if (event.isControlDown()) modifier = 0.1;
if (event.isSecondaryButtonDown()) modifier = 0.035;
mouseOldX = mousePosX;
mouseOldY = mousePosY;
mousePosX = event.getSceneX();
mousePosY = event.getSceneY();
mouseDeltaX = mousePosX - mouseOldX;
mouseDeltaY = mousePosY - mouseOldY;
double flip = -1.0;
if (event.isSecondaryButtonDown()) {
double newX = cameraXform2.t.getX() + flip * mouseDeltaX * modifierFactor * modifier * 2.0;
double newY = cameraXform2.t.getY() + 1.0 * -mouseDeltaY * modifierFactor * modifier * 2.0;
cameraXform2.t.setX(newX);
cameraXform2.t.setY(newY);
} else if (event.isPrimaryButtonDown()) {
double yAngle = cameraXform.ry.getAngle() - 1.0 * -mouseDeltaX * modifierFactor * modifier * 2.0;
double xAngle = cameraXform.rx.getAngle() + flip * mouseDeltaY * modifierFactor * modifier * 2.0;
cameraXform.ry.setAngle(yAngle);
cameraXform.rx.setAngle(xAngle);
}
}
});
}
private final PerspectiveCamera camera = new PerspectiveCamera(true);
private final Rotate cameraXRotate = new Rotate(-20.0, 0.0, 0.0, 0.0, Rotate.X_AXIS);
private final Rotate cameraYRotate = new Rotate(-20.0, 0.0, 0.0, 0.0, Rotate.Y_AXIS);
private final Rotate cameraLookXRotate = new Rotate(0.0, 0.0, 0.0, 0.0, Rotate.X_AXIS);
private final Rotate cameraLookZRotate = new Rotate(0.0, 0.0, 0.0, 0.0, Rotate.Z_AXIS);
private final Translate cameraPosition = new Translate(0.0, 0.0, 0.0);
private Xform cameraXform = new Xform();
private Xform cameraXform2 = new Xform();
private Xform cameraXform3 = new Xform();
private double cameraDistance = 25.0;
private double dragStartX = 0;
private double dragStartY = 0;
private double dragStartRotateX = 0;
private double dragStartRotateY = 0;
private double mousePosX = 0;
private double mousePosY = 0;
private double mouseOldX = 0;
private double mouseOldY = 0;
private double mouseDeltaX = 0;
private double mouseDeltaY = 0;
private Rotate yUpRotate = new Rotate(0.0, 0.0, 0.0, 0.0, Rotate.X_AXIS);
public Camera getCamera() {
return camera;
}
public Xform getCameraXform() {
return cameraXform;
}
}
private static class Xform extends Group {
Translate t = new Translate();
Translate p = new Translate();
public Rotate rx = new Rotate();
public Rotate ry = new Rotate();
Rotate rz = new Rotate();
Scale s = new Scale();
public Xform() {
rx.setAxis(Rotate.X_AXIS);
ry.setAxis(Rotate.Y_AXIS);
rz.setAxis(Rotate.Z_AXIS);
getTransforms().addAll(t, rz, ry, rx, s);
}
}
}
Edit: updated the code to support generating multiple copies of a single color and picking the center one, however this doesn't solve the issue either, it's just less visible :/
Update: the issue can be reproduced even with a 128x3 image (where it's just the red, green, blue color with 128 pixel rows)
Update 2: I can reproduce the same issue in my original code even with my original palette (that is a 128x512 image of colors that are all potentially used)
Update 3: I have decided to go for per pixel shading instead (i.e provide my mesh with a set of normals and add light sources to the scene(other than ambient)) what i wanted to do initially with the palette was to export all the vertex colors generated from a function that emulates gouraud shading but because of these interpolation issues i have went for per pixel shading (which looks better anyway, altho ideally i would've wanted to emulate gouraud shading as the game engine i use for which the javafx program is for also uses gouraud shading)