Hot questions for Using Lightweight Java Game Library in matrix multiplication

Question:

Follow-up for: Calculating world coordinates from camera coordinates

I'm multiplying a 2D vector with a transformation matrix (OpenGL's model-view matrix) to get world coordinates from my camera coordinates.

I do this calculation like this:

private Vector2f toWorldCoordinates(Vector2f position) {

    glPushMatrix();
    glScalef(this.zoom, this.zoom, 1);
    glTranslatef(this.position.x, this.position.y, 0);
    glRotatef(ROTATION, 0, 0, 1);

    ByteBuffer m = ByteBuffer.allocateDirect(64);
    m.order(ByteOrder.nativeOrder());
    glGetFloatv(GL_MODELVIEW_MATRIX, m);

    float x = (position.x * m.getFloat(0)) + (position.y * m.getFloat(4)) + m.getFloat(12);
    float y = (position.x * m.getFloat(16)) + (position.y * m.getFloat(20)) + m.getFloat(28);

    glPopMatrix();
    return new Vector2f(x, y);
}

Now I also want to do this vice-versa: calculate the camera coordinates for a position in the world. How can I reverse this calculation?


Answer:

To create a matrix representing the inverse transform to the one above, apply the transforms in reverse, with negative quantities for the rotation and translation and an inverse quantity for the zoom:

glRotatef(-ROTATION, 0, 0, 1);
glTranslatef(-this.position.x, -this.position.y, 0);
glScalef(1.0f / this.zoom, 1.0f / this.zoom, 1);

Then multiply by the position vector as before.

The alternative is to compute the inverse matrix, but this way is much simpler.

Question:

I've tried implementing a 3rd person camera multiply times, but i just can't seem to get it to work. The movement and the zoom work fine, but the rotation just causes me a lot of troubles:

  1. When changing the pitch of the camera (i.e. from how high you are looking at the player), the camera always rotates around the world x axis at the position of the player and not the camera x axis

  2. When changing the angle around the player, the camera always rotates around the world z axis at the position of the player and not the camera z axis

  3. When rotating the camera in any direction, it slowly moves in said direction, not staying at the fixed offset from the player position

I've already tried changing the matrix multiplication order, but that doesn't help and I've also tried using a look at matrix, but that wouldn't work at all, creating all sorts of other issues. I know, that using a math library would make this a lot easier, but that's not my goal. I want to at least try to understand the math behind this and understand my mistake.

My Matrix4f class:

import java.nio.FloatBuffer;
import java.util.Arrays;
import java.util.Iterator;

import com.engine.toolbox.ArrayUtils;
import com.engine.toolbox.BufferUtils;
import com.engine.toolbox.math.vectors.Vector3f;
import com.engine.toolbox.math.vectors.Vector4f;

public class Matrix4f extends Matrix{

    public float m11, m12, m13, m14, m21, m22, m23, m24, m31, m32, m33, m34, m41, m42, m43, m44;

    public Matrix4f(float m11, float m12, float m13, float m14,
                    float m21, float m22, float m23, float m24,
                    float m31, float m32, float m33, float m34,
                    float m41, float m42, float m43, float m44) {
        super(new Float[] { m11, m12, m13, m14, 
                            m21, m22, m23, m24, 
                            m31, m32, m33, m34, 
                            m41, m42, m43, m44});
        this.m11 = m11;
        this.m12 = m12;
        this.m13 = m13;
        this.m14 = m14;
        this.m21 = m21;
        this.m22 = m22;
        this.m23 = m23;
        this.m24 = m24;
        this.m31 = m31;
        this.m32 = m32;
        this.m33 = m33;
        this.m34 = m34;
        this.m41 = m41;
        this.m42 = m42;
        this.m43 = m43;
        this.m44 = m44;

    }

    @Override
    public Iterator<Float> iterator() {
        return Arrays.asList(this.toRawTypes()).iterator();
    }

    public Matrix4f invert() {
        Matrix4f matrixOfMinors =   new Matrix4f(   new Matrix3f(   m22, m23, m24,
                                                                    m32, m33, m34,
                                                                    m42, m43, m44).getDeterminant(), new Matrix3f(  m21, m23, m24,
                                                                                                                    m31, m33, m34,
                                                                                                                    m41, m43, m44).getDeterminant(), new Matrix3f(  m21, m22, m24,
                                                                                                                                                                    m31, m32, m34,
                                                                                                                                                                    m41, m42, m44).getDeterminant(), new Matrix3f(  m21, m22, m23,
                                                                                                                                                                                                                    m31, m32, m33,
                                                                                                                                                                                                                    m41, m42, m43).getDeterminant(),
                                                    new Matrix3f(   m12, m13, m14,
                                                                    m32, m33, m34,
                                                                    m42, m43, m44).getDeterminant(), new Matrix3f(  m11, m13, m14,
                                                                                                                    m31, m33, m34,
                                                                                                                    m41, m43, m44).getDeterminant(), new Matrix3f(  m11, m12, m14,
                                                                                                                                                                    m31, m32, m34,
                                                                                                                                                                    m41, m42, m44).getDeterminant(), new Matrix3f(  m11, m12, m23,
                                                                                                                                                                                                                    m31, m32, m33,
                                                                                                                                                                                                                    m41, m42, m43).getDeterminant(),
                                                    new Matrix3f(   m12, m13, m14,
                                                                    m22, m23, m24,
                                                                    m42, m43, m44).getDeterminant(), new Matrix3f(  m11, m13, m14,
                                                                                                                    m21, m23, m24,
                                                                                                                    m41, m43, m44).getDeterminant(), new Matrix3f(  m11, m12, m14,
                                                                                                                                                                    m21, m22, m24,
                                                                                                                                                                    m41, m42, m44).getDeterminant(), new Matrix3f(  m11, m12, m23,
                                                                                                                                                                                                                    m21, m22, m23,
                                                                                                                                                                                                                    m41, m42, m43).getDeterminant(),
                                                    new Matrix3f(   m12, m13, m14,
                                                                    m22, m23, m24,
                                                                    m32, m33, m34).getDeterminant(), new Matrix3f(  m11, m13, m14,
                                                                                                                    m21, m23, m24,
                                                                                                                    m31, m33, m34).getDeterminant(), new Matrix3f(  m11, m12, m14,
                                                                                                                                                                    m21, m22, m24,
                                                                                                                                                                    m31, m32, m34).getDeterminant(), new Matrix3f(  m11, m12, m23,
                                                                                                                                                                                                                    m21, m22, m23,
                                                                                                                                                                                                                    m31, m32, m33).getDeterminant());
        matrixOfMinors.m12 = -m12;
        matrixOfMinors.m14 = -m14;

        matrixOfMinors.m21 = -m21;
        matrixOfMinors.m23 = -m23;

        matrixOfMinors.m32 = -m32;
        matrixOfMinors.m34 = -m34;

        matrixOfMinors.m41 = -m41;
        matrixOfMinors.m43 = -m43;

        float temp = 0;

        temp = matrixOfMinors.m12;
        matrixOfMinors.m12 = matrixOfMinors.m21;
        matrixOfMinors.m21 = temp;

        temp = matrixOfMinors.m13;
        matrixOfMinors.m13= matrixOfMinors.m31;
        matrixOfMinors.m31 = temp;

        temp = matrixOfMinors.m23;
        matrixOfMinors.m23 = matrixOfMinors.m32;
        matrixOfMinors.m32 = temp;

        temp = matrixOfMinors.m14;
        matrixOfMinors.m14 = matrixOfMinors.m41;
        matrixOfMinors.m41 = temp;

        temp = matrixOfMinors.m24;
        matrixOfMinors.m24 = matrixOfMinors.m42;
        matrixOfMinors.m42 = temp;

        temp = matrixOfMinors.m34;
        matrixOfMinors.m34 = matrixOfMinors.m43;
        matrixOfMinors.m43 = temp;

        float d = this.getDeterminant();
        return new Matrix4f(matrixOfMinors.m11/d, matrixOfMinors.m12/d, matrixOfMinors.m13/d, matrixOfMinors.m14/d,
                            matrixOfMinors.m21/d, matrixOfMinors.m22/d, matrixOfMinors.m23/d, matrixOfMinors.m24/d,
                            matrixOfMinors.m31/d, matrixOfMinors.m32/d, matrixOfMinors.m33/d, matrixOfMinors.m34/d,
                            matrixOfMinors.m41/d, matrixOfMinors.m42/d, matrixOfMinors.m43/d, matrixOfMinors.m44/d);
    }

    public float getDeterminant() {
        return  m11*(new Matrix3f(  m22, m23, m24,
                                    m32, m33, m34,
                                    m42, m43, m44).getDeterminant()) - 
                m12*(new Matrix3f(  m21, m23, m24,
                                    m31, m33, m34,
                                    m41, m43, m44).getDeterminant()) + 
                m13*(new Matrix3f(  m21, m22, m24,
                                    m31, m32, m34,
                                    m41, m42, m44).getDeterminant()) - 
                m14*(new Matrix3f(  m21, m22, m23,
                                    m31, m32, m33,
                                    m41, m42, m43).getDeterminant());
    }

    public static Matrix4f identity() {
        return new Matrix4f(1, 0, 0, 0, 
                            0, 1, 0, 0, 
                            0, 0, 1, 0, 
                            0, 0, 0, 1);
    }

    public static Matrix4f perspective(float aspectRatio, float viewAngle, float nearPlane, float farPlane) {
        return new Matrix4f((float) (1/(aspectRatio*Math.tan((viewAngle/2)))),  0,                                      0,                                          0,
                            0,                                                  (float) (1/(Math.tan((viewAngle/2)))),  0,                                          0,
                            0,                                                  0,                                      -(farPlane+nearPlane)/(farPlane-nearPlane), (2*farPlane*nearPlane)/(farPlane-nearPlane),
                            0,                                                  0,                                      -1,                                         1);
    }

    public static Matrix4f translation(Vector3f translation) {
        Matrix4f result = Matrix4f.identity();
        result.m14 = translation.x;
        result.m24 = translation.y;
        result.m34 = translation.z;
        return result;
    }

    public static Matrix4f scale(Vector3f scale) {
        Matrix4f result = Matrix4f.identity();
        result.m11 = scale.x;
        result.m22 = scale.y;
        result.m33 = scale.z;
        return result;
    }

    public Matrix4f minus(Matrix4f m) {
        return new Matrix4f(m11-m.m11, m12-m.m12, m13-m.m13, m14-m.m14,
                            m21-m.m21, m22-m.m22, m13-m.m23, m24-m.m24,
                            m31-m.m31, m32-m.m32, m13-m.m33, m34-m.m34,
                            m41-m.m41, m42-m.m42, m13-m.m43, m44-m.m44);
    }

    public Matrix4f plus(Matrix4f m) {
        return new Matrix4f(m11+m.m11, m12+m.m12, m13+m.m13, m14+m.m14,
                            m21+m.m21, m22+m.m22, m13+m.m23, m24+m.m24,
                            m31+m.m31, m32+m.m32, m13+m.m33, m34+m.m34,
                            m41+m.m41, m42+m.m42, m13+m.m43, m44+m.m44);
    }

    public Matrix4f multiply(Matrix4f m) {
        Vector4f r1 = new Vector4f(m11, m12, m13, m14);
        Vector4f r2 = new Vector4f(m21, m22, m23, m24);
        Vector4f r3 = new Vector4f(m31, m32, m33, m34);
        Vector4f r4 = new Vector4f(m41, m42, m43, m44);

        Vector4f c1 = new Vector4f(m.m11, m.m21, m.m31, m.m41);
        Vector4f c2 = new Vector4f(m.m12, m.m22, m.m32, m.m42);
        Vector4f c3 = new Vector4f(m.m13, m.m23, m.m33, m.m43);
        Vector4f c4 = new Vector4f(m.m14, m.m24, m.m34, m.m44);

        return new Matrix4f(r1.dot(c1), r1.dot(c2), r1.dot(c3), r1.dot(c4), 
                            r2.dot(c1), r2.dot(c2), r2.dot(c3), r2.dot(c4),
                            r3.dot(c1), r3.dot(c2), r3.dot(c3), r3.dot(c4),
                            r4.dot(c1), r4.dot(c2), r4.dot(c3), r4.dot(c4));
    }

    public Vector4f multiply(Vector4f v) {
        return new Vector4f(m11*v.x+m12*v.y+m13*v.z+m14*v.w, 
                            m21*v.x+m22*v.y+m23*v.z+m24*v.w, 
                            m31*v.x+m32*v.y+m33*v.z+m34*v.w, 
                            m41*v.x+m42*v.y+m43*v.z+m44*v.w);
    }

    public static Matrix4f lookAt(Vector3f eye, Vector3f target, Vector3f up) {

        Vector3f direction = eye.minus(target).normalize();    
        Vector3f right = direction.cross(up).normalize();
        Vector3f camUp = right.cross(direction);

        direction = direction.negate();

        Matrix4f viewMatrix = new Matrix4f(
                right.x, right.y, right.z, -right.dot(eye),
                camUp.x, camUp.y, camUp.z, -camUp.dot(eye),
                direction.x, direction.y, direction.z, -direction.dot(eye),
                0, 0, 0, 1);

        return viewMatrix;
    }

    public String toString() {
        return  "" +    this.m11 + " " + this.m12 + " " + this.m13 + " " + this.m14 + "\n" + 
                        this.m21 + " " + this.m22 + " " + this.m23 + " " + this.m24 + "\n" + 
                        this.m31 + " " + this.m32 + " " + this.m33 + " " + this.m34 + "\n" + 
                        this.m41 + " " + this.m42 + " " + this.m43 + " " + this.m44 + "\n";
    }

    public FloatBuffer toFloatBuffer() {
        return BufferUtils.createFloatBuffer(ArrayUtils.toFloatArray(this.toRawTypes()));
    }
}

And my camera class:

import com.engine.componentsystem.Component;
import com.engine.eventsystem.eventtypes.engineevents.UpdateEvent;
import com.engine.eventsystem.eventtypes.mouseevents.MouseMovedEvent;
import com.engine.eventsystem.eventtypes.mouseevents.MousePressedEvent;
import com.engine.eventsystem.eventtypes.mouseevents.MouseReleasedEvent;
import com.engine.eventsystem.eventtypes.mouseevents.MouseScrolledEvent;
import com.engine.toolbox.math.matrix.Matrix4f;
import com.engine.toolbox.math.quaternion.Quaternion;
import com.engine.toolbox.math.vectors.Vector3f;

public class CameraComponent extends Component{

    private boolean right = false, left = false;
    private float mouseX = 0, mouseY = 0, oldMouseX, oldMouseY;

    private float distanceFromTarget = -5, angleAroundTarget;

    private Vector3f position = new Vector3f(0.0f), rotation = new Vector3f(0.0f), target;

    public CameraComponent() {}

    public Vector3f getPosition() {
        return position;
    }

    public Vector3f getRotation() {
        return rotation;
    }

    public Vector3f getTarget() {
        return target;
    }

    private float calculateHorizontalDistance() {
        return (float) (this.distanceFromTarget*Math.cos(Math.toRadians(this.rotation.x)));
    }

    private float calculateVerticalDistance() {
        return (float) (this.distanceFromTarget*Math.sin(Math.toRadians(this.rotation.x)));
    }

    public Matrix4f getViewMatrix() {
        Matrix4f result = Matrix4f.identity();
        Matrix4f rotationM = Quaternion.fromEuler(rotation).toMatrix();
        Matrix4f translation = Matrix4f.translation(position.negate());
        Matrix4f origin = Matrix4f.translation(target);
        Matrix4f Iorigin = Matrix4f.translation(target.negate());

        System.out.println("target: " + target);
        System.out.println("position: " + position);
        System.out.println("rotation: " + rotation);

        result = result.multiply(translation);
        result = result.multiply(origin);
        result = result.multiply(rotationM);
        result = result.multiply(Iorigin);
        return result;
    }

    @Override
    public void onAttach() {
        this.target = ((PositionComponent) parent.getComponent(PositionComponent.class)).getPosition();
    }

    @Override
    public boolean onMouseScrolled(MouseScrolledEvent event) {
        this.distanceFromTarget += event.getyOffset()*0.7f;
        return false;
    }

    @Override
    public boolean onMousePressed(MousePressedEvent event) {
        if(event.getButton() == 0 && event.getMods() == 1) {
            left = true;
        }
        if(event.getButton() == 1 && event.getMods() == 1) {
            right = true;
        }
        return false;
    }

    @Override
    public boolean onMouseReleased(MouseReleasedEvent event) {
        if(event.getButton() == 0) {
            left = false;
        }
        if(event.getButton() == 1) {
            right = false;
        }
        return false;
    }

    @Override
    public boolean onMouseMoved(MouseMovedEvent event) {
        oldMouseX = mouseX;
        oldMouseY = mouseY;
        mouseX = event.getX();
        mouseY = event.getY();
        if(left) {
            this.angleAroundTarget -= (mouseX-oldMouseX)*0.01f;
        }
        if(right) {
            this.rotation.x -= (mouseY-oldMouseY)*0.01f;
        }
        return false;
    }

    @Override
    public void onUpdate(UpdateEvent event) {
        this.target = ((PositionComponent) parent.getComponent(PositionComponent.class)).getPosition();
        float horizantalDistance = calculateHorizontalDistance();
        float verticalDistance = calculateVerticalDistance();
        float theta = ((RotationComponent) parent.getComponent(RotationComponent.class)).getRotation().y + angleAroundTarget;

        float xOffset = (float) (horizantalDistance*Math.sin(Math.toRadians(theta)));
        float zOffset = (float) (horizantalDistance*Math.cos(Math.toRadians(theta)));

        this.position.x = target.x - xOffset;
        this.position.y = target.y + verticalDistance;
        this.position.z = target.z - zOffset;

        this.rotation.y = (float) (Math.toRadians(180.0f) - theta);
    }

}

Answer:

So I managed to fix my problem, by

  1. Changing to Matrix multiplication order to rotate first and move then (I also added new functions to the Matrix4f class to make this easier)
result = result.rotate((float) -this.rotation.x, new Vector3f(1f, 0f, 0f));
result = result.rotate((float) this.rotation.y, new Vector3f(0f, 1f, 0f));
result = result.translate(position.negate());
  1. Fixing the mistake a made in the quaternion class which was breaking the rotation (i forgot the turn the degrees into radians)
roll = (float) Math.toRadians(roll);
pitch = (float) Math.toRadians(pitch);
yaw = (float) Math.toRadians(yaw);

The improved Camera class (I added float smoothing to remove all the jerkiness from the motions):

package com.assec.engine.componentsystem.components;

import com.assec.engine.componentsystem.Component;
import com.assec.engine.eventsystem.engineevents.UpdateEvent;
import com.assec.engine.eventsystem.mouseevents.MouseMovedEvent;
import com.assec.engine.eventsystem.mouseevents.MousePressedEvent;
import com.assec.engine.eventsystem.mouseevents.MouseReleasedEvent;
import com.assec.engine.eventsystem.mouseevents.MouseScrolledEvent;
import com.assec.engine.toolbox.math.SmoothFloat;
import com.assec.engine.toolbox.math.matrix.Matrix4f;
import com.assec.engine.toolbox.math.vectors.Vector3f;

public class CameraComponent extends Component{

    private boolean right = false, left = false;
    private float mouseX = 0.0f, mouseY = 0.0f, oldMouseX, oldMouseY;

    private SmoothFloat distanceFromTarget = new SmoothFloat(-10.0f, 5.0f);
    private SmoothFloat angleAroundTarget = new SmoothFloat(0.0f, 10.0f);

    private SmoothFloat pitch = new SmoothFloat(0.0f, 10.0f);

    private Vector3f position = new Vector3f(0.0f), rotation = new Vector3f(0.0f), target;

    public CameraComponent() {}

    public Vector3f getPosition() {
        return position;
    }

    public Vector3f getRotation() {
        return rotation;
    }

    public Vector3f getTarget() {
        return target;
    }

    private float calculateHorizontalDistance() {
        return (float) (this.distanceFromTarget.getActual()*Math.cos(Math.toRadians(this.rotation.x)));
    }

    private float calculateVerticalDistance() {
        return (float) (this.distanceFromTarget.getActual()*Math.sin(Math.toRadians(this.rotation.x)));
    }

    public Matrix4f getViewMatrix() {
        Matrix4f result = Matrix4f.identity();

        this.target = new Vector3f(0.0f);

        this.rotation.x = pitch.getActual();
        this.rotation.x%= 360;

        float horizontalDistance = calculateHorizontalDistance();
        float verticalDistance = calculateVerticalDistance();
        float theta = angleAroundTarget.getActual();

        float xOffset = (float) (horizontalDistance*Math.sin(Math.toRadians(theta)));
        float zOffset = (float) (horizontalDistance*Math.cos(Math.toRadians(theta)));

        this.position.x = target.x - xOffset;
        this.position.y = target.y + verticalDistance;
        this.position.z = target.z - zOffset;

        this.rotation.y = 360.0f - theta;
        this.rotation.y%= 360;

        result = result.rotate((float) -this.rotation.x, new Vector3f(1f, 0f, 0f));
        result = result.rotate((float) this.rotation.y, new Vector3f(0f, 1f, 0f));
        result = result.translate(position.negate());

        return result;
    }

    @Override
    public void onAttach() {
        this.target = ((PositionComponent) parent.getComponent(PositionComponent.class)).getPosition();
        this.target = new Vector3f(0.0f);
    }

    @Override
    public boolean onMouseScrolled(MouseScrolledEvent event) {
        this.distanceFromTarget.increaseTarget(-event.getyOffset()*0.7f);
        return false;
    }

    @Override
    public boolean onMousePressed(MousePressedEvent event) {
        if(event.getButton() == 0 && event.getMods() == 1) {
            left = true;
        }
        if(event.getButton() == 1 && event.getMods() == 1) {
            right = true;
        }
        return false;
    }

    @Override
    public boolean onMouseReleased(MouseReleasedEvent event) {
        if(event.getButton() == 0) {
            left = false;
        }
        if(event.getButton() == 1) {
            right = false;
        }
        return false;
    }

    @Override
    public boolean onMouseMoved(MouseMovedEvent event) {
        oldMouseX = mouseX;
        oldMouseY = mouseY;
        mouseX = (float) event.getX();
        mouseY = (float) event.getY();
        if(left) {
            this.angleAroundTarget.increaseTarget(-(mouseX-oldMouseX)*0.1f);
        }
        if(right) {
            this.pitch.increaseTarget(-(mouseY-oldMouseY)*0.1f);
        }
        return false;
    }

    @Override
    public void onUpdate(UpdateEvent event) {
        this.angleAroundTarget.update(0.01f);
        this.distanceFromTarget.update(0.01f);
        this.pitch.update(0.01f);
    }

}

The improved Matrix4f class:

package com.assec.engine.toolbox.math.matrix;

import java.nio.FloatBuffer;
import java.util.Arrays;
import java.util.Iterator;

import com.assec.engine.toolbox.ArrayUtils;
import com.assec.engine.toolbox.BufferUtils;
import com.assec.engine.toolbox.math.quaternion.Quaternion;
import com.assec.engine.toolbox.math.vectors.Vector3f;
import com.assec.engine.toolbox.math.vectors.Vector4f;

public class Matrix4f extends Matrix{

    public float m11, m12, m13, m14, m21, m22, m23, m24, m31, m32, m33, m34, m41, m42, m43, m44;

    public Matrix4f(float m11, float m12, float m13, float m14,
                    float m21, float m22, float m23, float m24,
                    float m31, float m32, float m33, float m34,
                    float m41, float m42, float m43, float m44) {
        super(new Float[] { m11, m12, m13, m14, 
                            m21, m22, m23, m24, 
                            m31, m32, m33, m34, 
                            m41, m42, m43, m44});
        this.m11 = m11;
        this.m12 = m12;
        this.m13 = m13;
        this.m14 = m14;
        this.m21 = m21;
        this.m22 = m22;
        this.m23 = m23;
        this.m24 = m24;
        this.m31 = m31;
        this.m32 = m32;
        this.m33 = m33;
        this.m34 = m34;
        this.m41 = m41;
        this.m42 = m42;
        this.m43 = m43;
        this.m44 = m44;

    }

    @Override
    public Iterator<Float> iterator() {
        return Arrays.asList(this.toRawTypes()).iterator();
    }

    public Matrix4f invert() {
        Matrix4f matrixOfMinors =   new Matrix4f(   new Matrix3f(   m22, m23, m24,
                                                                    m32, m33, m34,
                                                                    m42, m43, m44).getDeterminant(), new Matrix3f(  m21, m23, m24,
                                                                                                                    m31, m33, m34,
                                                                                                                    m41, m43, m44).getDeterminant(), new Matrix3f(  m21, m22, m24,
                                                                                                                                                                    m31, m32, m34,
                                                                                                                                                                    m41, m42, m44).getDeterminant(), new Matrix3f(  m21, m22, m23,
                                                                                                                                                                                                                    m31, m32, m33,
                                                                                                                                                                                                                    m41, m42, m43).getDeterminant(),
                                                    new Matrix3f(   m12, m13, m14,
                                                                    m32, m33, m34,
                                                                    m42, m43, m44).getDeterminant(), new Matrix3f(  m11, m13, m14,
                                                                                                                    m31, m33, m34,
                                                                                                                    m41, m43, m44).getDeterminant(), new Matrix3f(  m11, m12, m14,
                                                                                                                                                                    m31, m32, m34,
                                                                                                                                                                    m41, m42, m44).getDeterminant(), new Matrix3f(  m11, m12, m23,
                                                                                                                                                                                                                    m31, m32, m33,
                                                                                                                                                                                                                    m41, m42, m43).getDeterminant(),
                                                    new Matrix3f(   m12, m13, m14,
                                                                    m22, m23, m24,
                                                                    m42, m43, m44).getDeterminant(), new Matrix3f(  m11, m13, m14,
                                                                                                                    m21, m23, m24,
                                                                                                                    m41, m43, m44).getDeterminant(), new Matrix3f(  m11, m12, m14,
                                                                                                                                                                    m21, m22, m24,
                                                                                                                                                                    m41, m42, m44).getDeterminant(), new Matrix3f(  m11, m12, m23,
                                                                                                                                                                                                                    m21, m22, m23,
                                                                                                                                                                                                                    m41, m42, m43).getDeterminant(),
                                                    new Matrix3f(   m12, m13, m14,
                                                                    m22, m23, m24,
                                                                    m32, m33, m34).getDeterminant(), new Matrix3f(  m11, m13, m14,
                                                                                                                    m21, m23, m24,
                                                                                                                    m31, m33, m34).getDeterminant(), new Matrix3f(  m11, m12, m14,
                                                                                                                                                                    m21, m22, m24,
                                                                                                                                                                    m31, m32, m34).getDeterminant(), new Matrix3f(  m11, m12, m23,
                                                                                                                                                                                                                    m21, m22, m23,
                                                                                                                                                                                                                    m31, m32, m33).getDeterminant());
        matrixOfMinors.m12 = -m12;
        matrixOfMinors.m14 = -m14;

        matrixOfMinors.m21 = -m21;
        matrixOfMinors.m23 = -m23;

        matrixOfMinors.m32 = -m32;
        matrixOfMinors.m34 = -m34;

        matrixOfMinors.m41 = -m41;
        matrixOfMinors.m43 = -m43;

        float temp = 0;

        temp = matrixOfMinors.m12;
        matrixOfMinors.m12 = matrixOfMinors.m21;
        matrixOfMinors.m21 = temp;

        temp = matrixOfMinors.m13;
        matrixOfMinors.m13= matrixOfMinors.m31;
        matrixOfMinors.m31 = temp;

        temp = matrixOfMinors.m23;
        matrixOfMinors.m23 = matrixOfMinors.m32;
        matrixOfMinors.m32 = temp;

        temp = matrixOfMinors.m14;
        matrixOfMinors.m14 = matrixOfMinors.m41;
        matrixOfMinors.m41 = temp;

        temp = matrixOfMinors.m24;
        matrixOfMinors.m24 = matrixOfMinors.m42;
        matrixOfMinors.m42 = temp;

        temp = matrixOfMinors.m34;
        matrixOfMinors.m34 = matrixOfMinors.m43;
        matrixOfMinors.m43 = temp;

        float d = this.getDeterminant();
        return new Matrix4f(matrixOfMinors.m11/d, matrixOfMinors.m12/d, matrixOfMinors.m13/d, matrixOfMinors.m14/d,
                            matrixOfMinors.m21/d, matrixOfMinors.m22/d, matrixOfMinors.m23/d, matrixOfMinors.m24/d,
                            matrixOfMinors.m31/d, matrixOfMinors.m32/d, matrixOfMinors.m33/d, matrixOfMinors.m34/d,
                            matrixOfMinors.m41/d, matrixOfMinors.m42/d, matrixOfMinors.m43/d, matrixOfMinors.m44/d);
    }

    public float getDeterminant() {
        return  m11*(new Matrix3f(  m22, m23, m24,
                                    m32, m33, m34,
                                    m42, m43, m44).getDeterminant()) - 
                m12*(new Matrix3f(  m21, m23, m24,
                                    m31, m33, m34,
                                    m41, m43, m44).getDeterminant()) + 
                m13*(new Matrix3f(  m21, m22, m24,
                                    m31, m32, m34,
                                    m41, m42, m44).getDeterminant()) - 
                m14*(new Matrix3f(  m21, m22, m23,
                                    m31, m32, m33,
                                    m41, m42, m43).getDeterminant());
    }

    public static Matrix4f identity() {
        return new Matrix4f(1, 0, 0, 0, 
                            0, 1, 0, 0, 
                            0, 0, 1, 0, 
                            0, 0, 0, 1);
    }

    public static Matrix4f perspective(float aspectRatio, float viewAngle, float nearPlane, float farPlane) {
        return new Matrix4f((float) (1/(aspectRatio*Math.tan((viewAngle/2)))),  0,                                      0,                                          0,
                            0,                                                  (float) (1/(Math.tan((viewAngle/2)))),  0,                                          0,
                            0,                                                  0,                                      -((farPlane+nearPlane)/(farPlane-nearPlane)), -(2*farPlane*nearPlane)/(farPlane-nearPlane),
                            0,                                                  0,                                      -1,                                         1);
    }

    public static Matrix4f translation(Vector3f translation) {
        Matrix4f result = Matrix4f.identity();
        result.m14 = translation.x;
        result.m24 = translation.y;
        result.m34 = translation.z;
        return result;
    }

    public static Matrix4f scale(Vector3f scale) {
        Matrix4f result = Matrix4f.identity();
        result.m11 = scale.x;
        result.m22 = scale.y;
        result.m33 = scale.z;
        return result;
    }

    public Matrix4f minus(Matrix4f m) {
        return new Matrix4f(m11-m.m11, m12-m.m12, m13-m.m13, m14-m.m14,
                            m21-m.m21, m22-m.m22, m13-m.m23, m24-m.m24,
                            m31-m.m31, m32-m.m32, m13-m.m33, m34-m.m34,
                            m41-m.m41, m42-m.m42, m13-m.m43, m44-m.m44);
    }

    public Matrix4f plus(Matrix4f m) {
        return new Matrix4f(m11+m.m11, m12+m.m12, m13+m.m13, m14+m.m14,
                            m21+m.m21, m22+m.m22, m13+m.m23, m24+m.m24,
                            m31+m.m31, m32+m.m32, m13+m.m33, m34+m.m34,
                            m41+m.m41, m42+m.m42, m13+m.m43, m44+m.m44);
    }

    public Matrix4f multiply(Matrix4f m) {
        Vector4f r1 = new Vector4f(m11, m12, m13, m14);
        Vector4f r2 = new Vector4f(m21, m22, m23, m24);
        Vector4f r3 = new Vector4f(m31, m32, m33, m34);
        Vector4f r4 = new Vector4f(m41, m42, m43, m44);

        Vector4f c1 = new Vector4f(m.m11, m.m21, m.m31, m.m41);
        Vector4f c2 = new Vector4f(m.m12, m.m22, m.m32, m.m42);
        Vector4f c3 = new Vector4f(m.m13, m.m23, m.m33, m.m43);
        Vector4f c4 = new Vector4f(m.m14, m.m24, m.m34, m.m44);

        return new Matrix4f(r1.dot(c1), r1.dot(c2), r1.dot(c3), r1.dot(c4), 
                            r2.dot(c1), r2.dot(c2), r2.dot(c3), r2.dot(c4),
                            r3.dot(c1), r3.dot(c2), r3.dot(c3), r3.dot(c4),
                            r4.dot(c1), r4.dot(c2), r4.dot(c3), r4.dot(c4));
    }

    public Vector4f multiply(Vector4f v) {
        return new Vector4f(m11*v.x+m12*v.y+m13*v.z+m14*v.w, 
                            m21*v.x+m22*v.y+m23*v.z+m24*v.w, 
                            m31*v.x+m32*v.y+m33*v.z+m34*v.w, 
                            m41*v.x+m42*v.y+m43*v.z+m44*v.w);
    }

    public Matrix4f rotate(float angle, Vector3f axis) {
        Matrix4f rotation = Quaternion.fromEuler(axis.normalized().scaled(angle)).toMatrix();
        return this.multiply(rotation);
    }

    public Matrix4f translate(Vector3f translation) {
        Matrix4f translate = Matrix4f.translation(translation);
        return this.multiply(translate);
    }

    public static Matrix4f lookAt(Vector3f eye, Vector3f target, Vector3f up) {

        Vector3f direction = eye.minus(target).normalized();    
        Vector3f right = direction.cross(up).normalized();
        Vector3f camUp = right.cross(direction);

        direction = direction.negate();

        Matrix4f viewMatrix = new Matrix4f(
                right.x, right.y, right.z, -right.dot(eye),
                camUp.x, camUp.y, camUp.z, -camUp.dot(eye),
                direction.x, direction.y, direction.z, -direction.dot(eye),
                0.0f, 0.0f, 0.0f, 1.0f);
        return viewMatrix;
    }

    public String toString() {
        return  "" +    this.m11 + " " + this.m12 + " " + this.m13 + " " + this.m14 + "\n" + 
                        this.m21 + " " + this.m22 + " " + this.m23 + " " + this.m24 + "\n" + 
                        this.m31 + " " + this.m32 + " " + this.m33 + " " + this.m34 + "\n" + 
                        this.m41 + " " + this.m42 + " " + this.m43 + " " + this.m44 + "\n";
    }

    public FloatBuffer toFloatBuffer() {
        return BufferUtils.createFloatBuffer(ArrayUtils.toFloatArray(this.toRawTypes()));
    }
}

Question:

I try to merge multiple meshes with a transformation matrix into a single mesh. Each mesh has 4 data sets.

  1. Vertices
  2. Indices
  3. Texture Coordinates
  4. Normals

The way I'm trying to do it is supposed to be lazy and not cost that much CPU. It is a 3 step process.

  1. Multiply each vertex and normal with the transformation matrix.
  2. Merge the Vertices, Texture Coordinates and Normals of each mesh into 3 big arrays.
  3. Merge the Indices of each mesh into a single array but use the sum of the previous meshes as an offset. For example: If mesh 1 has 800 indices then 800 has to be added to all of the indices from mesh 2.

This method has two big problems.

  1. Duplicate vertices are not shared
  2. Parts that are invisible due to clipping are not removed

But that is OK as this is supposed to be a lazy method with not much CPU usage. It is already optimal for creating meshes for grass and bushes.

I have attempted an implementation of this method which looks like this:

public static final MeshData mergeLazy(List<MeshData> meshes, List<Matrix4f> transformations) {

    int lengthVertices = 0;
    int lengthNormals = 0;
    int lengthTexCoords = 0;
    int lengthIndices = 0;
    ArrayList<Integer> indexLengths = new ArrayList<>();

    for(MeshData mesh : meshes) {

        lengthVertices += mesh.getVertices().length;
        lengthNormals += mesh.getNormals().length;
        lengthTexCoords += mesh.getTextureCoordinates().length;
        int length = mesh.getIndices().length;
        lengthIndices += length;
        indexLengths.add(length);
    }

    float[] vertices = new float[lengthVertices];
    float[] texCoords = new float[lengthTexCoords];
    float[] normals = new float[lengthNormals];
    int[] indices = new int[lengthIndices];

    int iv = 0;
    int ivt = 0;
    int ivn = 0;
    int i = 0;
    int indexLength = 0;

    for(int im = 0; im < meshes.size(); im++) {

        MeshData mesh = meshes.get(im);
        float[] mVertices = mesh.getVertices();
        float[] mTexCoords = mesh.getTextureCoordinates();
        float[] mNormals = mesh.getNormals();
        int[] mIndices = mesh.getIndices();
        Matrix4f transformation = transformations.get(im);

        for(int index = 0; index < mVertices.length; index += 3) {

            Vector3f vertex = MatrixUtil.multiply(transformation, mVertices[index], mVertices[index + 1], mVertices[index + 2]);
            vertices[iv++] = vertex.x;
            vertices[iv++] = vertex.y;
            vertices[iv++] = vertex.z;

            Vector3f normal = MatrixUtil.multiply(transformation, mNormals[index], mNormals[index + 1], mNormals[index + 2]);
            normals[ivn++] = normal.x;
            normals[ivn++] = normal.y;
            normals[ivn++] = normal.z;
        }

        for(int index = 0; index < mTexCoords.length; index++) {

            texCoords[ivt++] = mTexCoords[index];
        }

        for(int index = 0; index < mIndices.length; index++) {

            indices[i++] = indexLength + mIndices[index];
        }

        indexLength += indexLengths.get(im);
    }

    MeshData data = new MeshData();
    data.setIndices(indices);
    data.setNormals(normals);
    data.setTextureCoordinates(texCoords);
    data.setVertices(vertices);

    return data;
}

In the end I actually have a single mesh and the multiplying of the transformation also works.... for rotation and scaling, but here come the problems.

The multiplying with the transformation does NOT work for the translation. My method for multiplying a matrix with a vector looks like this:

public static final Vector3f multiply(Matrix4f matrix, float x, float y, float z) {

    Vector3f result = new Vector3f();
    result.x = x * matrix.m00 + y * matrix.m01 + z * matrix.m02;
    result.y = x * matrix.m10 + y * matrix.m11 + z * matrix.m12;
    result.z = x * matrix.m20 + y * matrix.m21 + z * matrix.m22;
    return result;
}

And the second problem is that the textures of the second mesh are somewaht off.

Here is a picture:

As you can see the second mesh only has about 1/4 of the actual texture.

The code I used to generate this mesh looks like this:

    Material grassMaterial = new Material();
    grassMaterial.setMinBrightness(0.1F);
    grassMaterial.setColorMap(new Texture(new XImgTextureReader().read(new FileInputStream("res/textures/grass2.ximg"))));
    grassMaterial.setAffectedByLight(true);
    grassMaterial.setTransparent(true);
    grassMaterial.setUpwardsNormals(true);
    grassMaterial.setFog(fog);

    MeshData quad = Quad.generateMeshData(
        new Vector3f(0.0F, 1F, 0.0F),
        new Vector3f(0.0F, 0.0F, 0.0F),
        new Vector3f(1F, 0.0F, 0.0F),
        new Vector3f(1F, 1F, 0.0F)
    );

    StaticMesh grassMesh = new StaticMesh(MeshUtil.mergeLazy(Arrays.asList(quad, quad), Arrays.asList(
        MatrixUtil.createTransformationMatrx(
            new Vector3f(0.0F, 0.0F, 0.0F),
            new Vector3f(0.0F, 0.0F, 0.0F),
            new Vector3f(1.0F, 1.0F, 1.0F)
        ),
        MatrixUtil.createTransformationMatrx(
            new Vector3f(0F, 0.0F, -0F),
            new Vector3f(0.0F, 90.0F, 0.0F),
            new Vector3f(1.0F, 1.0F, 1.0F)
        )
    )));
    grassMesh.setCullMode(StaticMesh.CULLING_DISABLED);

    Entity grass = new Entity();
    grass.setShaderPipeline(shaderPipeline);
    grass.setMaterial(grassMaterial);
    grass.setMesh(grassMesh);
    grass.setTranslation(0, 0, 1);

My question now is: What did I do wrong? Why is the texture so weird and why does the multiplication with the transformation not work for the translation?

If you need more of the code, I have a GitHub Repo with the Eclipse Project here: https://github.com/RalleYTN/Heroica-Fabulis


Answer:

Thanks to @Rabbid76 I came closer to my answer and now I have finally found the problem. The first problem with the translation not working was fixed by multiplying the transformation vertically instead of horizontally. Thanks again @Rabidd76 .

And the reason why the textures where so weird is because I merged the indices incorrectly. I should not have taken the sum of all indices in the meshes before as offset but the sum of the vertices.

Here is now the working method:

public static final MeshData mergeLazy(List<MeshData> meshes, List<Matrix4f> transformations) {

    ArrayList<Float> vertices = new ArrayList<>();
    ArrayList<Float> texCoords = new ArrayList<>();
    ArrayList<Float> normals = new ArrayList<>();
    ArrayList<Integer> indices = new ArrayList<>();
    int offset = 0;
    int m = 0;

    for(MeshData mesh : meshes) {

        Matrix4f transformation = transformations.get(m);
        float[] mVertices = mesh.getVertices();
        float[] mNormals = mesh.getNormals();

        for(int index = 0; index < mesh.getVertices().length; index += 3) {

            Vector3f vertex = MatrixUtil.multiply(transformation, mVertices[index], mVertices[index + 1], mVertices[index + 2]);
            vertices.add(vertex.x);
            vertices.add(vertex.y);
            vertices.add(vertex.z);

            Vector3f normal = MatrixUtil.multiply(transformation, mNormals[index], mNormals[index + 1], mNormals[index + 2]);
            normals.add(normal.x);
            normals.add(normal.y);
            normals.add(normal.z);
        }

        ListUtil.addFloatArray(texCoords, mesh.getTextureCoordinates());
        int[] mIndices = mesh.getIndices();

        for(int index : mIndices) {

            indices.add(index + offset);
        }

        offset += mVertices.length / 3;
        m++;
    }

    MeshData mesh = new MeshData();
    mesh.setIndices(ListUtil.toPrimitiveIntArray(indices));
    mesh.setNormals(ListUtil.toPrimitiveFloatArray(normals));
    mesh.setTextureCoordinates(ListUtil.toPrimitiveFloatArray(texCoords));
    mesh.setVertices(ListUtil.toPrimitiveFloatArray(vertices));

    return mesh;
}

Question:

I have a matrix4f that I'm passing from my ShaderProgram class into my vertex shader class using uniform variables. This matrix is supposed to act as a translation for the vertices. The following is what the matrix looks like

1, 0, 0, 0,
0, 1, 0, 0,
0, 0, 1, 0,
0, 0, 0, 1

When I multiply that variable (Called "test") by the vertex points (Called gl_Vertex) nothing is visible, it just leaves a blank screen. This only happens when I multiply it by the uniform variable "test", if I multiply it by a new matrix4f with the same values, it works normally. If I use vector uniform variables instead of matrices it works as expected.

Am I passing the variable into the GLSL vertex shader class correctly? And if so, why is my quad not showing up on the screen?

Here is my vertex shader

#version 400 core

uniform vec4 translation;
uniform vec4 size;
uniform vec4 rotation;
uniform mat4 test;

in vec2 textureCoords;
in vec3 position;
out vec2 pass_textureCoords;

void main(void){
    //pass texture cords
    pass_textureCoords = textureCoords;

    //This works by multiplying by identity matrix
    //gl_Position = mat4(1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1) * gl_Vertex;

    //This works by passing vec4's not matrix4
    /*gl_Position = vec4(((gl_Vertex.x + translation.x)*size.x),
                        ((gl_Vertex.y + translation.y)*size.y),
                        ((gl_Vertex.z + translation.z)*size.z),
                        ((gl_Vertex.w + translation.w)*size.w)
                        );*/

    //this leaves a blank window
    gl_Position = test * gl_Vertex;

}

This is how I declare the uniform variable locations:

translationLocation = GL20.glGetUniformLocation(programID, "translation");
sizeLocation = GL20.glGetUniformLocation(programID, "size");
rotationLocation = GL20.glGetUniformLocation(programID, "rotation");
textureLocation = GL20.glGetUniformLocation(programID, "textureSampler");
testMat = GL20.glGetUniformLocation(programID, "test");

This is how I render the uniform variables

public void start(){
    GL20.glUseProgram(programID);

    Vector4f translation = offset.getTranslation();
    Vector4f size = offset.getSize();
    Vector4f rotation = offset.getRotation();

    GL20.glUniform4f(translationLocation, translation.x, translation.y, translation.z,      translation.w);
    GL20.glUniform4f(sizeLocation, size.x, size.y, size.z, size.w);
    GL20.glUniform4f(rotationLocation, rotation.x, rotation.y, rotation.z, rotation.w);

    FloatBuffer buff = BufferUtils.createFloatBuffer(16);
    offset.getTestTranslation().storeTranspose(buff);
    GL20.glUniformMatrix4(testMat, false, buff);

    GL20.glUniform1i(textureLocation, 0);
}

And this is how I declare my variables before passing it into GLSL

Vector4f translation;
Vector4f size;
Vector4f rotation;
Matrix4f testTranslation;

public Offset(){
    translation = new Vector4f(0, 0, 0, 0);
    size = new Vector4f(1, 1, 1, 1);
    rotation = new Vector4f(0, 0 , 0, 0);
    testTranslation = new Matrix4f();
    testTranslation.translate(new Vector3f(0,0,0));
}

Answer:

Well, it turns out that I was using the following method to convert the matrix4f into a floatBuffer

matrix4f.storeTranspose(buff)

When apparently that doesn't properly store the matrix into a float buffer. I'm now using this method to send the matrix to the vertex shader while rendering the shader program

public void setMatrixArray(boolean transposed, Matrix4f[] matrices){

    FloatBuffer matrixBuffer = BufferUtils.createFloatBuffer(16*matrices.length);
    for(int i = 0; i<matrices.length; i++)  {
        matrices[i].store(matrixBuffer);
    }
    matrixBuffer.flip();
    GL20.glUniformMatrix4(testMat,transposed,matrixBuffer);
}