feat(math): add Matrix3x3 and Quaternion for 3D transformations

feat(math): add Matrix3x3 and Quaternion for 3D transformations

- Add Matrix3x3 class for 3D transformations
- Add toMatrix method to Rotation with test
- Use matrix for point transformation
- Cache transformation matrix for performance
- Add Quaternion class for 3D rotations
- Use quaternion internally instead of Euler angles
- Use quaternion-based camera rotation
- Use rotation matrix for movement direction
- Use quaternion for lookAt rotation