opencv - Consistency of projecting points onto an undistorted image -

I want to project a point in 3D space coordinates in 3D space coordinates. I have calibrated intrints and extronics that I am using. I have the camera matrix K and deformation coefficient D. However, I want the projected image to be coordinated with the undistorted image.

From my research, I found two ways to do this. Use getOptimalNewCameraMatrix function of OpenCV to calculate the camera of a new undistorted image.

In this task with deformation coefficient D, Use the project point function using the raw camera matrix and get the approximate point.

Does the output of both methods match?

I think that something in your mind is missing.

Camera Matrix K and Distributed Coefficient D, are the criteria for disobedience (if your lenses are distorting the image like fisheye) they are called internal camera parameters.

If we change the context of computer graphics in computer terms, those criteria are those that you use to define the frustration of the scene, and, for example, they are used for the camera To get the focal length of

But they are not enough to do projection stuff.

For launch, if you think in the computer graphics word (for example OpenGLog), you need model-visual-projection matrix. Model is the matrix matrix which specifies the position of the object in the world. The visual matrix specifies the position of the camera, and the projection matrix specifies frustration (focal angle, perspective distortion, etc.).

If you want to know how to convert the model's point from 3D to 2D (or vice versa) you need projection and visual matrix (you have the model matrix because you have three digits You want to start with) And in the computer vision the visual matrix is ​​called the estrogenic parameter.

Therefore, you also need the astronomical parameters, which is the position of the camera in the world. For example, these parameters are rvec and tvec that require cv :: projectpoints .

If you want to calculate them, they are actually the output of cv :: solutionPnP that is what you want to do: Contrary to the known 2D projection on the camera screen Together with some known 3D points, this function gives you estrotic parameters (from which you see some open-open CV-enhanced-visual matrix for reality - whatever code via cv :: Rodrigues ).

Last note: While all the pictures you shoot with all the cameras are intuitive parameters fixed (when you do not change the focal length of the course), the astrinisk parameter is every time you have a different one Take the camera to take a new picture View point (which is: it changes the perspective point of the scene, so you want to find 3D-3D projection)

Hope you can get help !