How can I get the calibration data for camera and lidar data which I captured?

I have recorded the point cloud data using velodyne - VLP-16 Puck sensor and captured the data using logitech C270 (Normal Webcam). Now, I need to process the data for sensor fusion based object detection concept. For that, I need calib.mat file. But, I need to know how to calculate or get that .mat file. Without use of calib.mat (Calibration Mat file) file, I cannot detect the object in both the images(Image and .pcd data) in sensor fusion code.

People who are working or excel in this domain, kindly assist me on this.

Note:

I tried to get the data using camera calibrator but few values are null (means '0'). With that data i couldn't process further. Also i have a doubt, if I created the calib.mat file using one image, is that fine for all images(image & pcd data)? No need to change for different location of captured data?

Thanks in advance.

[Ramya Sivanandham]

I have tried to explore the LidarCameraCalibrationExample using vlp16 sensor data which is available in the matlab intself, but it is throwing following error:

"Visualize the detected checkerboard by using the helperShowCheckerboardPlanes function.

helperShowCheckerboardPlanes(ptCloudFileNames,indices)"

due to index exceeds number of array elements.

After filtered the point cloud and image files that are not used for detection process, image and point cloud files are became null!

But it is working properly with HDL64 sensor! Kindly check and assist me on this!

Answer 1

[Kritika Bansal]

Hi,

In order to get the calibration between lidar and camera, you can use the lidarCameraCalibrator app. This is a good place to help you get started with the process of calibration. You can refer to the following page for more details on getting started with the app: Get Started with Lidar Camera Calibrator.

The following page can help you with the recommended guidelines for the data collection process: Calibration Guidelines

-Kritika

[Ramya Sivanandham]

Hi Kritika,

Thank you for your information. I require another clarification from you. I have 1000 number of images to do the testing, in that case, should I create the checkerborad pattern for all of those images or can I create the calibration pattern with one image with the object is masked by the checker board pattern?

Kindly clarify my doubt..

Thanks in advance.

[Kritika Bansal]

Hi,

As mentioned in Calibration Guidelines, capture at least 10 pairs of image-pointCloud data. The data needs to have a rectangular checkerboard pattern held at different positions. The more the data, the better the calibration results. Please also note that the processing time depends on the number of data pairs provided to the app.

-Kritika

[Ramya Sivanandham]

Ok, Sure, thank you very much!. Timely help!

[Ramya Sivanandham]

Hi @Kritika Bansal and all,

I have an another doubts,

1. With the help of above said method & guidelines, if I generate a .mat file using LogiCam (Camera-1) and VLP-16 (LiDAR Sensor) at one location(In-house / Lab), will it be used for other locations like Roadways / different weather conditions / outside of the building?
2. Is this possible to test the data which is captured from Econ Camera (Camera-2) and VLP-16 (Lidar Sensor) with the same .mat file (Calibration data of Logicam & VLP-16) ?

Please excuse if it is silly question! Kindly clarify doubt.

Thanks in advance.

[Kritika Bansal]

Hi,

Please find the answers below:

1. Yes, you should be able to use for other locations provided the camera-lidar setup/ mounting remains the same.
2. No, if the camera changes you need to re-calibrate the lidar and camera again. This is because, with change of camera the camera intrinsics would also change.

-Kritika

[Ramya Sivanandham]

Yeah, OK.

Tons of Thanks for your quick response :)

[Ramya Sivanandham]

@Kritika Bansal & all,

Hi Kritika, as per the instruction, i have captured the data using checkerboard pattern. When i tried to calibrate through the lidarCameraCalibrator app by importing the camera and lidar(.pcd) data, it throwing the error that " Matching image-point cloud pair files not found". But, I have verified the number of images and frames which I imported. I couldn't find the bug where and what i have missed to do!

Kindly assist me on this!

Thanks in advance.

[Kritika Bansal]

Hi,

This typically happens when there are no matching pairs available in the data folders selected. Matching pair refers to an image and a point cloud file having same name. So, the app expects the names of the images to match with their corresponding point cloud.

For example-

0001.png & 0001.pcd

0002.png & 0002.pcd

... and so on.

[Ramya Sivanandham]

Oh, oh, OK... I got it. Yeah, different names only, I'll change it..

Thank you so much!!!!!!!!!!

Sorry for the silly questions.. but I dont know.. I just started to explore the Lidar toolbox and calibration activities..

[Ramya Sivanandham]

Hi Kritika and others,

I have an issue with the calibration of camera and lidar. I would like to know whether the process which I have done is proper or not! But I followed your instructions (Mathworks), but still not getting the matched pair.

1. with the help of checkerboard pattern, have captured the data from camera and lidar
2. captured the data as a frames through mScript.
3. Lidar data captured through the veloview, as a pcap file. From pcap file, pcd file has extracted with the help of matlab convertion. Here is the issue that number of frames in camera and pcd data are different. I couldn't find the similar frames to do the calibration.
4. another issue is, when i upload the lidar data, there won't be the proper ROI to detect the checkerboard. when i tried to adjust the ROI, placing of the ROI box is not happened.
5. Sometimes, 1 frame will get matched but it was detected the different region from the checkerboard pattern.

My obervation from this issues,

1. Due to different frame rate of lidar pcd data extraction from the pcap file, number of frame pair count is varying.
2. Due to the different frame rate issue, couldn't detect the detectected pair.
3. Due to improper ROI, detection is not happened.

Guys, Please let me know what i have missed and where should I concentrate or do the changes!!!!

I would like to try with cropped pcd data, but I didn't know. If someone helps me to crop the particular region of the input pcd data, will be more helpful to do this analysis.

or

Is any way to capture the sensor data at specified angle? (am using VLP-16 sensor)

Kindly assist me on this...

Thanks in advance.