Frame duration issues when trimming videos in MATLAB/ffmpeg?

Hi there,

I have two videos (A.MOV and B.MOV), that are supposed to be perfectly synced. Video A (3708 frames), however has 5 more frames than Video B (3703 frames). When I look at the timestamps of these frames, the extras are at the very end of the video. In addition, the last frame in each video is a "padded" frame, with a duration much larger (around 10 times) than that of the other frames.

I need my videos to be perfectly synced, with the same frame number. Since the extra frames are at the end of the video, I want to trim off the extra frames in A, as well as trim the "padded" long frame off of each video.

I currently am running the below code to do so,

% Specify the input and output file names
inputVideoFile = 'A.MOV';
intermediateVideoFile = 'A_trim_temp.mp4'; % Temporary file
finalOutputFile = 'A_trim_mat.mp4'; % Final output file

% Create VideoReader object
videoReader = VideoReader(inputVideoFile);

% Get the total number of frames and other properties
numFrames = videoReader.NumFrames;
frameRate = videoReader.FrameRate;
videoHeight = videoReader.Height;
videoWidth = videoReader.Width;

% Create VideoWriter object
videoWriter = VideoWriter(intermediateVideoFile, 'MPEG-4');
videoWriter.FrameRate = frameRate;
open(videoWriter);

% Read and write frames except the last 5
for i = 1:(numFrames - 5)
    frame = read(videoReader, i);
    writeVideo(videoWriter, frame);
end

% Close the VideoWriter object
close(videoWriter);

% Handle audio extraction and trimming
% This requires an external tool like FFmpeg, as MATLAB does not handle audio directly

% Command to extract audio
audioExtractCommand = sprintf('ffmpeg -i "%s" -q:a 0 -map a audio_original.wav', inputVideoFile);
[status, cmdout] = system(audioExtractCommand);
if status ~= 0
    error('Error extracting audio: %s', cmdout);
end

% Command to trim audio (matching the length of the trimmed video)
audioTrimCommand = sprintf('ffmpeg -i audio_original.wav -ss 0 -t %f -c copy audio_trimmed.wav', (numFrames - 5) / frameRate);
[status, cmdout] = system(audioTrimCommand);
if status ~= 0
    error('Error trimming audio: %s', cmdout);
end

% Command to merge trimmed audio with the video
audioMergeCommand = sprintf('ffmpeg -i "%s" -i audio_trimmed.wav -c:v copy -c:a aac -strict experimental "%s"', intermediateVideoFile, finalOutputFile);
[status, cmdout] = system(audioMergeCommand);
if status ~= 0
    error('Error merging audio and video: %s', cmdout);
end

% Clean up temporary audio files
delete('audio_original.wav');
delete('audio_trimmed.wav');
delete(intermediateVideoFile); % Delete the intermediate file

When I run this for both videos, they both return the same frame number (yay!). The timestamps of every frame between A_trim_mat and B_trim_mat match, except for the timestamp of the last frame ( A_trim_mat = 30.8910, B_trim_mat = 30.8890), and perhaps most concerningly, the frame duration between the last two frames of each video is negative (A_trim_mat = -0.0063, B_trim_mat = -0.0083). The duration for all other frames is constant at 0.0083 for both videos.

Any suggestions for how to remedy this are much appreciated. Ideally, I want the timestamps for every frame to match between A and B, and I want the frame duration for each frame to match between A and B.

[Sumukh]

Can you provide the video files being used in the code?

Answer 1

[Ayush]

Hi @Jade,

I understand you are facing issues with frame timestamp mismatch and the frame duration between the last two frames of each video is negative.

As no audio files are provided, some of the potential reasons for such errors are given below:

1. Ensure both videos have consistent frame rates:

You may want to check if they both are set to a common frame rate before proceeding with trimming. You can force the frame rate during your video writing process if needed.

2. You can use “FFmpeg” more extensively: In your code, you are handling audio and video separately, instead of doing this, you can use “FFmpeg” commands to trim both audio and video in one go. This helps in synchronizing both tracks more accurately.

This synchronizing error might be causing the issue.

Here’s an adjusted workflow using “FFmpeg” commands to trim both audio and video in one go.

Here’s the code for the same:

% Input files 
inputVideoA = 'A.MOV'; 
inputVideoB = 'B.MOV'; 
finalOutputA = 'A_trim_mat.mp4'; 
finalOutputB = 'B_trim_mat.mp4'; 

% Use FFmpeg for trimming 
trimACommand = sprintf('ffmpeg -i "%s" -vf "trim=end_frame=3703,setpts=N/FRAME_RATE" -c:v libx264 -preset fast -crf 18 -y "%s"', inputVideoA, finalOutputA); 
system(trimACommand); 

trimBCommand = sprintf('ffmpeg -i "%s" -vf "trim=end_frame=3703,setpts=N/FRAME_RATE" -c:v libx264 -preset fast -crf 18 -y "%s"', inputVideoB, finalOutputB); 
system(trimBCommand); 

% Now, handle audio extraction and merging, ensure you match the new lengths 
audioACommand = sprintf('ffmpeg -i "%s" -q:a 0 -map a audioA.wav', finalOutputA); 
system(audioACommand); 
audioBCommand = sprintf('ffmpeg -i "%s" -q:a 0 -map a audioB.wav', finalOutputB); 
system(audioBCommand); 

% Trim audio to match the video length 
trimAudioACommand = sprintf('ffmpeg -i audioA.wav -ss 0 -t %f -c copy audioA_trimmed.wav', (numFrames - 5) / frameRate); 
system(trimAudioACommand); 
trimAudioBCommand = sprintf('ffmpeg -i audioB.wav -ss 0 -t %f -c copy audioB_trimmed.wav', (numFrames - 5) / frameRate); 
system(trimAudioBCommand); 

% Merge audio back into video 
mergeACommand = sprintf('ffmpeg -i "%s" -i audioA_trimmed.wav -c:v copy -c:a aac -strict experimental "%s"', finalOutputA, finalOutputA); 
system(mergeACommand); 
mergeBCommand = sprintf('ffmpeg -i "%s" -i audioB_trimmed.wav -c:v copy -c:a aac -strict experimental "%s"', finalOutputB, finalOutputB); 
system(mergeBCommand); 

% Clean up temporary audio files 
delete('audioA.wav'); 
delete('audioB.wav'); 
delete('audioA_trimmed.wav'); 
delete('audioB_trimmed.wav'); 

You can read more about “FFmpeg” package here:

https://www.mathworks.com/matlabcentral/fileexchange/42296-ffmpeg-toolbox

Hope this helps!