Attaching Audio to an .mp4 file

Hello,

I am trying to add a sound burst to a .mp4 file. Currently I am able to correctly insert the burst into the audio of the .mp4 file but I am having trouble 're-attaching' the audio with the burst back to the original .mp4 file. I have found a bunch of resources online to help with this issue and adapted much of their code but I am still having some trouble with the ‘step’ function. After doing some reading I hope that the issue is not that the files have to be a .avi format because I am on a mac and it is having a hard time play .avi files. Also if there is an easier way to do this entire process please do not hesitate to tell me what I can do to attach the new audio back to the original .mp4 file.

Below is the code I have with comments that hopefully illustrate where my problem is occurring.

video_filename = '37vids_balanced/Baby_Babble.mp4';
[Audio, Fs1] = audioread("37vids_balanced/Baby_Babble.mp4"); %Reading in the audio of the original .mp4 file.
vid_length = length(Audio)/Fs1; %Actual length of video in seconds
burst_samp = Fs1 * .05; %How many samples are in the white noise burst 

start_point_sec = 6; %Where in the sound wave of the video audio to insert noise burst needs to be manually entered 
start_point_samp = start_point_sec * Fs1; %Converts the second insertion to sample amount 
[Audio(start_point_samp:(start_point_samp+ burst_samp),:)] = awgn(Audio(start_point_samp: start_point_samp+ burst_samp,:), -90,'measured','dB');
sound(Audio, Fs1); %Check to see if the burst has been inserted correctly 

out_filename = '37vids_balanced_noise/Baby_Babble.mp4'; %Different folder to save the burst noise
videoFReader = VideoReader(video_filename); %Reading in the original .mp4 file 
FR = videoFReader.FrameRate; %Getting the frame rate for the original file 
SamplesPerFrame = floor(Fs1/FR); 
videoFWriter = vision.VideoFileWriter(out_filename, 'FileFormat', 'MPEG4', 'AudioInputPort', true, 'FrameRate', FR); %Creating object to store the new video in with the startle burst included 
videoframes = read(videoFReader); %Reads in all the video frames 
for framenum = 0 : size(videoframes, 4)-1 %The amount of frames in the video - 1 
   videoFrame = videoframes(:,:,:,framenum+1) %Takes one frame at a time 
   this_audio = Audio(framenum*SamplesPerFrame + 1 : min(end, (framenum+1)*SamplesPerFrame), :); %Adds the new adjusted audio from the .mp4 file and adds it to the current frame
   if size(this_audio,1) < SamplesPerFrame
       this_audio(SamplesPerFrame,:) = 0; %If there are more elements in the audio for the frame than the samples for the frame it becomes 0 
   end
   step(videoFWriter, videoFrame, this_audio); %This is where MATLAB is having a problem. It says there are too many input arguments for the step function. 
end
delete(videoFReader);
release(videoFWriter)

The problem I am having is this error:

Error using vision.VideoFileWriter/step
Too many input arguments. Expected 1 (in addition to System object), got 2.

Any help would be very much appreciated

Answer 1

[Abhishek]

Hi @Matthew Gehr

I understand that you are trying to insert a sound burst into the audio of an ‘.mp4’ video file and then re-attach this modified audio back to the original video. Then try to write the video with the new audio using the ‘step’ function in the ‘vision.VideoFileWriter’ object., that throws an error.

This happens because ‘vision.VideoFileWriter’ does not support audio input when writing to the MPEG-4 (.mp4) file format, even if one sets 'AudioInputPort' to true.

Here is what is happening:

When you create the object of ‘vision.VideoFileWriter’ class like this:

videoFWriter = vision.VideoFileWriter('output.mp4', ... 
    'FileFormat', 'MPEG4', ... 
    'AudioInputPort', true, ... 
    'FrameRate', FR); 

MATLAB does not throw an error, but it silently sets ‘AudioInputPort’ back to false. You can confirm this by running the following:

get(videoFWriter) 

You will get an output like this:

  struct with fields:

             Filename: 'output.mp4'
           FileFormat: 'MPEG4'
      AudioCompressor: 'None (uncompressed)'
      VideoCompressor: 'None (uncompressed)'
            FrameRate: 30
        AudioDataType: 'int16'
       FileColorSpace: 'RGB'
              Quality: 75
    CompressionFactor: 10
       AudioInputPort: 0

As you can see, the value for ‘AudioInputPort’ is set to 0(false). This means when later you call the ‘step(videoFWriter,frame,audio)’, it will throw an error because the object doesn’t expect audio at all.

So unfortunately, even if you request audio input for ‘.mp4’, it will not be honoured, and you will hit a mismatch in argument count when calling the ‘step’ method.

As a workaround, you can use the ‘AVI’ format, instead of a ‘.mp4’ format, along with a supported compressor, like ‘MJPEG Compressor’. Please refer the below code to implement the workaround:

videoFWriter = vision.VideoFileWriter('output.avi', ... 
    'FileFormat', 'AVI', ... 
    'AudioInputPort', true, ... 
    'VideoCompressor', 'MJPEG Compressor'); 

Hope this workaround helps. You may refer to the ‘vision.VideoFileWriter’ documentation for more details: https://www.mathworks.com/help/releases/R2021a/vision/ref/vision.videofilewriter-system-object.html