Convert Signal into a image for deep CNN input

Hi,

I'm doing a work on creating a deep CNN network that has a multi input of images. In particoular electrograms in time domain, sswt, hht, cwt.

How can i convert a 500x1 signal into a 100x100 matrix that will become an image with significant info input for the CNN?

I thought something like this

M=zeros(100,100);
y=floor(mean(reshape(sig,[5 100]))); %returns the mean of 5 elements along the vector of the signal

for i=1:size(M,1)
    for j=1:size(M,2)
        M(i,j)=y(i);
    end
end
figure
image(M)

f=getframe;
p=f.cdata;
I=imresize(p,[100 100]);
imwrite(I,'filename.jpeg')
%ready to be saved

%or in a easier way, but i don't know if this will have useful info for the
%CNN
plot(M)
f=getframe;
p=f.cdata;
I=imresize(p,[100 100]);
imwrite(I,'filename.jpeg')

These are not the saved images, all of the parameters of them, like colormap, caxis and etc will be tweaked later.

Thanks in advance.

Answer 1

[Abhishek]

Hi Christian,

I understand you want to know how to convert an ECG signal into a 2D image to be used as input for a CNN model.

However, since you have not provided information regarding the signal or database you are using and intended objective for the model, I assume it's likely to deal with the cardiac dynamics of ECG, such as Arrhythmia, Normal Sinus or any other rhythm. For CNN to learn the graphical deflections, or any abnormal parameters, the best option would be sample ECG for a cycle (for example, between a R-R interval or a QRS complex). Then you can either save it as a time series or convert it to spectrogram for frequency analysis, depending upon your use case.

You can save the plots programmatically either by using exportgraphics or first getimage and then using imsave.

You can refer to following examples to read about popular datasets used for similar applications: -

1. Wavelet Time Scattering for ECG Signal Classification - MATLAB & Simulink Example (mathworks.com)
2. Classify Time Series Using Wavelet Analysis and Deep Learning - MATLAB & Simulink Example (mathworks.com)

[CHRISTIAN COSSU]

Thank you for your answer,

This is how the dataset is composed:

The works you have shown are a little different i suppose, these signals are electrograms, registered in an invasive way into the heart chambers with an electrocatetere(lead), the point of the work is to provide the deep CNN info images about 4 different rappresentations of the signals, like this:

I've created a function that saves these rappresentations into [227 227 3] images, these will be the input to:

which is a network composed by 4 alexnets in serie added together(the layer concat has been substitued with an add layer), because i thought it would be too difficoult to do a convolution in 12 layers of images.

However the point of the work is regression, not classification, i'm trying to develop it in order tho have the network return the onset of the ventricular tachycardia(which is the annotation of the electrophysiologist in the column 5)

What do you think about this?

[Abhishek]

Prima facia, this should work with the later image type that you posted in the question.

However, given the size of input data, use of datastores would be suggested to reduce complexity of the CNN. There are built-in datastores for deep learning applications which are well suitable for regression tasks. You can use 2-D image sequence and can also specify parallel or multi-GPU training.

Please go through the following documentation to know more about datastores:-

Datastores for Deep Learning - MATLAB & Simulink (mathworks.com)

[CHRISTIAN COSSU]

Thank you,

I've already seen the types of datastore and i've tried to compose a solution

trainImgsALEX_SQ_cwt = imageDatastore(strcat(directory_train,'\alexsqueeze',p,trasform(1)),'IncludeSubfolders',true,'LabelSource','foldernames');
    trainImgsALEX_SQ_hht = imageDatastore(strcat(directory_train,'\alexsqueeze',p,trasform(2)),'IncludeSubfolders',true,'LabelSource','foldernames');
    trainImgsALEX_SQ_sswt = imageDatastore(strcat(directory_train,'\alexsqueeze',p,trasform(3)),'IncludeSubfolders',true,'LabelSource','foldernames');
    trainImgsALEX_SQ_time = imageDatastore(strcat(directory_train,'\alexsqueeze',p,trasform(4)),'IncludeSubfolders',true,'LabelSource','foldernames');

    array_labels=arrayDatastore(annotations_train(:,k));
    trainImgsALEX_SQ = combine(trainImgsALEX_SQ_cwt,trainImgsALEX_SQ_hht,trainImgsALEX_SQ_sswt,trainImgsALEX_SQ_time,array_labels);
    
    
    testImgsALEX_SQ_cwt = imageDatastore(strcat(directory_test,'\alexsqueeze',p,trasform(1)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');
    testImgsALEX_SQ_hht = imageDatastore(strcat(directory_test,'\alexsqueeze',p,trasform(2)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');
    testImgsALEX_SQ_sswt = imageDatastore(strcat(directory_test,'\alexsqueeze',p,trasform(3)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');
    testImgsALEX_SQ_time = imageDatastore(strcat(directory_test,'\alexsqueeze',p,trasform(4)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');

    testImgsALEX_SQ=combine(testImgsALEX_SQ_cwt,testImgsALEX_SQ_hht,testImgsALEX_SQ_sswt,testImgsALEX_SQ_time);

    valImgsALEX_SQ_cwt = imageDatastore(strcat(directory_val,'\alexsqueeze',p,trasform(1)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');
    valImgsALEX_SQ_hht = imageDatastore(strcat(directory_val,'\alexsqueeze',p,trasform(1)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');
    valImgsALEX_SQ_sswt = imageDatastore(strcat(directory_val,'\alexsqueeze',p,trasform(1)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');
    valImgsALEX_SQ_time = imageDatastore(strcat(directory_val,'\alexsqueeze',p,trasform(1)),'IncludeSubfolders',true,'LabelSource','foldernames','FileExtensions','.jpg');
    
    valImgsALEX_SQ=combine(valImgsALEX_SQ_cwt,valImgsALEX_SQ_hht,valImgsALEX_SQ_sswt,valImgsALEX_SQ_time);

This code is implemented in a for that goes from 1:10 for the Kfold cross validation, however the datastore that I'm creating is CombinedDataStore.

I'm facing some problems tho, like these:

And it often says that I can't do regression work with input images, how can i solve these type of problems?

If you want i can send you the scripts and the functions that I'm using.

The main problem is, how can i have the network have 4 input images(the ones above), combinedatastore works for this type of task?

The other problem is, how can i have it return the regression(the onset number)?