Invalid ID during netcdf writing

I am new to netcdf files, and trying to write a file containing geophysical data. I have 600x2158 datapoints, but each column has a set of coordinates, so my final matrix should be 600x2158x2158. This is large, so I wrote it as an ndSparse matrix. My error arises before I even try to write the ndSparse matrix of my data into the netcdf file. When I try to write the vector of y-coordinates (2158 positions) into the netcdf file, I get the following error:

Error using netcdflib

The NetCDF library encountered an error during execution of

'putVarDouble' function - 'Not a valid ID (NC_EBADID)'.

Error in netcdf.putVar (line 88)

netcdflib(funcstr,ncid,varid,varargin{:});

The same happens if I try to write the 600-position z-vector, or the 2158-position x-vector.

If I try to write the ndSparse data matrix, I just get an error saying the format isn't recognised, so I need to find a solution for that too!

Thanks in advance!

Answer 1

[Prasanna]

Hi,

The error you're encountering, 'Not a valid ID (NC_EBADID)', typically indicates a problem with how you're referencing the NetCDF file or variable IDs within your MATLAB code. This error can occur if the file or variable ID is incorrect, not properly opened or created, or if you're attempting to access a file or variable that has already been closed. This error can also occur when using long and unusual filenames in windows when using ‘netcdf’ open. So, a workaround can also be to rename the file, or use Unix machines to work with them.

If your data matrix is too large to convert to a full matrix or to fit into memory, you'll need to consider a different approach. ‘NetCDF’ itself does not directly support sparse matrices. You might need to:

• Write the data in chunks, if possible, using the start, count, stride parameters of ‘netcdf.putVar’.
• Consider storing the sparse matrix data (i.e., the non-zero elements and their indices) as separate variables in the ‘NetCDF’ file and reconstructing the sparse matrix when reading the file.

For more information regarding the error obtained, refer the following. The first link provides a potential answer to the ‘NC_EBADID’ error obtained due to size limit. The second link provides some information on how to troubleshoot the present NetCDF file:

• https://www.mathworks.com/matlabcentral/answers/465635-hdf-error-nc_ehdferr-big-netcdf
• https://www.mathworks.com/matlabcentral/answers/500132-error-reading-netcdf-files

Hope this helps.