'HDF error (NC_EHDFERR)' during execution of 'close' function using matlab.int​ernal.imag​esci.netcd​flib

One of our users has some code which unexpectedly fails with error messages and creates a corrupted NetCDF file in MATLAB versions R2020b and newer.

This is the code:

outfile='test.nc';
if exist(outfile,'file')
  delete(outfile);
end
nrow=128; ncol=128;
nccreate(outfile,'shape','dimensions',{'lon',nrow,'lat',ncol,'time',inf},'chunksize',[nrow,ncol,1],'datatype','int16','fillvalue',0,'format','netcdf4','deflatelevel',9);
nccreate(outfile,'axis', 'dimensions',{'lon',nrow,'lat',ncol,'time',inf},'chunksize',[nrow,ncol,1],'datatype','double','fillvalue',0,'format','netcdf4','deflatelevel',9);
nccreate(outfile,'lfloc','dimensions',{'lon',nrow,'lat',ncol,'time',inf},'chunksize',[nrow,ncol,1],'datatype','int16','fillvalue',0,'format','netcdf4','deflatelevel',9);
nccreate(outfile,'islnd','dimensions',{'lon',nrow,'lat',ncol},'chunksize',[nrow,ncol],'datatype','int16','fillvalue',0,'format','netcdf4','deflatelevel',9);
nccreate(outfile,'lon','dimensions',{'lon',nrow,'lat',ncol});
nccreate(outfile,'lat','dimensions',{'lon',nrow,'lat',ncol});
islnd=zeros(nrow,ncol);
islnd(:,:)=1;
nccreate(outfile,'time','dimensions',{'time',inf});
ncwrite(outfile,'islnd',int16(islnd));

These are the error messages:

Warning: The following error was caught while executing 'onCleanup' class destructor:
Error using matlab.internal.imagesci.netcdflib
The NetCDF library encountered an error during execution of 'close' function - 'HDF error
(NC_EHDFERR)'.

Error in netcdf.close (line 16)
matlab.internal.imagesci.netcdflib('close', ncid);

Error in internal.matlab.imagesci.nc/close (line 145)
                netcdf.close(rootid);

Error in ncwrite>@()ncObj.close() (line 84)
cleanUp = onCleanup(@()ncObj.close());

Error in onCleanup/delete (line 80)
            obj.task();

Error in ncwrite (line 61)
if nargin > 0

Error in test (line 15)
ncwrite(outfile,'islnd',int16(islnd)); 
> In ncwrite (line 61)
In test (line 15) 
Error using matlab.internal.imagesci.netcdflib
The NetCDF library encountered an error during execution of 'inqVar' function - 'HDF
error (NC_EHDFERR)'.

Error in netcdf.inqVar (line 26)
[varname,xtype,dimids,natts] = matlab.internal.imagesci.netcdflib('inqVar', ncid, varid);

Error in internal.matlab.imagesci.nc/varInfo (line 1372)
                    netcdf.inqVar(gid,varid);

Error in internal.matlab.imagesci.nc/write (line 754)
            varinfo  = this.varInfo(gid, varid);

Error in ncwrite (line 87)
ncObj.write(varName, varData, start, stride);

Error in test (line 15)
ncwrite(outfile,'islnd',int16(islnd))

However, the same code succeeds and creates a normal NetCDF file in MATLAB versions R2020a and earlier. Any suggestions as to why the error starts in MATLAB R2020b and how to resolve it?

Answer 1

[Jasvin]

Hi Hans,

I have been investigating related bug reports for quite some time and I think I have the possible source of this problem.

It is most likely because of calling "nccreate" on a variable name that is also a dimension name. So, in this case, there are three such instances from the sample code atleast, i.e., "lat", "lon", "time". When a dimension shares the same name as a previously defined variable for NetCDF4-classic files, an error code-101 (EHDFERROR = Error at HDF5 Layer) is thrown during the final close. If "nc_close" is called again then a "SegV" occurs. This did not happen with NetCDF v4.3.3.1 but with 4.6.1 and later versions of NetCDF.

This problem is supposed to have first been reported in R2019a because of changes that were made to NetCDF which is a third-party library.

A possible fix mentioned on the following NetCDF GitHub issue is to define the dimension before creating the variable:

https://github.com/Unidata/netcdf-c/issues/1772

You could also try reproducing the same issue on R2021a. If the issue persists then I would recommend you create a technical support case by contacting MathWorks Technical Support using the following link:

https://www.mathworks.com/support/contact_us.html

Hope this helps!

[Hans Vahlenkamp]

Your analysis and proposed solution is exactly what we needed. Modifying the code to define the dimensions before the variables allowed creating a correct NetCDF file with no errors. We found that this issue started in MATLAB R2020b apparently due to its embedded NetCDF being upgraded to 4.7.x; the older MATLAB versions have embedded NetCDF 4.6.x and older.

Thanks!

Answer 2

[Charles Lee]

Hi Hans,

It might be a few months late, but it seems we encountered the same issue. When I was using R2020a, I could write to a netCDF file using the following function:

% ET attribute
nccreate(outfile,'ET','datatype','double','Dimensions',{'lat' row 'lon' col},'DeflateLevel',5);
ncwriteatt(outfile,'ET','unit','mm d-1');
ncwriteatt(outfile,'ET','long_name','Total Evapotranspiration');
% lon attribute
nccreate(outfile,'lon','datatype','double','Dimensions',{'lon'},'DeflateLevel',5);
ncwriteatt(outfile,'lon','units','degrees_east');
ncwriteatt(outfile,'lon','long_name','longitude');
% lat attribute
nccreate(outfile,'lat','datatype','double','Dimensions',{'lat'},'DeflateLevel',5);
ncwriteatt(outfile,'lat','units','degrees_north');
ncwriteatt(outfile,'lat','long_name','latitude');
% write-in
ncwrite(outfile,'ET',ET);
ncwrite(outfile,'lat',lat);
ncwrite(outfile,'lon',lon);

However, upon switching to the R2023b version, I encountered an error similar to yours. I found that the problem was resolved when I made changes to the following format:

% lon attribute
nccreate(outfile,'lon','datatype','double','Dimensions',{'lon',col},'DeflateLevel',5);
ncwriteatt(outfile,'lon','units','degrees_east');
ncwriteatt(outfile,'lon','long_name','longitude');
ncwrite(outfile,'lon',lon);
% lat attribute
nccreate(outfile,'lat','datatype','double','Dimensions',{'lat',row},'DeflateLevel',5);
ncwriteatt(outfile,'lat','units','degrees_north');
ncwriteatt(outfile,'lat','long_name','latitude');
ncwrite(outfile,'lat',lat);
% ET attribute
nccreate(outfile,'ET','datatype','double','Dimensions',{'lat',row,'lon',col},'DeflateLevel',5);
ncwriteatt(outfile,'ET','unit','mm d-1');
ncwriteatt(outfile,'ET','long_name','Total Evapotranspiration');
ncwrite(outfile,'ET',ET);

To summarize, the key change was this: I first defined the 'lat' and 'lon' variables along with their dimensions because these two variables are related to the dimensions of the 'ET' variable.

Hope this still proves helpful to you!