Arragement of vectors and matrix in text file
Show older comments
I have 2D array of matrix(in excel or text file). This file contains GP height for my study area. 1st column of this file is latitude values and 1st row of this file is longitude values. Now corresponding to these latitude and longitude values there are GP_height values.
Please see that, from (2 row , 2colum) to (2,17) are values of GP_Height on the geographic position whose latitude are fixed (as 1st column) and longitude vary (along row values).
There are such trend in the each row and column.Actually this is 2D array.
Now i want to make a file (text or excel which will suited for the helper of my problem) in which 1st column consist on latitude value(which was fixed in my above explanations) corresponding to each longitude values in the 2nd column and third column will be their corresponding GP_Height.
This file( in excel and text) has been attach with new modifications.
For the batter understanding of my required output see my sample output my comment below.
Thanks in advance for assistance
2 Comments
Muhammad Usman Saleem
on 14 Sep 2015
Edited: Muhammad Usman Saleem
on 15 Sep 2015
Muhammad Usman Saleem
on 14 Sep 2015
Edited: Muhammad Usman Saleem
on 15 Sep 2015
Accepted Answer
It's difficult to understand how your data must be processed because the various parts of your example do not match (values, sizes, etc), and the explanations are a bit convoluted.
So let's start with an example based on what I understood and we can discuss further from there:
% - Read source file.
fSpec = ['GPHeight_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
data = textscan( fileread( 'Forward file.txt' ), fSpec ) ;
% - Get pressure from lookup table of pressures and first z+1 (indexing in
% MATLAB starts at 1). Update the code if z can vary within an input file.
pressures = 1000 : -100 : 100 ; % To adjust to your setting.
pressure = pressures(data{1}(1)+1) ;
% - Extract/aggregate data.
latId = data{2} + 1 ; % x + 1
GP = horzcat( data{3:end} ) ;
% - Define latitudes (based on ID? can vary among files?) and
% longitudes (fixed?).
lat = linspace( 22.5, 42.5, numel( latId )) ; % To adjust to your setting.
lon = linspace( 60.5, 78.5, size( GP, 2 )) ; % To adjust to your setting.
% - Build arrays of lon/lat which correspond to GP.
[LON, LAT] = meshgrid( lon, lat(latId) ) ;
% - Build output data: start by building 3 columns giving priority
% to same lats, and then transpose so it is well shaped for FPRINTF.
data = [reshape(LAT.',[],1), reshape(LON.',[],1), reshape(GP.',[],1)].' ;
% - Export: build file name based on pressure, 'prepend' header.
fId = fopen( sprintf( 'Output_%dhpa.txt', pressure ), 'w' ) ;
fprintf( fId, 'latitude\tlongitude\tGP_value\n' ) ;
fprintf( fId, '%.3f\t%.3f\t%.3f\n', data(:) ) ;
fclose( fId ) ;
See if you understand, and you will certainly have to modify the order/length/values of lon/lat, the direction of data, etc, but this is a starting point.
35 Comments
Muhammad Usman Saleem
on 15 Sep 2015
This new version of your question is much simpler, see below:
pressure = 1000 ;
% - Read file, get longitudes and array of raw data (lat and gp).
fId = fopen( 'Data.txt', 'r' ) ;
lon = sscanf( fgetl( fId ), '%f' ) ;
raw = fscanf( fId, '%f%*c' ) ;
fclose( fId ) ;
% - Separate raw data into latitudes and GP values.
raw = reshape( raw, numel(lon)+1, [] ).' ;
lat = raw(:,1) ;
GP = raw(:,2:end) ;
% - Build arrays of lon/lat which match the structure of array GP.
[LON, LAT] = meshgrid( lon, lat ) ;
% - Build output data: start by building 3 columns giving priority
% to same lats, and then transpose so it is well shaped for FPRINTF.
data = [reshape(LAT.',[],1), reshape(LON.',[],1), reshape(GP.',[],1)].' ;
% - Export: build file name based on pressure, 'prepend' header.
fId = fopen( sprintf( 'Output2_%dhpa.txt', pressure ), 'w' ) ;
fprintf( fId, 'latitude\tlongitude\tGP_value\n' ) ;
fprintf( fId, '%.3f\t%.3f\t%.3f\n', data(:) ) ;
fclose( fId ) ;
Let me know if you have any question.
Muhammad Usman Saleem
on 16 Sep 2015
Muhammad Usman Saleem
on 16 Sep 2015
Edited: Muhammad Usman Saleem
on 16 Sep 2015
Muhammad Usman Saleem
on 16 Sep 2015
Muhammad Usman Saleem
on 16 Sep 2015
Edited: Muhammad Usman Saleem
on 16 Sep 2015
But the problem is that, this data is not arrange into three columns in text file.
It is, but you have to adapt the output to your system so it is displayed and/or interpreted correctly. See the \n in the format specs of the last calls to FPRINTF? This means new line. On some systems you will need to add a carriage return \r because the new line only is not displayed properly. This typically happens when you try to open text files using the basic Windows Notepad (Notepad++ on the contrary is able to figure out which type of file it is). If you really need a text file as output, update the following lines:
fprintf( fId, 'latitude\tlongitude\tGP_value\r\n' ) ;
fprintf( fId, '%.3f\t%.3f\t%.3f\r\n', data(:) ) ;
But you can also decide to export to XLS(X) using XLSWRITE.
Here is a solution which combines my first and your new information, and shows how to export to XLSX. See if it works, you may have to adapt it a little.
% - Read source file.
fSpec = ['GPHeight_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
data = textscan( fileread( 'Data1.txt' ), fSpec ) ;
% - Extract/gather all pId=x and gp values.
pId = data{1} ;
GP = horzcat( data{3:end} ) ;
% - Build arrays of lon/lat which correspond to GP.
[lon, lat] = meshgrid( 60:79, 22:39 ) ;
% - Iterate through pressure IDs and process.
for p = 0 : 23
% Get relevant block of GP (the one thta corresponds to current p).
gp = GP(pId==p,:) ;
% Build 3 columns data array and cell array for exporting to XLSX.
data = [reshape(lat.',[],1), reshape(lon.',[],1), reshape(gp.',[],1)] ;
dataCell = [{'latitude', 'longitude', 'GP_value'}; num2cell( data )] ;
% Verbose/export.
fprintf( 'Export for pressure ID %d.\n', p ) ;
xlswrite( sprintf( 'Output_%d.xlsx', p ), dataCell ) ;
end
Muhammad Usman Saleem
on 16 Sep 2015
I showed you above how to export to text file. If you don't understand, maybe it's better if I explain it to you rather than just updating the code(?) If you are in a hurry let me know and I will just perform the update. Otherwise I am happy to spend a little more time on my answer and explain what you don't understand so you get something more than just code out of this thread.
EDIT: I made the update quickly for you so you have the code, but let me know if you need explanations. I don't recommend using the pressure in the output file name, because you have non-integer values that would introduce a second . in file names.
pressures = [1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, ...
100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1] ;
% - Read source file.
fSpec = ['GPHeight_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
data = textscan( fileread( 'Data1.txt' ), fSpec ) ;
% - Extract/gather all x and gp values.
X = data{1} ;
GP = horzcat( data{3:end} ) ;
% - Build arrays of lon/lat which correspond to GP.
[lon, lat] = meshgrid( 60:79, 22:39 ) ;
% - Iterate through pressure IDs and process.
for x = 0 : 23
% Get relevant block of GP (the one thta corresponds to current p).
gp = GP(X==x,:) ;
% Build 3 columns data array.
data = [reshape(lat.',[],1), reshape(lon.',[],1), reshape(gp.',[],1)].' ;
% Verbose.
fprintf( 'Export for pressure ID %d -> p=%.1fhpa.\n', x, pressures(x+1) ) ;
% Export.
fId = fopen( sprintf( 'Output_%d.txt', x), 'w' ) ;
fprintf( fId, 'latitude\tlongitude\tGP_value\r\n' ) ;
fprintf( fId, '%.3f\t%.3f\t%.3f\r\n', data(:) ) ;
fclose( fId ) ;
end
Muhammad Usman Saleem
on 17 Sep 2015
Edited: Muhammad Usman Saleem
on 17 Sep 2015
Here is the code split in parts with explanations
pressures = [1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1] ;
We pre-define a vector of pressures. There is no need to do it here at the beginning, but as it is fixed, it can be defined early, in an initialization phase, so we can use it if needed. The only thing that matters is that it is fixed, so we should not define it in the loop below.
% - Read source file.
fSpec = ['GPHeight_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
We want to read the file content. One way to do it is to use TEXTSCAN and define a string that is called format spec (see the doc for the details) which defines the format of the data in the file content (more precisely, it defines the way we read/interpret the content). This format spec can contain special codes that define e.g. the position and type of numbers that we want to extract. The format spec for reading a line of your input file such as
GPHeight_A[0][13], 27, 34, 38, 37, ...
where there are 20 comma-separated numbers after the ] is the string
'GPHeight_A[%f][%f], %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f'
where %f tells TEXSCAN that it must read/convert/output a floating point number. We could have built it by hand and declared
fSpec = 'GPHeight_A[%f][%f], %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f, %f' ;
as you are always reading 20 numbers. Yet, we sometimes just prefer concatenating a given number of times a single chunk of format spec (here 20 times ', %f') with REPMAT, rather than typing them all by hand. This makes the solution flexible in the sense that you can reuse the code almost as it is in another project where you have to read e.g. 400 numbers just by changing 20 for 400, instead of having to add 380 more terms by hand.
data = textscan( fileread( 'Data1.txt' ), fSpec ) ;
This line reads the file content and passes it as well as the format spec that we just discussed to TEXTSCAN. With these two inputs, TEXTSCAN extracts numbers and outputs a cell array of columns (the first is what you call x, the second is what you call z, the third is the first column of GP values, etc).
In what follows we extract the first column which is a vector with all values of x, and we pack columns 3 to the end in a array of all GP values. We don't use z as latitudes are fixed.
% - Extract/gather all x and gp values.
X = data{1} ;
GP = horzcat( data{3:end} ) ;
Now comes the smart part. When you have an array of values which correspond to latitudes and longitudes, and you don't know how to repeat longitudes and latitudes, and how to unfold the array of values so they match, an easy way to proceed is to build arrays of longitudes and latitudes with the same structure as the array of values, and to unfold them all the same way (so they correspond by construction). The call to MESHGRID builds these arrays of longitudes and latitudes. If you display them in your command window, you will see that they match the structure of an array of GP values for one pressure level.
% - Build arrays of lon/lat which correspond to GP.
[lon, lat] = meshgrid( 60:79, 22:39 ) ;
At this stage we have everything we need to extract and reshape data. We iterate through all pressure levels
% - Iterate through pressure IDs and process.
for x = 0 : 23
% Get relevant block of GP (the one thta corresponds to current p).
gp = GP(X==x,:) ;
Here X is the vector of all pressure IDs, and x is a scalar. When x equals 10 for example, the line gp = GP(X==x,:) extracts the block of all GP values that corresponds to the 11th pressure.
Now we have three arrays: lon, lat, gp, but you need to export them in columns and not as arrays. We therefore reshape/unfold them to express them in column. The last transpose is a detail at this stage; the purpose is to give the data structure an orientation that is well suited for the last call to FPRINTF.
% Build 3 columns data array.
data = [reshape(lat.',[],1), reshape(lon.',[],1), reshape(gp.',[],1)].' ;
% Verbose.
fprintf( 'Export for pressure ID %d -> p=%.1fhpa.\n', x, pressures(x+1) ) ;
Now the last bit. We open a text file for writing, write in this text file the column headers, and write the data array in one shot. Then we close the file. You see in the second call to FPRINTF that there is a format spec string again, which defines the way elements of the data array must be represented in the file: %.3f\t means floating point with three digits after the decimal point, followed by a tabulation.
% Export.
fId = fopen( sprintf( 'Output_%d.txt', x), 'w' ) ;
fprintf( fId, 'latitude\tlongitude\tGP_value\r\n' ) ;
fprintf( fId, '%.3f\t%.3f\t%.3f\r\n', data(:) ) ;
fclose( fId ) ;
end
If you need more information about format specs, you can find it in the doc of FPRINTF and/or TEXTSCAN
doc fprintf
doc textscan
The last point about not using the pressure in the file name is that you have this 1.5hpa pressure which would lead to a file name with a double point
Output_1.5hpa.txt
This may be troublesome for some applications that may not allow/manage . in file names except for separating the base name from the extension.
Hope it helps.
Muhammad Usman Saleem
on 18 Sep 2015
Cedric
on 19 Sep 2015
You're most welcome.
Muhammad Usman Saleem
on 14 Jan 2016
Cedric
on 18 Jan 2016
Dear Muhammad Usman Saleem,
Please copy/paste the error message, and attach the code/data if they changed.
Thanks,
Cedric
Muhammad Usman Saleem
on 19 Jan 2016
Edited: Muhammad Usman Saleem
on 19 Jan 2016
Cedric
on 19 Jan 2016
The problem comes from the fact that you are not working with the same grid. Formerly, you had 20 longitudes and 18 latitudes if I remember well, and now you have 19 longitudes and 17 latitudes.
You have to adapt the line
[lon, lat] = meshgrid( 60:79, 22:39 ) ;
to your new grid.
Muhammad Usman Saleem
on 22 Jan 2016
Cedric
on 22 Jan 2016
/doc xlswrite
Muhammad Usman Saleem
on 26 Mar 2016
Edited: Muhammad Usman Saleem
on 2 Apr 2016
Muhammad Usman Saleem
on 31 Mar 2016
Muhammad Usman Saleem
on 1 Apr 2016
Edited: Muhammad Usman Saleem
on 2 Apr 2016
Walter Roberson
on 2 Apr 2016
You have
gp = GP(X==x,:) ;
% Build 3 columns data array.
data = [reshape(lat.',[],1), reshape(lon.',[],1), reshape(gp.',[],1)].' ;
lat and lon are the same size as each other, and their size is fixed, not varying with the data. They are 9 x 9, so after the reshape() there would be 81 values for each of the first and second columns.
gp is determined by selecting a portion of the GP data. How much? It has multiple columns, and going back to the input format of the most recent file structure we see that it is columns 3 to end where the third column is one of the [] enclosed ones which is followed by 8 numeric values, so there are 9 columns including column 3. To match the 81 total entries we need for the column, we would need to be selecting 81/9 = 9 rows. But when we look at the data, there are clearly exactly 8 rows for each of the x values 0 through 16, leading to 8 * 9 = 72 entries for the third column that needs to have 81 entries.
You could have easily found this with a "dbstop if error" and checking the number of elements in lat and long and gp.
You need to fix the values that you meshgrid() over. And considering the way you transpose the results you get out of meshgrid() I recommend that you use ndgrid() with no transpose instead of using meshgrid()
Muhammad Usman Saleem
on 2 Apr 2016
Walter Roberson
on 2 Apr 2016
"Please reply timely"
I sleep. I cook. I eat. I clean house. I do my taxes.
Muhammad Usman Saleem
on 4 Apr 2016
Walter Roberson
on 4 Apr 2016
Put in a breakpoint and check size(data) . It could be that your gp is empty.
Muhammad Usman Saleem
on 4 Apr 2016
Walter Roberson
on 4 Apr 2016
Put a breakpoint in at the line
data = [reshape(lat,[],1), reshape(lon,[],1), reshape(gp,[],1)].' ;
and run your code. When it stops, show
size(LAT)
size(LON)
size(gp)
then single-step and show
size(data)
Muhammad Usman Saleem
on 4 Apr 2016
Edited: Muhammad Usman Saleem
on 4 Apr 2016
Walter Roberson
on 6 Apr 2016
What it means is that your line
gp = GP(X==x,:) ;
is finding no places that X==x .
Look at the first column of your file. Every one of your lines starts with [0] and you have X = data{1} so your X is going to be a vector that contains only 0's. That will match everything on the first iteration of "for x = 0 : 16", selecting all lines. But then when you reach the "x = 1" case, there are no 1's in X at all, so gp will come out empty, and that is going to mess up the sizes of the rest of your calculations.
Why is it that all your lines start with "[0]" ?
I am going to suggest a quick-and-dirty repair to your code, which I have pointed out is also using the wrong starting column for defining GP. You can fix a couple of your problems at the same time if you change
fSpec = ['[%f][%f][%f]', repmat( ', %f', 1, 8 )] ;
to
fSpec = ['[%*f][%f][%f]', repmat( ', %f', 1, 8 )] ;
The "%*f" means that a number must be found there (the 0 of [0]) but that the number will then be ignored, so the data returned by the textscan will start with the second column of your input -- the column that varies from 0 to 16.
Muhammad Usman Saleem
on 6 Apr 2016
Muhammad Usman Saleem
on 6 Apr 2016
Edited: Walter Roberson
on 6 Apr 2016
More Answers (0)
Categories
Find more on Standard File Formats in Help Center and File Exchange
Products
on 14 Sep 2015
on 6 Apr 2016
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
