Category Archives: Matlab

[Matlab] Operator Precedence

[In this post I will talk about one subtle difference between the operator precedence in C and Matlab, you can start from the third paragraph if you are familiar with the notion of operator precedence.]

Operator Precedence is really important. I have learnt it the hard way earlier this week. I decided to write about it, hopefully after reading this, you will never fall into the same trap like I did.

For those who are not familiar with the terminology, operator precedence (also called the order of operation) refers a set of rules that defines the order of calculations/procedures should follow in a given mathematical operation. It’s basically what we all learned in first grade. Multiplication has higher priority than addition. 1+2×3 = 7 not 9, because the multiplication operation goes first.

In mathematics, people follow the same rule for operator precedence. However, when it comes to programming, the situation can be a little different. Different programming languages may define operator precedence differently. The specific precedence rule that threw me off involves the comparison operators. In C-style languages, comparison operators are divided into two different categories: (1) the equal comparisons, including equal (==) and not equal (!=)  and (2) the unequal comparisons, including smaller than (<), greater than (>), smaller than or equal to (<=), and greater than or equal to (>=). In C-style languages, the unequal comparisons (i.e. >, <, <=, >=) precedes the equal comparisons (i.e. ==, !=).

Therefore,

1<=2==4>3

means

(1<=2)==(4>3)

and will evaluate to be true.

However, in Matlab, the equal comparisons and the unequal comparisons have the same level of priority. The same expression:

1<=2==4>3

will execute from left to right, as:

((1<=2)==4)>3

and evaluate to be false (note, there are also type conversions going on here).

This can be pretty difficult to catch, especially when you are like me, having this hard wired C-style operator precedence in my head and trying to do some complicated matrix comparison in one line of Matlab code.

So, I have learned my lesson: (1) know the operator precedence. (2) parentheses are my friends. (3) break one line of complicated code into several lines of simpler codes when necessary.

I hope you learned something useful, and see you until next time.

[Matlab Trick] Permute

In the previous post, I introduced the problem of converting a three dimensional matrix (1xmxp) to a two dimensional matrix of size mxp using the squeeze function of matlab. In this post I am introducing a new solution, using function permute.
If you have read the previous post, you can jump directly to solution 2.

The Problem: I was given a three dimensional matrix (1xmxp), I need to convert it to a two dimensional matrix of size pxm.
The Trivial Solution: Nested loop. In addition to the time complexity, the solution make me feel like a dummy.

What I did instead: After 5 mins of google search, I found out a couple of solutions that I found helpful. I will share these with you below.

Let me explain with a concrete example. We need to generate a three dimensional matrix first, say size 1x3x4.

mat(:,:,1)=[1 2 3];
mat(:,:,2)=[4 5 6];
mat(:,:,3)=[7 8 9];
mat(:,:,4)=[10 11 12];

The goal is to convert this into a two dimensional matrix of size 4×3. The new matrix should look like the following:

new_mat =
[1 2 3;
4 5 6;
7 8 9;
10 11 12;]

Solution 1: Squeeze.
This is explained in the previous post.

Solution 2: Permute and Reshape

 new_mat = permute(mat,[3 2 1]);
 % function permute rearranges the dimensions of matrix mat 
 % The order of the rearrangement is specified in the second argument
 % In this example, the third dimension and the first dimension was swapped.

The resulting matrix is just what we wanted :)

[Matlab Trick] Squeeze

The Problem: I was given a three dimensional matrix (1xmxp), I need to convert it to a two dimensional matrix of size pxm.
The Trivial Solution: Nested loop. In addition to the time complexity, the solution make me feel like a dummy.

What I did instead: After 5 mins of google search, I found out a couple of solutions that I found helpful. I will share these with you below.

Let me explain with a concrete example. We need to generate a three dimensional matrix first, say size 1x3x4.

mat(:,:,1)=[1 2 3];
mat(:,:,2)=[4 5 6];
mat(:,:,3)=[7 8 9];
mat(:,:,4)=[10 11 12];

The goal is to convert this into a two dimensional matrix of size 4×3. The new matrix should look like the following:

new_mat =
[1 2 3;
4 5 6;
7 8 9;
10 11 12;]

Solution 1: Squeeze.

t = squeeze(mat) 
% the squeeze command removes the singleton dimensions
% a singleton dimension is a dimension of size 1 

t is a 3×4 matrix shown below:

t =
[1 4 7 10;
 2 5 8 11;
 3 6 9 12;]

Now we simply need to transpose it to get to our desired matrix.

new_mat = transpose(t)

One thing worth-noticing about the squeeze command: it simply removes the singleton dimensions. Applying squeeze function to matrix of size 1x3x4 (mat1 in the example below), or size 3x4x1 (mat2 in the example below), or size 3x1x4 will all result in matrix 3×4. This is illustrated below:

mat1(:,:,1)=[1 2 3];
mat1(:,:,2)=[4 5 6];
mat1(:,:,3)=[7 8 9];
mat1(:,:,4)=[10 11 12];

mat2(:,:,1)=[1; 2; 3];
mat2(:,:,2)=[4; 5; 6];
mat2(:,:,3)=[7; 8; 9];
mat2(:,:,4)=[10; 11; 12];

t1 = squeeze(mat1);
t2 = squeeze(mat2);

t1 and t2 are both 3×4, and they are exactly the same. I don’t know about everyone else, but I did not expect this to begin with.

Solution 2: Permute and Reshape
I will talk about this solution in my next blog post! see you soonish!

[Matlab] Import big file

The problem: I have this big txt file (well, relatively big, a little more than 200Mb) that I need to import into matlab and process.This file contain a table with characters and integers. The first row of the table is the column header. For my purpose, I only need data from specific columns (about 20 columns).

My initial solution: Since the file is not super huge and I do have a pretty decent machine. I just went with the importdata function. since it handles delimited txt file that contain string and integers pretty well in general.

The result: It took a long time, probably about 20 mins. And the file was not imported completely. The Header row was correctly imported, but only 4 rows of data were imported (I am still confused about why this is happening, and why Matlab did not even throw an error). But, I did learn something useful. It turned out that I have about 7000000 columns (according to the size of my header row).

My thought: I am sure you are thinking about the same thing as well, If I only need about 20 columns, it does not make much sense to read the whole file in. And apparently, reading the whole file in (at least using importdata function) has failed me.

The new plan:
(1) import the header, figure out which columns need to be imported
(2) import the content of the table line by line, only store the columns needed.

The code:
Alright, so here is how I did it. if you want to try this yourself, you can copy (start copying from the line that starts with YKZ) the following sample data into a text editor and save a txt file (in my example code, the sample data is saved in a txt file named text_file.txt) and see how it goes.

=============== [Sample data (space delimited)] ===============

YKZ Timestamp Temp Humidity Wind Weather
06-Sep-2013 01:00:00 6.6 89 4 clear
06-Sep-2013 05:00:00 5.9 95 1 clear
06-Sep-2013 09:00:00 15.6 51 5 mainly_clear
06-Sep-2013 13:00:00 19.6 37 10 mainly_clear
06-Sep-2013 17:00:00 22.4 41 9 mostly_cloudy
06-Sep-2013 21:00:00 17.3 67 7 mainly_clear
09-Sep-2013 01:00:00 15.2 91 8 clear
09-Sep-2013 05:00:00 19.1 94 7 n/a

==== [code (copy the code below to your matlab editor and run it directly) ] ======

% (1) import the header, figure out which columns need to be imported
clear all;
selected_col = {‘YKZ’,’Temp’,’Wind’};% specify columns need to be extracted by column names
fid = fopen(‘text_file.txt’);% create a file identifier. This does not load the content of the file
tline = fgetl(fid);% fgetl return the next line of the file identified by the fid. This function return a character string.
header = strsplit(tline,’ ‘); % now I parse the returned string using function strsplit, as a result I get the row header of the table. the first argument of strsplit is the string to be parsed, the second argument is the delimiter. Notice that the delimiter is optional, if a delimiter is not specified it uses white spaces.
idx = ismember(header,selected_col); % now we create a logical array to indicate the columns to be extracted.

% (2) import the content of the table line by line, only store the columns needed.
tdata={}; % initiate variable to store data
while ~feof(fid) % do the following step until end-of-file is reached
    tline = fgetl(fid); % every time fgetl is called, it reads in a new line from the file.
    delim_line = strsplit(tline,’ ‘); % parse the current line
    tdata(end+1,:)=delim_line(idx); % select data in columns specified by logical variable idx
end
fclose(fid); % close the original file

% (3) clean up
% Now, the extracted data are stored in cell array tdata (8×3 cell array)
% If you type tdata{1,2} in your command window, you should get:
ans =
6.6
% It is really important to recognize that 6.6 is not a number here.
% Now type class(tdata{1,2}) in your command window, you get:
ans =
char
% 6.6 here is of type char, so are all the other “numbers” in variable tdata. It is quite easy to convert them to numbers. Just one line of code.
num_data=cell2mat(cellfun(@(s) str2double(s), tdata(:,2:end), ‘UniformOutput’,false));
% the cellfun function apply str2double function to every cell in tdata(:,2:end) which convert them to type double, and function cell2mat convert the resulting cell array to matrix.

% (4) ta da, here is the data
col_header = header(idx); % column header
row_header = tdata(:,1); % row header
data = num_data; % here is the table content
clearvars -except col_header row_header data % clear all other intermediate variables

I hope you find this post helpful and feel free to leave a comment :)