How To Load Data With Numpy With No Fixed Column Size

How can we load a text file with tab delimited values but with no fixed column size in the way that the missing values are skipped completely ending up with a list/array or whateve

Solution 1:

Maybe you could use pandas

If your file looks like this:

1   2   3   4   5   6
1   2
8.0 9   97  54

Then doing this:

import pandas as pd
pd.read_csv('yourfile.txt',sep='\t')

gives:

123456012NaNNaNNaNNaN1899754NaNNaN

To convert to a numpy array:

np.array(pd.read_csv('yourfile.txt',sep='\t'))


array([[  1.,   2.,  nan,  nan,  nan,  nan],
       [  8.,   9.,  97.,  54.,  nan,  nan]])