How To Split The Data Among The Multiple Groups In Hdf5 File?

I have one some data which looks something like this: Generated by trjconv : P/L=1/400 t= 0.00000 11214 1P1 aP1 1 80.48 35.36 4.25 2P1 aP1 2 37.45 3.

Solution 1:

Your code has a number of small errors when reading and converting the "footer" line. I modified the code and got it working....but not sure if it does exactly what you want. I used the same group and dataset definitions. So, the footer data is written to this data set:

/particles/lipids/positions/box/boundary/box_size

That comes from the following group and dataset definitions:

particles_grp = hdf.require_group('particles/lipids/positions')
box_grp = particles_grp.create_group('box')
bound_grp = box_grp.create_group('boundary')
edge_data = bound_grp.create_dataset('box_size'....

Corrections are required in several places: First, you need to change the definition of parse1 to match the 3 fields.

# Format for footer# FROM:fmtstring1 = '1s 1s 5s 7s 7s 7s'# TO:fmtstring1 = '10s 10s 10s'

Next, you need to modify where and how the box_size dataset is created. You need to create it like the others: as an extensible dataset (maxshape=() parameter) ABOVE while True: loop. This is what I did:

edge_ds_step = edge_grp.create_dataset('step', dtype=np.uint64, shape=(0,), maxshape=(None,), compression='gzip', shuffle=True)
# Create empty 'box_size' dataset hereedge_data = bound_grp.create_dataset('box_size', dtype=np.float32, shape=(0,3), maxshape=(None,3), compression='gzip', shuffle=True)

Finally, here is the modified code to:

1. Parse the footer string to a tuple,

2. Map the tuple to an np.array of floats, shape=(1,3),

3. Resize the dataset and finally

4. Load the array into the dataset.

   footer = parse1( f.readline().encode('utf-8') )
   dat = np.array(footer).astype(float).reshape(1,3)
   new_size = edge_data.shape[0]+1
   edge_data.resize(new_size, axis=0)
   edge_data[new_size-1:new_size,:] = dat