Writing And Reading A Dictionary In Specific Format (python)

Sorry another newbie query :| To build upon the suggestion which was given here, optimizing I need to be able to incrementally build a dictionary i.e. one key: value at a time in

Solution 1:

You could look into using the ZODB in combination with the included BTrees implementation.

What that gives is a mapping-like structure that writes individual entries separately to the object store. You'd need to use savepoints or plain transactions to flush data out to the storage, but you can handle huge amounts of data this way.