dcnum.write.queue_writer_thread

Classes

QueueWriterThread

Write events from a queue to an .rtdc file

Module Contents

class dcnum.write.queue_writer_thread.QueueWriterThread(*args, **kwargs)

Bases: dcnum.write.queue_writer_base.QueueWriterBase, threading.Thread

Write events from a queue to an .rtdc file

Events coming from a queue cannot be guaranteed to be in order. The QueueWriterThread uses a EventStash to sort events into the correct order before sending them to the ChunkWriter for storage.

Parameters:

• event_queue – A queue object to which other processes or threads write events as tuples (frame_index, events_dict).

• write_queue_size – A mp.Value that is populated with the number of event chunks waiting to be written to the output file by the ChunkWriter.

• feat_nevents – This 1D array contains the number of events for each frame in the input data. This serves two purposes: (1) it allows us to determine how many events we are writing when we are writing data from write_threshold frames, and (2) it allows us to keep track how many frames have actually been processed (and thus we can expect entries in event_queue for). If an entry in this array is -1, this means that there is no event in event_queue. See write_threshold below.

• path_out – Output path for writer

• hdf5_dataset_kwargs – Dictionary of keyword arguments (e.g. “compression”) for HDF5 dataset creation.

• write_threshold – This integer defines how many frames should be collected at once and put into writer_dq. For instance, with a value of 500, at least 500 items are taken from the event_queue (they should match the expected frame index, frame indices that do not match are kept in a EventStash). Then, for each frame, we may have multiple or None events, so the output size could be 513 which is computed via np.sum(feat_nevents[idx:idx+write_threshold]).