topsim.core.buffer.HotBuffer

class topsim.core.buffer.HotBuffer(capacity, max_ingest_data_rate)

HotBuffer represents the ingest-facing part of the Buffer. Observation data is intended to stay in the HotBuffer only temporarily, and ideally is moved to the ColdBuffer as soon as possible.

Transition to the ColBuffer is not instantaneous; rather it depends on the data rate supported by the ColdBuffer, which may be defined differently to the HotBuffer based on the Buffer config JSON.

Methods

__init__(capacity, max_ingest_data_rate)
has_capacity_for(observation_size)	Check if the HotBuffer has capacity (checks self.current_capacity).
has_stored_observations()	Hides the dictionary nature of observations stored in the Buffer.
has_waiting_observations()	Check if there are observations in the buffer to transfer to cold buffer.
next_observation_for_processing()	Produces the next observation without removal
observation_for_transfer()
process_incoming_data_stream(...)	During Ingest, the buffer will coordinate the incoming data from the observation.
receive_observation(observation, ...)	For an observation that needs to be moved to ColdBuffer storage, we must 'receive' it.
remove(observation)	Removes an observation from the cold buffer and updates the capacity accordingly
transfer_observation(observation, ...)