How to buffer data written to PI AF Attributes to both members of PI DA collective?

Hello gmalek​ ,

as mentioned in my response to your other recent question, please do not try to break the Known Servers Table (KST) of a PI Collective.

When creating a custom app, you usually do not need to think about if the receiving PI Data Archive is a standalone PI Data Archive or a PI Collective. PI Buffer Subsystem is your friend and helps you with a) Buffering, b) Fanning and c) Compression.

Buffering is performed through queues for each member of a PI Collective. If a member is down or otherwise unavailable, the queue will grow until it can be emptied again. Under normal circumstances, writes should be immediately performed. They go through the buffer queue but this should not result into data arriving delayed server side.

Fanning means that PI Buffer Subsystem takes care to update the time series data on all members of the PI Collective and a PI Collective can consist of a Primary and up to 99 secondary nodes. Another function performed by PI Buffer Subsystem is Compression. A 'compressed' marker tells the PI Snapshot Subsystem that the compression algorithm is already applied and this way it is ensured that all Collective members receive the same updates.

Please note that there is also Point locking involved. This means a PI Point which receives time series data through a PI Buffer Subsystem instance is locked against unbuffered writes. The only way to write a PI Point with a buffer lock is through PI Buffer Subsystem. With earlier PI versions, only a single PI Buffer Subsystem instance could own the PI Point. These days it is possible to write time series update through multiple sources but they must all be buffered.

Configuring PI Buffer Subsystem should be one of your first actions because PI Buffer Subsystem plays important roles (buffering, compression, fanning).

Without PI Buffer Subsystem, you will likely only be able to update the PI Point on the Primary.

With regards to the Tuning parameter Rick_Davin_3.0​ mentioned. Please do not allow writes on Secondary PI Collective members as this is usually not necessary as explained before.

Hello gmalek​ ,

as mentioned in my response to your other recent question, please do not try to break the Known Servers Table (KST) of a PI Collective.

When creating a custom app, you usually do not need to think about if the receiving PI Data Archive is a standalone PI Data Archive or a PI Collective. PI Buffer Subsystem is your friend and helps you with a) Buffering, b) Fanning and c) Compression.

Buffering is performed through queues for each member of a PI Collective. If a member is down or otherwise unavailable, the queue will grow until it can be emptied again. Under normal circumstances, writes should be immediately performed. They go through the buffer queue but this should not result into data arriving delayed server side.

Fanning means that PI Buffer Subsystem takes care to update the time series data on all members of the PI Collective and a PI Collective can consist of a Primary and up to 99 secondary nodes. Another function performed by PI Buffer Subsystem is Compression. A 'compressed' marker tells the PI Snapshot Subsystem that the compression algorithm is already applied and this way it is ensured that all Collective members receive the same updates.

Please note that there is also Point locking involved. This means a PI Point which receives time series data through a PI Buffer Subsystem instance is locked against unbuffered writes. The only way to write a PI Point with a buffer lock is through PI Buffer Subsystem. With earlier PI versions, only a single PI Buffer Subsystem instance could own the PI Point. These days it is possible to write time series update through multiple sources but they must all be buffered.

Configuring PI Buffer Subsystem should be one of your first actions because PI Buffer Subsystem plays important roles (buffering, compression, fanning).

Without PI Buffer Subsystem, you will likely only be able to update the PI Point on the Primary.

With regards to the Tuning parameter Rick_Davin_3.0​ mentioned. Please do not allow writes on Secondary PI Collective members as this is usually not necessary as explained before.

Hi Gregor,

Thank you for your responses. Networking issues on our servers prevent us from obtaining external data at the moment, so we are trying to run the program from a client workstation as a temporary workaround. Correct me if I'm wrong, but I don't believe the PI Buffer Subsystem normally runs on client workstations. If this is true, then what is the appropriate way to write to both members of a collective when PI Buffer Subsystem is not available?