While using Datapipes, there are 4 event types i.e. Add, Insert, Update and Refresh Can somebody elaborate on the different event types and what are the differences?

Hello pushpakchakraborty​ ,

I believe you are referring to the AFDataPipeAction Enumeration. Please let us know if the differentiation provided with the documentation is not clear.

Can you please explain the following event type please?

Insert 

The event represents the insertion of a new data value before the last event in the time series 

What does new data value before the last event in time series mean? Is it for out of order events?

Hello pushpakchakraborty​ ,

traditionally the last event in the time series is referred to as Snapshot. Another definition is the event (value, timestamp, status, flags) with the most recent timestamp. This event still is only known by PI Snapshot Subsystem. Assume an event with a more recent timestamp arrives and "survives" compression, the previous Snapshot is pushed towards PI Archive Subsystem, so at this time, we consider the event as archived.

Traditionally the Snapshot value can only be 15 minutes into the future, so ServerTime + 15m. To allow to also record value predictions, the PI Data Archive has been enhanced to also support future data. Since we differ between historical and future PI Points and it is a PI Point attribute "future" which when set to true configures a PI Point as future points. Future points do not use PI Snapshot Subsystem, if I recall correctly, but events become recorded to specific future archives. For future PI Points, the event with the latest timestamp is the last event in the time series.

Another thing to note for historical PI Points (future=false) is that when an event is received with a timestamp before the Snapshots timestamp, we talk about Out Of Order (OOO) insertion. With OOO data, no compression is applied.

I believe it is easier for you to focus on AFDataPipeType Enumeration and ask yourself if you are interested to be informed about new Snapshot or new Archive events. Snapshot events will not necessarily make it into the archive due to compression but are a good choice to e.g. update trends with recently received updates. When monitoring Archive updates, you will be informed about archived values but you may see value changes kind of late with late eventually meaning multiple minutes or even hours, depending on scan frequency, exception and compression.

If you are monitoring future data or e.g. like to get informed about RangeDelete updates, you should be going with TimeSeries.

I suggest you to create a simple command line application as the sample shared with PIDataPipe Class and test certain scenarios of data updates you are interested in. Exercising with real world examples and observing the actual behavior(s) is likely the best school.