• State Not Answered
  • Replies 1 reply
  • Subscribers 2 subscribers
  • Views 69 views
  • Users 0 members are here
Options
Related

KB article / white paper describing the intricacies of the backfill algorithm

Alistair

I recall reading a while ago an KB article or a white paper that went into depth concerning the analysis backfill process and I'm looking for it again.

In it, there was mention that when 'delete existing data' was selected, the data was deleted and recalculated in chunks, maybe of a day or so. As such, most recalculated data went into the archive as OOO data, but if the end time was * then deleting the last chunk of data would include the current snapshot and thus that last chunk of recalculated values would go in as new snapshots and be subject to compression.

I've just mentioned this to a colleague and I'd like to be able to back it up with the documentation (and to reassure myself that I haven't imagined it).

Does anyone know the document I'm thinking of and can you point me to it, or anything else that confirms or refutes my understanding?

 

Parents
  • 0

    If you login to my.osisoft.com, you can search on phrase "Analytics Backfill". Lots of results comes back. You may then filter on KB Articles. The first in that list looks interesting:

     

    KB01056 Backfilling and Recalculating Analyses with Asset Analytics

     

    Some sections of that KB are also found in the PI Server User Guide.

     

    Your post blends a few different topics such as Compression and OOO data.

     

    Regarding Compression, it is typically applied when data passes through the Snapshot Subsystem on the PI Data Archive. During that flow to the Snapshot SS, a decision is made whether the previous snapshot is to be written to archives or dropped off due to compression. Whenever someone writes data prior to the existing snapshot, compression is not applied because that data does not flow to the Snapshot SS. Rather it goes directly to the archives as uncompressed data, and since it occurs before the current snapshot, it will be OOO.

     

    A "well-known secret" about Analysis Recalculations is that it will be uncompressed and OOO unless you were to delete existing data INCLUDING the current snapshot. That means any recalculations would be treated as new incoming snapshots flowing into the Snapshot SS, and therefore also flowed to the Compression Subsystem. But if you only delete for a range prior to the current snapshot, then compression is not applied.

Reply
  • 0

    If you login to my.osisoft.com, you can search on phrase "Analytics Backfill". Lots of results comes back. You may then filter on KB Articles. The first in that list looks interesting:

     

    KB01056 Backfilling and Recalculating Analyses with Asset Analytics

     

    Some sections of that KB are also found in the PI Server User Guide.

     

    Your post blends a few different topics such as Compression and OOO data.

     

    Regarding Compression, it is typically applied when data passes through the Snapshot Subsystem on the PI Data Archive. During that flow to the Snapshot SS, a decision is made whether the previous snapshot is to be written to archives or dropped off due to compression. Whenever someone writes data prior to the existing snapshot, compression is not applied because that data does not flow to the Snapshot SS. Rather it goes directly to the archives as uncompressed data, and since it occurs before the current snapshot, it will be OOO.

     

    A "well-known secret" about Analysis Recalculations is that it will be uncompressed and OOO unless you were to delete existing data INCLUDING the current snapshot. That means any recalculations would be treated as new incoming snapshots flowing into the Snapshot SS, and therefore also flowed to the Compression Subsystem. But if you only delete for a range prior to the current snapshot, then compression is not applied.

Children
No Data