How do I know when StoreForward data is securely stored in Historian?

Question

In a meeting with a customer, who have had challenges on their network that triggered StoreForward on one or more AppEngines I received a collection of questions related to how it works, how to visualize status for the operator and whether it is 100% bulletproof. 
 Objectviewer lists these attributes related to StoreForward: 
 
 Historian.InStoreForward 
 Historian.InStoreForwardMask 
 Historian.StoreForwardDataLost 
 Historian.StoreForwardDeletionThreshold 
 Historian.StoreForwardMinDuration 
 Historian.StoreForwardProblem 
 Historian.StoreForwardSpaceAvailable 
 
 DeletionThreshold and MinDuration is part of the configuration, but where to find the details on the rest? 
 I guess they can be used to inform the operator &ndash; but details are needed to ensure that we are not misleading the operators. 
 In my test environment I can see that InStoreForward toogles if I stop and start the Historian, but does the value &lsquo;false&rsquo; mean that the Historian is back online AND data is forwarded to the Historian? 
 Or how should I verify the transfer is completed? 
 Will Historian acknowledge the transfer of data when they are in the RAM of the Server? - or will it happen when data is merged into the HistoryBlocks.

Hardik Shah · Answer

Michael Vittrup 
 When the Historian becomes available again, the App Engine immediately sends all data from the Store-and-Forward folder to the Historian. The time required for this transfer depends on the size of the stored data. You can verify this by checking the Store-and-Forward folder after the Historian is available: if there are no .dat files present, it means all data has been successfully transferred to the Historian.

Hardik Shah · Answer

Jakob Drescher There is a separate service called HCAL which maintain connection between Historian and 
 App Engine.You can refer below image.

I am not more clear about the trigger point but i do have observed that sometime data wasn't push to Historian in some scenario. and create some bad.dat folder in the App Engine Store Forward, In that case we have to manual push those data to historian.

Richard Köhlerstrand · Answer

Hi Michael! This is the reference i can find related to the Historian attributes on an engine. But I would also think it is interesting to understand the deeper mechanisms that evaluate some of these. 
 
 Run-Time Attributes

Engine.Historian.BufferCount

Obsolete. Do not use. Use the BufferSize attribute instead.

Read-Only

Engine.Historian.ConnectionPartner

The name of the partner Historian Server, if one is configured for the Historian. If no partner Historian is detected, the value of this attribute is an empty string.

Read-Only

Engine.Historian.ConnectState

The physical state of the connection between HCAL and the historian. This value should remain TRUE as long as HCAL is able to communicate with the historian, regardless of whether it is historizing data or not. A network failure, for example, will cause the ConnectState attribute to be set to FALSE. Even if the ConnectState attribute is FALSE, the AppEngine may still be connected to the historian and appear listed as a client in the historian System Management Console. In a redundant historian setup, the connect state is TRUE if the connection is good for at least one of the historians. To determine state of an individual historian in a redundant setup, use the ConnectStateMask attribute.

Read-Only

Engine.Historian.ConnectStateCmd

This attribute has no effect on the connection between the engine and historian (feature has not been implemented).

Supervisory, User

Engine.Historian.ConnectStateMask

A bitmask where the lowest order bit represents the connection state of the primary Historian and the second bit represents the connection state of the partner. If no partner is configured, the second bit will always be 0.

Read-Only

Engine.Historian.ForwardingChunkSize

Obsolete. Do not use.

Read-Only

Engine.Historian.ForwardingDelay

Obsolete. Do not use.

Read-Only

Engine.Historian.InStoreForward

If set to TRUE, HCAL is in store-and-forward mode. In a redundant historian setup, the InStoreForward attribute is TRUE if either one of the historians is in store-and-forward mode. To determine state of an individual historian in a redundant setup, use the InStoreForwardMask attribute.

Read-Only

Engine.Historian.InStoreForwardMask

A bitmask where the lowest order bit represents the store-and-forward state of the primary Historian and the second bit represents the store-and-forward of the partner. If no partner is configured, the second bit will always be 0.

Read-Only

Engine.Historian.ProcessInterval

Obsolete. Do not use.

Read-Only

Engine.Historian.StoreForwardProblem

If set to TRUE, the HCAL on the active engine is unable to store data while in store-and-forward mode. In a redundant historian setup, the StoreForwardProblem attribute is TRUE if either one of the historians has a store-and-forward issue.

Read-Only

Engine.Historian.StoreForwardSpaceAvailable

The available storage space, in MB, for the HCAL of the active engine.

Read-Only

Engine.Historian.StoreFwdDataLost

Set to TRUE when HCAL is in store-and-forward mode, the store-and-forward deletion threshold has been exceeded, and history data has been lost. This value remains TRUE until the connection to the historian has been re-established and store-and-forward is no longer in effect.

Read-Only

Configuration Attributes

Enable storage to historian

Engine.Historian.Enabled

If enabled, the values for attributes that have historization enabled will be stored to the specified historian.

None

Enable Tag Hierarchy

Engine.Historian.EnableTagHierarchy

If enabled, the model view is automatically replicated to the Historian node associated with the Galaxy. You can control this behavior at run time without undeploying and redeploying the engine of any affected objects.

Supervisory, User

Historian

Engine.Historian.Connection

The string to use for the connection to the historian. For the Historian, this is the node name. 
 If two Historian Servers are configured as partners, the node identified by this string becomes the primary historian for this engine. The partner information is configured and maintained on the Historian Server. 
 Click the ellipse button to browse for the historian on the network.

Read-Only

TCP Port

Engine.Historian.ConnectionTCPPort

The TCP port on the Historian Server node to which history data will be sent. The TCP port is configured when the Historian Server is installed. The default is 32568.

Read-Only

Enable compression

Engine.Historian.CompressionEnable

If enabled, history data is compressed before it is sent over the network to the Historian Server. This is used to reduce network utilization, but enabling this setting will increase the CPU load.

Read-Only

Throttling network bandwidth

Engine.Historian.BandwidthLimit

Specifies limit of bandwidth usage, in kbps, for network communication used by HCAL when communicating with the Historian. A value of 0 disables this feature (default). For more information on estimating your bandwidth needs, see the performance and sizing recommendations for the Historian Server in the System Platform Installation Guide . 
 On networks with limited bandwidth (for example, 256 kbps) you may want keep half of the bandwidth available for other traffic, so you would set this value to 128 kbps. Consider when the historian connection is lost, and store-and-forward data has accumulated on AppEngine nodes. When the connection between an AppEngine node and historian is restored, all the store-and-forward data is sent at once, and may use the entire network bandwidth (on a slow network) and cause the AppEngines to again lose communication with the Historian and go into store-and-forward data again. By setting a limit here of 2/3 the total bandwidth, this problem of AppEngines going in and out of store-and-forward can be avoided. 
 This limit applies to all history-related communication, including tag creation, sending real-time data, sending late data, and sending store-and-forward data.

Read-Only

Wait to send incomplete packets

Engine.Historian.BufferAgeLimit

Specifies the maximum time, in milliseconds, for keeping a partially-filled Historian Client Access Layer (HCAL) buffer before sending it to the Historian Server. If the buffer is full, then it is sent immediately, regardless of any value configured in this field. If you have fast-changing data, this setting is irrelevant. If you have slow-changing data and limited network bandwidth, you might want to increase this value so that you have less "chatter" on the network. The default is 1,000 milliseconds. Valid values are between 1,000 and 30,000 milliseconds.

Read-Only

Pre-processing buffer size

Engine.Historian.BufferSize

The total size, in MB, of all buffers used by Historian Client Access Layer (HCAL). The default and minimum value is 8. If you have very high data bursts, then you should increase this value. If this is too low, data loss occurs and error messages related to buffer overflows appear in the ArchestrA Logger. Increase this value in increments of 10 MB.

Supervisory, User

Store forward threshold

Engine.Historian.StoreForwardDeletionThreshold

The size, in MB, of free space to reserve on the HCAL store-and-forward disk. The space designated will not be used during store-and-forward. This value cannot be a negative number.

Read-Only

Store forward minimum duration

Engine.Historian.StoreForwardMinDuration

The minimum duration, in seconds, for HCAL to function in store-and-forward mode. HCAL will function in store-and-forward mode for this length of time even if the condition that caused HCAL to function in store-and-forward mode no longer exists.

Supervisory, User

Reconnect as soon as possible & do not mark disconnects

Engine.Historian.LateDataEnabled

Specifies how trends appear during communication disconnects between Application Server and Historian. It does not affect how trends of history data appear after communications have been restored. 
 If TRUE, no gap will appear in client-side trends for the disconnect interval. While disconnected, the interval is filled in with the last-received value before the disconnect. 
 If FALSE, a gap will appear in client-side trends for the disconnect interval. NULL values are injected on disconnect to create the gap. 
 In both cases, after reconnect, the interval will be filled in with store-and-forward data.

Supervisory, User