• State Not Answered
  • Replies 13 replies
  • Subscribers 3 subscribers
  • Views 330 views
  • Users 0 members are here
Options
Related

Weird behavior processing IEnumerable<AFEventFrame>

andrew_Inwood

I have some code where I search for some event frames, then try to process the results.

The issue I have is that processing the IEnumerable results is very, very slow. It consistently takes my application time from 5 seconds to 45 seconds.

 

Here is the code:

        public void LoadClosedTasks(CurrentShift currentShift)
        {
            //CodeTimer LoadClosedTasksTimer = new CodeTimer();
            if (currentShift == null) return;
            List<AFSearchTokenBase> tokens = AppliedSearchTokens.ToList();                  //Copy the list
            tokens.RemoveAll(x => x.Filter == AFSearchFilter.InProgress);                   //Remove tokens for "In Progress"
            tokens.Add(new AFSearchValueToken("|COMPLETED_SHIFT_ID", currentShift.ShiftID.ToString()));
            IEnumerable<AFEventFrame> RetrievedFrames = null;
            using (AFEventFrameSearch search = new AFEventFrameSearch(LogDatabase, "baseSearch", tokens))
            {
                //DateTime searchStart = DateTime.Now;
                RetrievedFrames = search.FindObjects(0, true, 0);
            }
            ConcurrentBag<OperatorTask> createdFrames = new ConcurrentBag<OperatorTask>();  //Add to concurrent bag to avoid multi-thread issues.

            return;

            Parallel.ForEach(RetrievedFrames, frame =>
                {
                    //OperatorTask frameInstance = new OperatorTask();
                    //frameInstance.PopulateFromFrame(MainForm, frame, this);
                    //createdFrames.Add(frameInstance);

                });
            }

For info, this routine is called using an await Task.Run.

You can see where I place the "return" statement in the code, to bypass the Parallel.ForEach() block. With return there, the application loads (doing lots of other stuff) in 4.5 seconds. If I remove the return statement, the application takes 45 seconds. And this is with the code body in the Parallel.ForEach block commented out, as it shows above! It is not the code in the block that is slow, because there is no code running and I still get the problem.

I also tried a synchronous ForEach across the collection. Same issue.

This suggests to me that the simple act of enumerating the returned search is taking 40 seconds. (Timing it confirms that).

 

So... how many elements is it iterating?

 

Glad you asked. The search results are returning exactly 1 result!

 

Any idea on what is happening here? Search.FindObjects does have LoadFull set to true, but switching it to false doesn't help. Setting the page size doesn't help. I really am at a loss on how to fix this.

Parents
  • 0

    You have obfuscated so much of your code that its difficult to focus on the problem. For example, you have an AppliedSearchTokens property that offers NO clues as to its contents. Does it filter on an event frame template? Does it filter on an element?

     

    Does attribute |COMPLETED_SHIFT_ID use a data reference? Some times, never, always?

     

    Performance is not determined by how many event frames are in your result set. It's about how many event frames are sent to the local host performing any client-side filtering.

     

    The RetrievedFrames property should be defined as a List or IList<AFEventFrame>, and the search loop should have

     

     RetrievedFrames = search.FindObjects(0, true, 0).ToList();

     

     

  • 0 in reply to Rick_Davin_3.0

    Hi Rick,

    AppliedSearchTokens is just a general list of tokens common to searches for a specific asset. There are three:

    1.) AFSearchFilterToken on AfSearchFilter.TemplateName

    2.) AFSearchValueToken on "|SITE" attribute (name data type)

    3.) AFSearchValueToken on "|Completed_Shift_ID", which is the GUID of an EventFrame (EventFrame.ID)

     

    Why do you recommend converting RetrievedFrames to a List or IList? The search.FindObjects method returns IEnumerable<AFEventFrame>; all I want to do is enumerate over it. List is derived from IEnumerable, and I don't need any of the additional properties that List gives me.

  • 0 in reply to andrew_Inwood

    You did not answer my question about "|Completed_Shift_ID", Does it use a data reference? You say it is a Guid of an event frame. What relation, if any, does the "|Completed_Shift_ID" have to the found event frame's ID property?

     

    Does the "|SITE" attribute have a data reference?

     

    An AFSearch is a disposable object. It applies server-side filters against ALL event frames in the PIFD, and may send event frames to the client for additional client-side filtering. It also is a forward-only iteration. All of that means that if you want to persist found event frames beyond the scope of the disposable search, then you need to persist to the some appropriate object, and IEnumerable<AFEventFrame> is NOT such an object. That way, when you want to process over the found event frames, you do not have to take a lengthy search again, but rather have them readily available in a nice memory structure such as a List<AFEventFrame>. As for the List properties you do not need, may I be so bold as to suggest that you can ignore them?

     

    And, no, List is not derived from IEnumerable. Rather, List implements the IEnumerable interface by providing concrete code for the interface method signatures.

     

Reply
  • 0 in reply to andrew_Inwood

    You did not answer my question about "|Completed_Shift_ID", Does it use a data reference? You say it is a Guid of an event frame. What relation, if any, does the "|Completed_Shift_ID" have to the found event frame's ID property?

     

    Does the "|SITE" attribute have a data reference?

     

    An AFSearch is a disposable object. It applies server-side filters against ALL event frames in the PIFD, and may send event frames to the client for additional client-side filtering. It also is a forward-only iteration. All of that means that if you want to persist found event frames beyond the scope of the disposable search, then you need to persist to the some appropriate object, and IEnumerable<AFEventFrame> is NOT such an object. That way, when you want to process over the found event frames, you do not have to take a lengthy search again, but rather have them readily available in a nice memory structure such as a List<AFEventFrame>. As for the List properties you do not need, may I be so bold as to suggest that you can ignore them?

     

    And, no, List is not derived from IEnumerable. Rather, List implements the IEnumerable interface by providing concrete code for the interface method signatures.

     

Children
No Data