Wednesday 15 August 2012

performance - LINQ Out of Memory Error -


I am inquiring about 200k records and using all the memory of the server (not surprisingly). I am new to LINQ, so I got the following code which should help me but I do not have a way to use it:

  Public Fixed IEnumerable & lt; IEnumerable & lt; T & gt; & Gt; Batch & lt; T & gt; (This IEnumerable & lt; T & gt; Collection, Int Batch Sees) {List & lt; T & gt; Nextbatch = new list & lt; T & gt; (Batch system); Foreign currency (t items in the collection) {nextbatch.Add (item); If (Next batch.count == batch size) {next batch yield yield; Nextbatch = new list & lt; T & gt; (Batch system); }} If nextbatch returns yield (next batch; gt; }   

Source:

This is my code that creates "out of memory" error. How do I include a new batch function in my code?

  var crmMetrics = _crmDbContext.tpm_metricsSet.Where (a => a.modifiedOn> = final runrate); Forresh (version CRMMTR in CRM metrics) {Metric = New Metric (); Metric.ProductKey = crmMetric.tpm_Product.Id; DbContext.Metrics.Add (metric); DbContext.SaveChanges (); }    

This is an extension method, so if it is part of a stable class and Your code contains a reference to the namespace of the class that you can do:

  var crmMetricsBatches = _crmDbContext.tpm_metricsSet. Where (a = & gt; a.modifiedOn> = lastRunDate). Other numbers () / / !! .batch (20);   

This will not help except . From ANNMereble () , you still store all the data in memory, but now are in section 20. This is because you can not use the method directly against IQueryable : The Entity Framework will try to translate it into SQL, but there is definitely no clue how to do this.

As TG said, skip and low is made for more:

  var crmMetricsPage = _crmDbContext.tpm_metricsSet. Where (a = & gt; a.modifiedon & gt; = lastRunDate). Orderbie (a => a. ??) // Some assets you choose. Skip (no page * pages) Move (page size);   

Where you are going to code from not page , the number of that page ( - 1 ) to 0 need number to Skip and Low > Expressions , and EF knows how to change SQL. Paging Code> Once the number of recorded queries is high, but the resources are spared. One condition is that you can set a pageSize in advance. I do not know that it fits with your argument.

If you can not use paging, you should try to narrow the filter ( ou (a = & gt; a.ModifiedOn & gt; = lastRunDate ), For example try to get data in batches of a day or week.

No comments:

Post a Comment