Thursday 15 January 2015

Core data iterate over fetchrequest in chunks with setFetchLimit only processing half the records -


I'm trying to process several objects in a certain size (batch section). This loop starts working, but it only processes half of the records. The relevant piece of code is:

  {// database to prepare the product without images in NSFetchRequest * productFetchRequest = [NSFetchRequest fetchRequestWithEntityName: @ "product"]; // Sort first according to final converted photo NSSortDescriptor * sortDescriptor = [[NSSortDescriptor alloc] initWithKey: @ "photoModificationDate" ascending: no]; [ProductFetchRequest set certificatescript: @ [sortedcriptter]]; NSPredicate * predicate = [NSPredicate predicateWithFormat: predicateString]; [ProductFetchRequest set pediatry: counting]; // Get the total count first NSUInteger numberoffproducts = [self.backgroundMOC countForFetchRequest: productFetchRequest error: & amp; Error]; NSLog (@ "Receiving images:% D product", numberoffproducts); // Set the batch size to get the amount of data nsuenter batch size = 25; [ProductFetchRequest setFetchBatchSize: batch size]; [ProductFetchRequest setFetchLimit: Batch Size]; // Get the products in batch (NSUINTER offset = 0; offset & lt; numberofproducts; offset + batch size) {@autoreleasepool {{productFetchRequest setFetchOffset: offset]; NSArray * products = [self.backgroundMOC executeFetchRequest: productFetchRequest error: & amp; Error]; NSLog (@ "Offset:% D, Number of Products:% d", Offset, [Product Count]); If (! Products) {do not return; } (Products in the product * product) {nslog (@ "download photo for product:% @", product.number); [Self-download endorseproduct: product]; } [Self-saving and resetback mouc]; }} Yes return;   

}

}

The log shows that for the first half of the count (numberOfProducts), it acts as expected, so that part of 25 products is processed is. After that first half, there are 0 records as a result to bring in the loop. If I try the same code again, then only half of the remaining record is processed, so the total 3/4 What am I doing wrong? Note that the managed object contact is not saved only, but it also resets the save after saving for saving. If I do not do this, the program crashes properly after downloading about 3000 pictures.

First point: There may be some basic misconception that fetchLimit and fetchBatchSize

fetchLimit and fetchoffset determine how many records are received < P> fetchBatchSize indicates how many records are retrieved during a visit to the continuous store. In this case, the result of 4 visits will be for 100 million, one fetchBatchSize store in 25 stores (with or without fetchBatchSize ). (In other words, 4 executed SQL statements for specific SQLite stores. However, this is all behind the scenario.)

In this way, code snippet

  request. FetchLimit = X; Request.fetchBatchSize = x;   

The number of visits to the store will always be in a way.

The other point: I'm not sure that your setup with the second MOC is very understandable. I think you are already in a background thread. As far as I know that resetting the MOC is quite expensive. This is not really necessary if you disable the undo manager of the MOC. For looping, I believe you can get all the records and take care of fetchBatchSize discrete "chunking". Due to defective behavior of core data, your @autoreleasepool will only bring limited benefits in the loop

@autoreleaspool useful when you download images It is probably enough to batch this part of the process.

It is being said, you may not want to change the thing that is (working).

Third point: You calculate the number of records based on an unknown (for us) prediction string. Is it dynamic? Not sure whether this is also not part of the problem. After all, it does not know what it is, it is amazing that the number of records changes.

Finally: Check whether you can do without resetting your MOC.

No comments:

Post a Comment