Optimizing Memory Usage in LINQ: A Case Study with IEnumerable
Today I was dealing with an IEnumerable which would iterate over 4 millions predictions to group by Fraud (a boolean predicted value) and count number of records. As a side note, each prediction (output) was an object with 300+ properties (it is a lot, but on purpose).
Below is an snippet of the code:
var metrics = predictions.GroupBy(p => p.Fraud).Select(g => $"{g.Key};{g.Count()}")
Half way through, I received an "Out of memory exception". After a few minutes thinking about, that was how I solved the problem (pay attention to the Select clause before GroupBy):
var metrics = predictions.Select(p => p.Fraud).GroupBy(fraud => fraud).Select(g => $"{g.Key};{g.Count()}")
By putting a Select before GroupBy I was able to abstract and send forth only the info I needed (out of the 300+ I have), speeding up, sparing memory and getting out of the exception.
Please comment and let me know what do you think about the problem I faced and the solution/path I took.
Senior Systems Analyst at Banco Daycoval
9 个月Angelo Vinicius Nobre Comar ?