Parallel threads with C#

Example of some code where I decided to take the data from the DB in batches instead than all in one go, which was timing out. This could have been done one after the other, just waiting for the previous one to finish before requesting next batch, but I decided to give it a go and try to get them all at the same time by making separate calls and waiting for them all to finish to build the result in order.

Important bits in commented lines.

using System.Text;
using System.Threading.Tasks;

    public abstract class BaseExporterRepository : BaseDataRepository<DbContext>
    {
        private readonly IDbContextFactory _contextFactory;
        protected readonly int batchSize = 5000;
        protected virtual string GetHeader() => "";
        protected virtual Task<StringBuilder> BuildBatchAsync(DbContext ctx, int skip) => null;
        protected virtual Task<int> CountAllAsync() => null;

        protected BaseExporterRepository(IDbContextFactory contextFactory) : base(contextFactory)
        {
            _contextFactory = contextFactory;
        }

        public async Task<string> Build()
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendLine(GetHeader());

            var totalRows = await CountAllAsync();
            var batches = (int)Math.Ceiling((decimal)totalRows / batchSize);

            var buildBatchTasks = new List<Task<StringBuilder>>(); // Prepare list of Tasks
            using (UnitOfWork)
            {
                for (int i = 0; i < batches; i++)
                {
                    var ctx = _contextFactory.GetContext<DbContext>();
                    var task = BuildBatchAsync(ctx, batchSize * i);
                    buildBatchTasks.Add(task); // Add Task to list. 
                                               // Notice I'm not executing it yet, no "await" here
                }
            }
            await Task.WhenAll(buildBatchTasks.ToList()); // Now I await for them all at the same time
            foreach (var task in buildBatchTasks) sb.Append(task.Result); // For each of'em I take the Result
            
            var result = sb.ToString();
            return result;
        }
    }

Warning
I had to create multiple EF Contexts to achieve this, doing so slowed down the local computer a very lot, which probably means that such technique, in this particular scenario wasn’t the best approach, as a Context takes so many resources. Still it was cool to see this request going super fast (2-3sec) compared with the half a minute it took without it.

Leave a Reply

Your email address will not be published. Required fields are marked *