Parallel threads with C#

Home / Web programming / Parallel threads with C#

Example of some code where I decided to take the data from the DB in batches instead than all in one go, which was timing out. This could have been done one after the other, just waiting for the previous one to finish before requesting next batch, but I decided to give it a go and try to get them all at the same time by making separate calls and waiting for them all to finish to build the result in order.

using System.Text;
using System.Threading.Tasks;

    public abstract class BaseExporterRepository : BaseDataRepository<DbContext>
    {
        private readonly IDbContextFactory _contextFactory;
        protected readonly int batchSize = 5000;
        protected virtual string GetHeader() => "";
        protected virtual Task<StringBuilder> BuildBatchAsync(DbContext ctx, int skip) => null;
        protected virtual Task<int> CountAllAsync() => null;

        protected BaseExporterRepository(IDbContextFactory contextFactory) : base(contextFactory)
        {
            _contextFactory = contextFactory;
        }

        public async Task<string> Build()
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendLine(GetHeader());

            var totalRows = await CountAllAsync();
            var batches = (int)Math.Ceiling((decimal)totalRows / batchSize);

            var buildBatchTasks = new List<Task<StringBuilder>>();
            using (UnitOfWork)
            {
                for (int i = 0; i < batches; i++)
                {
                    var ctx = _contextFactory.GetContext<DbContext>();
                    var task = BuildBatchAsync(ctx, batchSize * i);
                    buildBatchTasks.Add(task);
                }
            }
            await Task.WhenAll(buildBatchTasks.ToList());
            foreach (var task in buildBatchTasks) sb.Append(task.Result);
            
            var result = sb.ToString();
            return result;
        }
    }

Warning
I had to create multiple EF Contexts to achieve this, doing so slowed down the local computer a very lot, which probably means that such technique, in this particular scenario wasn’t the best approach, as a Context takes so many resources. Still it was cool to see this request going super fast (2-3sec) compared with the half a minute it took without it.

Leave a Reply

Your email address will not be published. Required fields are marked *