Add logic to check the number of items in the DB queue that have not yet been loaded when the mem queue is empty to process jobs more quickly. Otherwise waits till all downloads are finished then reads another x number of rows from DB
Add logic to check the number of items in the DB queue that have not yet been loaded when the mem queue is empty to process jobs more quickly.
Otherwise waits till all downloads are finished then reads another x number of rows from DB