Due to the use of a sudo env, the records were being added to the sudo cache one by one instead of all at once. This meant the prefetching was not able to load all records at once, leading to prohibitive times when processing thousands of records.