Improving Training Speed for Faster RCNN

I am attempting to train a Faster RCNN using my own data set (504 images [250x375x3], 3 classes). Object sizes vary widely -- very small to relatively large -- and there are often multiple objects per image. The input layer dimensions for the NN were set as 32x32x3. The script I prototyped with successfully ran the example provided by Mathworks (for developing and training training a vehicle detector). However, when I swap things out with my own data and ground truth, Step 1 gets trained relatively quickly (18 minutes using a single Titan X GPU), however when it seems to be getting stuck on Step 2 (runs for multiple hours without progress). It simply states "--> Extracting region proposal from 302 training images". I suspect there is something different about my own data that slows things to a crawl, but I have not been able to identify the problem. Can anyone recommend things to try to speed up Step 2 - either pre-processing procedures or options to specify? Thanks!