uModbus on Raspberry Pi

I'm trying to add modbus RTU communications to a raspberry pi board with a USB to RS485 converter.  I set up the server sample code on the raspberry pi and I am using Modbus Poll to send the requests.  Watching the transmit and receive LEDs  on the sending side, I am seeing a large timing difference between transmit and receive.  I am seeing delays between 150mS and 800mS.  Any ideas?