PortAudio uses what it calls 'default suggested latencies' but in callback streaming mode, they can result in wildly inaccurate buffer sizing (e.g. the user requests a buffer size of 128 but PortAudio actually instructs ASIO to use a much bigger size).
What we do now is to improve PortAudio's suggested latency calculation by basing it on the actual buffer size requested by the user.