A lot of depends on the "Backbone" of the ISP. High congestion can slow it down during peak hours. A lot of ISP took this into consideration when they first build it, but with the increase in user in the past years most network infrastructures weren't build to handle such high network traffic...