There are a lot of factors.
If you can close out firewalls, proxyservers, intrusion preventions, you can try an alternative Distribution an Patch Setting on the DMZ Server.
Choose a setting that does not throttle cpu or bandwith.
What version are you on? There are some known issues with CSA download times on 2017 versions. This was addressed in 2017.1 Service Update 1. See the readme below.