Even without Client Certificate-Based Security, the clients will need to authenticate via SSL when interacting with Core Web Services. Broker runs during the Provisioning process in order to get the validation certificate so it can use Web Services such as ServerAuthentication.
When the device is failing vs Succeeding, is there a difference in what PXE Representative its getting the WinPE WIM Image from? Its possible you have a PXE Representative with a "Stale" version of the WinPE Image on it that is causing the failures.
If you are saying other computers provision fine but this one times out then check the bios and make sure the date and time are correct.