In this article Al Pascual discusses what happens if you just configure IIS to access Windows Authentication and would the request from Silverlight for the clientaccesspolicy.xml or the crossdomain.xml file are being requested with your Windows Authentication over a few firewalls.
If integrated Windows authentication fails, due to improper user credentials or some other problem, the browser will prompt the user to enter their user name and password.
So if is Silverlight requesting that file protected by Integrated Windows authentication in a background thread, the browser will not display the prompt, therefore, Silverlight won’t access that resource and your application will fail miserably.