|
security
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
Microsoft URL ScanOur web servers run IIS5 and we also make use of the Microsoft URL Scan
utility: (http://www.microsoft.com/technet/security/tools/urlscan.mspx). By default Microsoft's URL scan utility blocks a number HTTP Methods including "HEAD". We have a number of clients concerned that blocking the HEAD method will interfer with web crawlers (google bot and msn bot). I can't really find any information which indicates if this is true or not. Our sites are being listed in search engines so I suspect its a non issue but I can't really find any official documentation to backup my opinions. Can anyone provide any information? Thanks in advance, Brad Hi Brad,
I'm not familiar with the details about how Internet search engines like Goggle crawling on web sites. However from my point of view, they must use GET verb instead of HEAD to get contents being indexed. So if your site can be properly listed by Google or MSN searching, I assume they are still working smoothly without the HEAD verb being allowed. If your clients do have concern on are concerned with this, I think you can just enable HEAD verb in URLScan.ini and this will not bring any additional security risk. URLScan blocks HEAD verb due to it's not being frequently used but not indicating it's a potential flaw. Please let me know if you have any further questions. Thanks. Best Regards, WenYuan Wang Microsoft Online Community Support ================================================== Get notification to my posts through email? Please refer to: http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif ications. Note: The MSDN Managed Newsgroup support offering is for non-urgent issues where an initial response from the community or a Microsoft Support Engineer within 1 business day is acceptable. Please note that each follow up response may take approximately 2 business days as the support professional working with you may need further investigation to reach the most efficient resolution. The offering is not appropriate for situations that require urgent, real-time or phone-based interactions or complex project analysis and dump analysis issues. Issues of this nature are best handled working with a dedicated Microsoft Support Engineer by contacting Microsoft Customer Support Services (CSS) at: http://msdn.microsoft.com/subscriptions/support/default.aspx. ================================================== This posting is provided "AS IS" with no warranties, and confers no rights. Brad wrote on Tue, 18 Jul 2006 20:25:00 -0400:
Show quoteHide quote > Our web servers run IIS5 and we also make use of the Microsoft URL Scan utility: HEAD is handy to determine if the page has changed since it was last indexed > (http://www.microsoft.com/technet/security/tools/urlscan.mspx). > > By default Microsoft's URL scan utility blocks a number HTTP Methods > including "HEAD". We have a number of clients concerned that blocking the > HEAD method will interfer with web crawlers (google bot and msn bot). > > I can't really find any information which indicates if this is true or > not. Our sites are being listed in search engines so I suspect its a non > issue but I can't really find any official documentation to backup my > opinions. > > Can anyone provide any information? > > Thanks in advance, > Brad (and as only the header information is returned, is much more bandwidth friendly), and if so it will then request the page to index the content. However, looking at my own IIS logs, I don't see any of the bots that are indexing the site using it. In a site made up of only dynamic pages (eg. ASP/ASP.Net) HEAD requests are often next to useless as the last modified date returned is almost always the current server date/time. Blocking HEAD requests may save server resources as those dynamic pages won't be run unless data will be returned to the client (even with a HEAD request the code in the page will need to be executed, it's just that IIS discards the content that would be returned for a GET/POST request) Dan
Show quote
Hide quote
"Brad Baker" <brad@nospam.nospam> wrote in message In my life URLSCAN creates more problems for me than solutions.... what I donews:O$ZoGnsqGHA.4912@TK2MSFTNGP05.phx.gbl... > Our web servers run IIS5 and we also make use of the Microsoft URL Scan > utility: (http://www.microsoft.com/technet/security/tools/urlscan.mspx). > > By default Microsoft's URL scan utility blocks a number HTTP Methods > including "HEAD". We have a number of clients concerned that blocking the > HEAD method will interfer with web crawlers (google bot and msn bot). > > I can't really find any information which indicates if this is true or not. > Our sites are being listed in search engines so I suspect its a non issue > but I can't really find any official documentation to backup my opinions. > > Can anyone provide any information? > > Thanks in advance, > Brad > > instead is use the Microsoft Baseline Security Checker on the server and follow the advice and I never use URLSCAN !!! PS.: things only break on the server (for me) after I use that tool, so I don't use it ! Hi Brad,
Just want to check if the issue has been resolved? If it still persists, please don't hesitate to update here. We'll go on to assist you on it. Thanks. :) Best Regards, WenYuan Wang Microsoft Online Community Support 
Other interesting topics
|
|||||||||||||||||||||||
