Wget user agent avoidance
I use wget a LOT. Many times in working with a website I’ve got a ssh console opened to the site and if there’s a plugin that I’d like to download it doesn’t make sense to download and then upload, I want to just download it directly from xyz.com webserver to the website where it will be installed. However, many times people have set up rules to block wget downloads. Yes, I know people use wget to suck down entire websites, it uses up bandwidth, etc. etc. Well. Wget is capable of sending a different user-agent.
First off, the way you are tipped off that wget is disliked is if you’ve just visited the site and then try wget and get a message like this…
“HTTP request sent, awaiting response… 403 Forbidden”
essentially, if you add –user-agent=MSIE it might bypass user-agent filtering.