I'm trying to webscrape a website and it appears to be feeding me bogus HTML with the WebClient.DownloadData() method.
Is there a way for me to "fool" the website that I'm a browser of sorts?
Adding this header still doesn't fix the issue:
Client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
Is there something else I can try? 🙂
If it helps at all. I'm trying to download the source of a ThePirateBay Search.
As you can see, the source shows what is should, seed information for the movies etc. But when I use the DownloadData() method, I get random torrent results, nothing at all related to what I'm search for.
Try adding a user agent header so it thinks you are one of the major browsers (IE, FF, etc)