Python – How to interact with a webpage with python


Basically, I want to be able to mimic the functions of an internet browser, but without having to actually graphically render it. My ideal scenario would be, for example, a script which opens a connection with a webpage, makes all the necessary calls and interactions, and can list what javascript went off (this is a trivial example, but is the sort of thing I'm looking for). I basically want to be able to "browse" from python while logging various events that happen. The real dream would be able to program in reactions (such as sending a click if there's a box that looks like this or that, for example).

Is there a nice way to do this? Thank you.

Best Solution

You should check out twill. It works on the command line but also has a python api.