The Requests module for Python makes capturing and working with HTML code from any website.
Requests comes installed in many of the Python distributions, you can test if it is installed on yours machine by running the command: import requests
If that command fails, then you’ll need to install the module using Conda or Pip
import requests t = requests.get('http://aiwithai.com') print(t.text)
As you can see, using just 3 lines of code you can return the HTML from any website
Requests has plenty more features, here are couple I use commonly
t.status_code == returns the status of your get request. If all goes well, it will return 200, otherwise you will get error codes like 404
You can also extract your results into json