Querying json api and http endpoint using only python endpoints
posts python programmingOccationally I would have a http api endpoint that I would like to access and use via a python script. However most examples online requires installing https://docs.python-requests.org/en/latest/index.html. This is certainly a good advice for more complex scripts... but for stand-alone scripts where you just want to use only builtin python features... it would fail. Below are some functions that does away with such requirements.
Also note that I've added user-agent support here as some endpoints I try to use, assumes that you are trying to access it via another browser. This will not work for more locked down apis, but it shall be fine for most cases.
import urllib.request
import json
def get_url_content(url: str) -> str:
"""
Retrieves the content of a given URL as a string.
Args:
url (str): The URL to retrieve content from.
Returns:
str: The content of the response.
"""
try:
headers = {
'User-Agent': 'Mozilla/5.0'
}
request = urllib.request.Request(url, headers=headers)
response = urllib.request.urlopen(request)
if not response.getcode() == 200:
raise Exception(f"Failed to retrieve content. Status code: {response.getcode()}")
return response.read().decode('utf-8')
except urllib.error.HTTPError as e:
print(f"HTTP Error: {e}")
return None
except Exception as e:
print(f"An error occurred: {e}")
return None
def get_url_json_api(url: str) -> dict:
"""
Retrieves JSON content from a given URL.
Args:
url (str): The URL to retrieve JSON content from.
Returns:
dict: The parsed JSON content.
"""
received_content = get_url_content(url)
if not received_content:
return None
try:
return json.loads(received_content)
except json.JSONDecodeError as e:
print(f"Failed to parse JSON. Error: {e}")
return None