Writing Import.io Data to a Database using Python

This article will go over how to write Import.io data to a MySQL database. This method can be altered by changing the SQLAlchemy URI and database connector. You can read more about SQLAlchemy here.

 Dependencies

- Requests

pip3 install requests

- Pandas

pip3 install pandas

- MySQL Connector

pip3 install mysql-connector

1. Go to the Integrate tab for your Import.io extractor
2. Copy the CSV from the last successful run endpoint 
3. Replace the endpoint into the <ENDPOINT> below with the CSV endpoint

import requests
import pandas as pd
import mysql.connector
from sqlalchemy import create_engine

API_ENDPOINT = '<ENDPOINT>'

session = requests.Session()
response = session.get(url=API_ENDPOINT).content

df = pd.read_csv(io.StringIO(response.decode('utf-8-sig')))

print(df.head())

engine = create_engine('mysql+mysqlconnector://os.environ['MYSQL_USER']:os.environ['MYSQL_PASSWORD']@os.environ['MYSQL_HOST_IP']:os.environ['MYSQL_PORT']/sandbox', echo=False)

# Writing Dataframe to Mysql and replacing table if it already exists
df.to_sql(name='Importio', con=engine, if_exists = 'replace', index=False)