- Data with size ranging from few lines to millions of lines are generated daily as CSV streams and files or delimiter separated streams and files. Examples of such data sources include Stock Exchanges, Weather Stations, Sensors in factory Settings, Computer Programs monitoring for various events and so on.
- In a comma-separated file, commas separate fields and every record starts in a new line.
- The pandas DataFrame class supports serializing and de-serializing of CSV in an extenstive way through the read_csv() method.
- The read_csv() method of pandas DataFrame class reads a CSV file and loads each record as a row in the DataFrame.
- In the similar way the pandas DataFrame class supports operations like reading and writing DataFrame contents from/to MySQL; and reading and writing DataFrame contents from/to PostgreSQL.
Example–To load data from a CSV file into a pandas DataFrame:
# Example Python program to read a CSV file
# and load the contents into a pandas DataFrame object
import pandas as pds
# Name of the CSV file
fileName = "./ServerResponseTimes.csv";
# Read CSV file into a pandas DataFrame instance
dataFrame = pds.read_csv(fileName);
# Print the contents of the DataFrame onto the console
print("Contents of the DataFrame as read from the .csv file:");
# Print the index of the DataFrame
print("Index of the DataFrame:");
Contents of the DataFrame as read from the .csv file:
Server Name Response Time(ms)
0 Host1 12
1 Host2 25
2 Host3 10
3 Host4 40
4 Host5 71
Index of the DataFrame:
RangeIndex(start=0, stop=5, step=1)
Example-To load a binary stream of CSV records into a pandas DataFrame:
The read_csv() is capable of reading from a binary stream as well. The Python example code below constructs a bytes literal and creates a BytesIO stream out of it. The byte stream is passed to the read_csv() method which parses the bytes from the stream and loads into a DataFrame.
# Example Python program to load data from a binary stream of CSV
# into a pandas DataFrame
import pandas as pds
# Create a CSV bytes literal
csvData = b"Symbol, Price \r\n Example1, 10 \r\n Example2, 20";
# Make a binary stream
binaryStream = io.BytesIO(csvData);
# Make pandas to read the CSV from the binary stream and load it into a DataFrame
dataFrame = pds.read_csv(binaryStream);
print("Contents of the DataFrame loaded from a binary stream of CSV:")
Contents of the DataFrame loaded from a binary stream of CSV:
0 Example1 10
1 Example2 20