Pandas Data Exploration Cheat Sheet

Table of Contents

1. Core Concepts
2. Typical Usage Methods
3. Common Practices
4. Best Practices
5. Code Examples
6. Conclusion
7. FAQ
8. References

Core Concepts

DataFrames and Series

• DataFrame: A two - dimensional labeled data structure with columns of potentially different types. It is similar to a spreadsheet or a SQL table. For example, it can represent a table of students with columns for their names, ages, and grades.
• Series: A one - dimensional labeled array capable of holding any data type. A single column of a DataFrame is a Series. For instance, the “ages” column of the students’ DataFrame would be a Series.

Indexing

• Label - based indexing: Uses the row and column labels to access data. For example, using the student’s name (a label) to access their grade.
• Integer - based indexing: Uses integer positions to access data, similar to traditional Python list indexing.

Missing Values

Data often contains missing values, represented as NaN (Not a Number) in pandas. Handling missing values is an important part of data exploration.

Typical Usage Methods

Reading Data

import pandas as pd

# Read a CSV file
df = pd.read_csv('data.csv')

# Read an Excel file
df = pd.read_excel('data.xlsx')

Inspecting Data

# View the first few rows
print(df.head())

# View the last few rows
print(df.tail())

# Get basic information about the DataFrame
print(df.info())

# Get the shape of the DataFrame (rows, columns)
rows, columns = df.shape

# Get descriptive statistics
print(df.describe())

Selecting Data

# Select a single column
column = df['column_name']

# Select multiple columns
columns = df[['column1', 'column2']]

# Select rows by label
row = df.loc['row_label']

# Select rows by integer position
row = df.iloc[0]

Handling Missing Values

# Check for missing values
print(df.isnull().sum())

# Drop rows with missing values
df = df.dropna()

# Fill missing values with a specific value
df = df.fillna(0)

Common Practices

Checking Data Types

print(df.dtypes)

This helps in identifying if the data types of columns are appropriate. For example, a column that should be numeric might be read as a string.

Unique Values

print(df['column_name'].unique())

This is useful for categorical variables to understand the different categories present.

Value Counts

print(df['column_name'].value_counts())

It shows the frequency of each unique value in a column.

Best Practices

Use Chaining

df = pd.read_csv('data.csv').dropna().head()

Chaining operations makes the code more concise and readable.

Be Cautious with In - Place Operations

In - place operations modify the original DataFrame. It’s better to create a new DataFrame when possible to avoid unexpected changes.

Document Your Steps

Add comments to your code to explain what each step is doing, especially when performing complex data exploration tasks.

Code Examples

Example 1: Exploring a Sales Dataset

import pandas as pd

# Read the sales data
sales_df = pd.read_csv('sales_data.csv')

# Check the basic information
print("Basic Information:")
sales_df.info()

# Check the number of missing values
missing_values = sales_df.isnull().sum()
print("\nMissing Values:")
print(missing_values)

# Get the descriptive statistics of the 'sales' column
sales_stats = sales_df['sales'].describe()
print("\nSales Statistics:")
print(sales_stats)

# Find the unique regions
regions = sales_df['region'].unique()
print("\nUnique Regions:")
print(regions)

Conclusion

A pandas data exploration cheat sheet is an invaluable tool for Python developers working with data. By understanding the core concepts, typical usage methods, common practices, and best practices, developers can efficiently explore data, identify potential issues, and lay a solid foundation for further data analysis and modeling.

FAQ

Q1: Can I use pandas to read data from a database?

Yes, pandas provides functions like read_sql to read data from various databases such as MySQL, PostgreSQL, etc.

Q2: How can I handle outliers during data exploration?

You can use statistical methods like the inter - quartile range (IQR) to identify outliers and then decide whether to remove or transform them.

Q3: Is it possible to perform data exploration on a large dataset?

Yes, but you may need to use techniques like sampling or chunking to manage memory usage.

References

• Pandas official documentation: https://pandas.pydata.org/docs/
• “Python for Data Analysis” by Wes McKinney

• ←Previous
• Next→