Sorting, Removing Duplicates, and Plotting in Pandas

When working with data analysis in Python, pandas is the most popular library. Two essential skills are:

1. Sorting data – ordering it for analysis
2. Handling duplicates – removing repeated entries
3. Plotting – visualizing trends and patterns

Let’s break this down for a novice with clear examples.

1️⃣ Sorting Data in Pandas

Sorting helps you organize your data by a column or multiple columns.

Example Dataset

import pandas as pd

data = {
    "Product": ["A", "B", "C", "D"],
    "Sales": [1500, 700, 2000, 700],
    "Profit": [300, 100, 500, 50]
}

df = pd.DataFrame(data)
print(df)

Output Table:

| Product | Sales | Profit |
|---------|-------|--------|
| A       | 1500  | 300    |
| B       | 700   | 100    |
| C       | 2000  | 500    |
| D       | 700   | 50     |

Sort by Single Column

Sort by Sales ascending:

df_sorted = df.sort_values(by="Sales")
print(df_sorted)

Sort by Sales descending:

df_sorted_desc = df.sort_values(by="Sales", ascending=False)
print(df_sorted_desc)

Sort by Multiple Columns

Sort by Sales ascending and then Profit descending:

df_sorted_multi = df.sort_values(by=["Sales", "Profit"], ascending=[True, False])
print(df_sorted_multi)

2️⃣ Removing Duplicates

Sometimes, datasets have repeated rows.

Example Dataset with Duplicates

data_dup = {
    "Product": ["A", "B", "C", "B", "D"],
    "Sales": [1500, 700, 2000, 700, 700]
}

df_dup = pd.DataFrame(data_dup)
print(df_dup)

Output Table:

| Product | Sales |
|---------|-------|
| A       | 1500  |
| B       | 700   |
| C       | 2000  |
| B       | 700   |
| D       | 700   |

Remove Duplicate Rows

df_no_dup = df_dup.drop_duplicates()
print(df_no_dup)

Keep First or Last

df_no_dup_first = df_dup.drop_duplicates(keep='first')  # default
df_no_dup_last = df_dup.drop_duplicates(keep='last')

Remove Duplicates Based on a Column

df_no_dup_product = df_dup.drop_duplicates(subset=["Product"])
print(df_no_dup_product)

3️⃣ Basic Plotting with Pandas

Pandas integrates easily with Matplotlib to create quick visualizations.

3.1 Line Plot

import matplotlib.pyplot as plt

df.plot(x="Product", y="Sales", kind="line", title="Sales Line Plot")
plt.show()

3.2 Bar Plot

df.plot(x="Product", y="Sales", kind="bar", color="skyblue", title="Sales Bar Chart")
plt.show()

3.3 Scatter Plot

df.plot(x="Sales", y="Profit", kind="scatter", color="red", title="Sales vs Profit")
plt.show()

3.4 Histogram (Distribution of Values)

df["Sales"].plot(kind="hist", bins=5, color="orange", title="Sales Distribution")
plt.show()

3.5 Pie Chart

df.plot.pie(y="Sales", labels=df["Product"], autopct="%1.1f%%", title="Sales Distribution Pie")
plt.show()

4️⃣ Full Example – Sorting, Removing Duplicates, Plotting