Introduction to Data Analysis Using Python

Course: Python Training

Topic: Introduction to Data Analysis Using Python

Course Link: Python Training Course at JustAcademy

Demo Registration: Register for a Course Demo

Official Website: JustAcademy

1. Introduction to Data Analysis

Data analysis is the process of collecting, inspecting, cleaning, transforming, and interpreting data to discover useful information, identify patterns, answer questions, and support decision-making. Organizations generate data through sales transactions, customer interactions, website activity, surveys, financial records, and operational systems. Analyzing this information helps them understand performance and make evidence-based decisions.

Python is widely used for data analysis because it has readable syntax, flexible data structures, and libraries for numerical calculations, data manipulation, statistical analysis, and visualization. Python enables analysts to automate repetitive tasks and work with datasets ranging from small spreadsheets to large collections of records.

For example, a retail company can use Python to analyze monthly sales, identify its best-selling products, calculate average order values, and compare performance across different locations.

Applications of Data Analysis

• Business analysis: Measuring revenue, expenses, profits, and business growth.
• Marketing analysis: Evaluating campaigns, customer engagement, and conversion rates.
• Financial analysis: Examining transactions, budgets, and spending patterns.
• Healthcare analysis: Studying authorized and appropriately protected healthcare datasets.
• Education analysis: Understanding student performance and course completion.
• E-commerce analysis: Identifying popular products and customer purchasing patterns.
• Operations analysis: Monitoring inventory, delivery times, and productivity.

2. Learning Objectives

After studying this topic, learners should be able to:

• Understand the data analysis workflow.
• Identify common data types and data sources.
• Use Python libraries such as Pandas, NumPy, and Matplotlib.
• Create and inspect Pandas Series and DataFrames.
• Load data from CSV and Excel files.
• Identify missing values and duplicate records.
• Filter, sort, group, and summarize data.
• Calculate basic descriptive statistics.
• Create charts to communicate findings.
• Complete a small data analysis project.

3. Understanding the Data Analysis Workflow

Data analysis is a structured process. The exact steps depend on the problem, but most projects involve defining a question, collecting data, cleaning it, exploring it, performing analysis, communicating results, and validating conclusions.

Why Is a Structured Workflow Important?

• It keeps the analysis focused on a meaningful question.
• It helps reveal data quality problems early.
• It improves reproducibility and consistency.
• It reduces the risk of drawing conclusions from incorrect data.
• It makes findings easier to explain and verify.

4. Setting Up the Python Data Analysis Environment

Python data analysis can be performed in Jupyter Notebook, JupyterLab, an IDE such as Visual Studio Code, or a regular Python script. Notebook environments are particularly useful for learning because code, output, explanations, and charts can appear together.

Installing Required Libraries

python -m pip install pandas numpy matplotlib openpyxl

This command installs Pandas, NumPy, Matplotlib, and OpenPyXL. OpenPyXL can be used by Pandas to read or write many Excel workbook files in the XLSX format.

Importing the Libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

5. Understanding Data Types

Before analyzing a dataset, identify the type of information stored in each column. Incorrect data types can cause calculation errors or make filtering and sorting unreliable.

Common Data Categories

Checking Python Data Types

quantity = 5
price = 249.99
product = "Keyboard"
available = True

print(type(quantity))
print(type(price))
print(type(product))
print(type(available))

Output:

<class 'int'>
<class 'float'>
<class 'str'>
<class 'bool'>

6. Introduction to Pandas

Pandas is a Python library designed for working with structured and labeled data. It provides tools for loading files, selecting rows and columns, cleaning data, calculating statistics, grouping records, and combining datasets.

Two fundamental Pandas structures are the Series and the DataFrame.

Creating a Series

A Series is a one-dimensional labeled data structure.

import pandas as pd

sales = pd.Series([1200, 1500, 1800, 1350])

print(sales)

A Series automatically receives an index, starting at zero by default.

Creating a DataFrame

A DataFrame is a two-dimensional table with rows and columns. Each column can contain a different data type.

import pandas as pd

data = {
    "Product": ["Laptop", "Keyboard", "Mouse", "Monitor"],
    "Quantity": [3, 10, 15, 5],
    "Price": [55000, 1200, 500, 12000]
}

df = pd.DataFrame(data)

print(df)

Output:

    Product  Quantity  Price
0    Laptop         3  55000
1  Keyboard        10   1200
2     Mouse        15    500
3   Monitor         5  12000

Series vs. DataFrame

7. Loading Data from Files

Real-world analysis often begins with data stored in CSV files, Excel workbooks, or JSON documents. Pandas provides functions for reading these formats into a DataFrame.

Reading a CSV File

import pandas as pd

df = pd.read_csv("sales.csv")

print(df.head())

Reading an Excel File

df = pd.read_excel("sales.xlsx")

print(df.head())

Reading a JSON File

df = pd.read_json("sales.json")

print(df.head())

Useful File-Loading Options

Check that the file path is correct and that the file format matches the function being used. For large datasets, loading only the necessary columns or rows can reduce memory usage.

8. Inspecting and Understanding a Dataset

After loading data, inspect its dimensions, column names, data types, and sample records. This helps you understand what the dataset contains before performing calculations.

print(df.head())       # First five rows
print(df.tail())       # Last five rows
print(df.shape)        # Number of rows and columns
print(df.columns)      # Column names
print(df.dtypes)       # Data types
print(df.info())       # Structure and non-null counts
print(df.describe())   # Summary statistics for numeric columns

Important Inspection Methods

9. Data Cleaning and Preparation

Data cleaning involves identifying and correcting problems that can affect the accuracy of an analysis. Real datasets may contain missing values, duplicate rows, inconsistent text, invalid numbers, or incorrect date formats.

Checking Missing Values

print(df.isna().sum())

This returns the number of missing values in each column.

Removing Duplicate Rows

df = df.drop_duplicates()

Remove duplicates only after determining that the records are genuinely repeated. Two rows that look similar may represent separate legitimate transactions.

Handling Missing Values

# Remove rows that contain missing values
clean_df = df.dropna()

# Fill missing values in a numeric column
df["Sales"] = df["Sales"].fillna(0)

Filling missing sales values with zero is appropriate only when a missing value truly means no sales. If the value is unknown, replacing it with zero could distort the analysis. Choose a strategy based on the meaning of the data.

Converting Data Types

df["Sales"] = pd.to_numeric(df["Sales"], errors="coerce")
df["OrderDate"] = pd.to_datetime(df["OrderDate"], errors="coerce")

Using errors="coerce" converts unparseable values to missing values. Inspect those records before deciding whether to correct, exclude, or retain them.

10. Selecting Rows and Columns

Pandas offers several ways to select the information required for analysis.

Selecting One Column

products = df["Product"]
print(products)

Selecting Multiple Columns

subset = df[["Product", "Sales"]]
print(subset)

Selecting Rows with loc and iloc

# Select rows using index labels
print(df.loc[0])

# Select the first row by position
print(df.iloc[0])

# Select the first three rows and first two columns
print(df.iloc[0:3, 0:2])

loc selects by index labels, while iloc selects by integer position. The distinction matters when a DataFrame has a custom index.

11. Filtering Data

Filtering helps isolate records that meet specific conditions. It is useful when investigating high-value sales, selected regions, specific products, or transactions within a date range.

# Find rows where sales exceed 10000
high_sales = df[df["Sales"] > 10000]

# Find records for a specific region
west_sales = df[df["Region"] == "West"]

# Combine conditions
filtered = df[
    (df["Sales"] > 5000) &
    (df["Region"] == "West")
]

print(filtered)

Use & for AND conditions and | for OR conditions in Pandas boolean filters. Put each comparison inside parentheses.

Using isin()

selected = df[df["Region"].isin(["West", "North"])]

Using between()

medium_sales = df[df["Sales"].between(5000, 15000)]

12. Sorting and Ranking Data

Sorting organizes records by one or more columns. It can reveal top-performing products, the largest transactions, or the most recent activity.

# Sort by sales in ascending order
ascending = df.sort_values("Sales")

# Sort from highest sales to lowest
descending = df.sort_values("Sales", ascending=False)

# Sort by region and then sales
sorted_df = df.sort_values(
    ["Region", "Sales"],
    ascending=[True, False]
)

print(descending)

To select the highest sales values, you can also use:

top_sales = df.nlargest(5, "Sales")
print(top_sales)

13. Basic Statistical Analysis

Descriptive statistics summarize a dataset and help analysts understand its central values, variation, and distribution.

Example: Calculating Statistics

sales = pd.Series([1200, 1500, 1800, 1350, 2100])

print("Total:", sales.sum())
print("Mean:", sales.mean())
print("Median:", sales.median())
print("Minimum:", sales.min())
print("Maximum:", sales.max())

Output:

Total: 7950
Mean: 1590.0
Median: 1500.0
Minimum: 1200
Maximum: 2100

Statistics should be interpreted in context. For example, a mean can be affected by extreme values, so the median may provide a better picture of a typical transaction in a highly skewed dataset.

14. Grouping and Aggregating Data

Grouping allows analysts to summarize records by categories such as product, region, department, or month. Pandas uses groupby() to group records and apply calculations to each group.

data = {
    "Region": ["West", "West", "North", "North", "South"],
    "Sales": [12000, 15000, 9000, 11000, 13000]
}

df = pd.DataFrame(data)

regional_sales = df.groupby("Region")["Sales"].sum()

print(regional_sales)

Output:

Region
North    20000
South    13000
West     27000
Name: Sales, dtype: int64

Using Multiple Aggregations

summary = df.groupby("Region").agg(
    total_sales=("Sales", "sum"),
    average_sales=("Sales", "mean"),
    transaction_count=("Sales", "count")
)

print(summary)

This example calculates total sales, average sales, and transaction counts for each region. Grouped summaries are especially useful for business reporting.

15. Creating New Columns

New columns can be calculated from existing fields. This is useful for creating revenue, profit, tax, discount, or performance metrics.

data = {
    "Product": ["Laptop", "Mouse", "Keyboard"],
    "Quantity": [2, 10, 5],
    "UnitPrice": [50000, 500, 1200]
}

df = pd.DataFrame(data)

df["Revenue"] = df["Quantity"] * df["UnitPrice"]

print(df)

Calculating Profit

df["Cost"] = [80000, 3000, 4500]
df["Profit"] = df["Revenue"] - df["Cost"]

print(df[["Product", "Revenue", "Cost", "Profit"]])

Before creating calculated columns, verify the units and meaning of each input. For example, revenue should not be confused with profit, and percentages should be represented consistently.

16. Working with Dates

Date analysis helps identify seasonal trends, monthly performance, delivery delays, and changes over time. Convert date columns to a datetime type before using date-specific operations.

df = pd.DataFrame({
    "OrderDate": ["2026-01-05", "2026-02-10", "2026-02-22"],
    "Sales": [12000, 15000, 9000]
})

df["OrderDate"] = pd.to_datetime(df["OrderDate"])

df["Year"] = df["OrderDate"].dt.year
df["Month"] = df["OrderDate"].dt.month
df["MonthName"] = df["OrderDate"].dt.month_name()

print(df)

Monthly Sales Summary

df["Month"] = df["OrderDate"].dt.to_period("M")

monthly_sales = df.groupby("Month")["Sales"].sum()

print(monthly_sales)

Check date formats and time zones when analyzing data from multiple systems. Incorrect date parsing can move records into the wrong reporting period.

17. Introduction to NumPy

NumPy is a Python library for numerical computing. Its main data structure is the multidimensional array, called an ndarray. NumPy supports efficient element-wise operations, mathematical functions, array indexing, and statistical calculations.

Creating NumPy Arrays

import numpy as np

numbers = np.array([10, 20, 30, 40])

print(numbers)

Performing Calculations

numbers = np.array([10, 20, 30, 40])

print(numbers + 5)
print(numbers * 2)
print(np.mean(numbers))
print(np.max(numbers))
print(np.min(numbers))

Output:

[15 25 35 45]
[20 40 60 80]
25.0
40
10

Boolean Filtering in NumPy

numbers = np.array([10, 20, 30, 40, 50])

result = numbers[numbers > 25]

print(result)

Output:

[30 40 50]

NumPy is especially useful for numerical operations, while Pandas adds labeled columns, row indexes, and convenient tools for working with tabular data.

18. Combining Datasets

Sometimes information is stored in multiple tables. For example, one dataset may contain order information while another contains customer details. Pandas provides merge() and concat() to combine data.

Using Merge

orders = pd.DataFrame({
    "CustomerID": [1, 2, 3],
    "Sales": [1200, 1800, 900]
})

customers = pd.DataFrame({
    "CustomerID": [1, 2, 3],
    "CustomerName": ["Asha", "Ravi", "Neha"]
})

merged = pd.merge(
    orders,
    customers,
    on="CustomerID",
    how="inner"
)

print(merged)

Using Concatenation

january = pd.DataFrame({"Sales": [1000, 1500]})
february = pd.DataFrame({"Sales": [1800, 1200]})

combined = pd.concat(
    [january, february],
    ignore_index=True
)

print(combined)

Use merge() to combine tables based on matching keys. Use concat() to stack compatible tables vertically or combine them along another axis. Check for unmatched keys and duplicate records after combining datasets.

19. Introduction to Data Visualization

Data visualization represents information graphically. Charts can make trends, comparisons, distributions, and relationships easier to understand than raw tables alone.

Creating a Bar Chart

import matplotlib.pyplot as plt

products = ["Laptop", "Mouse", "Keyboard"]
sales = [55000, 5000, 12000]

plt.bar(products, sales)
plt.title("Sales by Product")
plt.xlabel("Product")
plt.ylabel("Sales Amount")
plt.tight_layout()
plt.show()

Creating a Line Chart

months = ["Jan", "Feb", "Mar", "Apr"]
sales = [12000, 15000, 14000, 18000]

plt.plot(months, sales, marker="o")
plt.title("Monthly Sales Trend")
plt.xlabel("Month")
plt.ylabel("Sales Amount")
plt.tight_layout()
plt.show()

Creating a Histogram

order_values = [250, 400, 350, 700, 900, 450, 500, 1200]

plt.hist(order_values, bins=5)
plt.title("Distribution of Order Values")
plt.xlabel("Order Value")
plt.ylabel("Frequency")
plt.tight_layout()
plt.show()

Choose the chart that answers the question being investigated. Include clear labels and units, avoid misleading scales, and do not rely on a chart alone when the underlying numbers or sample size are important.

20. Exploratory Data Analysis (EDA)

Exploratory Data Analysis is the process of examining a dataset to understand its structure, identify patterns, discover anomalies, and form questions for deeper analysis. EDA usually combines summary statistics, visualizations, and data quality checks.

Common EDA Tasks

• Inspect the number of rows and columns.
• Review column types and missing values.
• Calculate descriptive statistics.
• Check duplicates and unusual values.
• Explore relationships between variables.
• Compare categories using grouped summaries.
• Visualize distributions and trends.

# General dataset inspection
print(df.shape)
print(df.head())
print(df.info())
print(df.describe())

# Missing values
print(df.isna().sum())

# Correlations between numeric columns
print(df.select_dtypes(include="number").corr())

Correlation measures how two numerical variables vary together. Correlation alone does not prove that one variable causes changes in another. Interpret it alongside domain knowledge, data quality, and other evidence.

21. Mini Project: Sales Data Analysis

Project Objective: Analyze a small sales dataset to calculate revenue, compare products, summarize sales by region, and visualize the results.

Step 1: Create the Dataset

import pandas as pd
import matplotlib.pyplot as plt

data = {
    "Product": ["Laptop", "Mouse", "Keyboard", "Laptop", "Mouse", "Monitor"],
    "Region": ["West", "North", "West", "South", "North", "South"],
    "Quantity": [2, 10, 5, 1, 8, 3],
    "UnitPrice": [50000, 500, 1200, 50000, 500, 12000]
}

df = pd.DataFrame(data)

print(df)

Step 2: Calculate Revenue

df["Revenue"] = df["Quantity"] * df["UnitPrice"]

print(df[["Product", "Quantity", "UnitPrice", "Revenue"]])

Step 3: Calculate Total Revenue

total_revenue = df["Revenue"].sum()

print("Total Revenue:", total_revenue)

For this sample dataset, the total revenue is 207,600.

Step 4: Find Revenue by Product

product_revenue = (
    df.groupby("Product")["Revenue"]
    .sum()
    .sort_values(ascending=False)
)

print(product_revenue)

Step 5: Compare Regions

region_revenue = df.groupby("Region")["Revenue"].sum()

print(region_revenue)

Step 6: Visualize Product Revenue

product_revenue.plot(kind="bar")
plt.title("Revenue by Product")
plt.xlabel("Product")
plt.ylabel("Revenue")
plt.tight_layout()
plt.show()

Step 7: Interpret the Results

• Identify which product contributes the most revenue.
• Compare revenue totals between regions.
• Check whether high revenue comes from high quantities, high prices, or both.
• Consider whether the dataset is large and representative enough to support a business decision.

This is a small learning dataset. Its results illustrate analysis methods but should not be treated as evidence about an actual company's performance.

22. Common Mistakes in Data Analysis

23. Best Practices for Python Data Analysis

• Start with a clear question and define the metrics needed to answer it.
• Keep a clean copy of the original data when practical.
• Use descriptive variable and column names.
• Inspect data before changing or deleting records.
• Use vectorized Pandas and NumPy operations when appropriate.
• Break long scripts into reusable functions.
• Document assumptions and data cleaning decisions.
• Validate calculations against known totals or sample records.
• Use charts with meaningful titles, labels, and units.
• Protect sensitive information and follow applicable data governance rules.
• Communicate limitations as well as findings.

24. Practice Exercises

1. Create a Pandas DataFrame containing five products, quantities, and prices.
2. Calculate the total revenue for each product.
3. Load a CSV file and display its first ten rows.
4. Find missing values in every column.
5. Remove confirmed duplicate records.
6. Filter rows where revenue exceeds a chosen threshold.
7. Sort products by revenue in descending order.
8. Calculate the mean, median, minimum, and maximum of a numerical column.
9. Group sales by region and calculate the total for each region.
10. Convert a date column to datetime and summarize sales by month.
11. Create a bar chart comparing revenue across products.
12. Combine customer and order datasets using a common identifier.
13. Write a short summary of three findings from a dataset and mention its limitations.

25. Interview Questions and Answers

Q1. What is data analysis?

Data analysis is the process of inspecting, cleaning, transforming, and interpreting data to discover useful information and support decisions.

Q2. Why is Python used for data analysis?

Python offers readable syntax, automation capabilities, and libraries such as Pandas, NumPy, and Matplotlib for manipulating, calculating, and visualizing data.

Q3. What is Pandas?

Pandas is a Python library for working with structured data. It provides tools for data loading, cleaning, filtering, grouping, and analysis.

Q4. What is a DataFrame?

A DataFrame is a two-dimensional labeled data structure containing rows and columns. Its columns can hold different data types.

Q5. What is a Series?

A Series is a one-dimensional labeled array that can hold values of different supported data types.

Q6. What is NumPy used for?

NumPy provides efficient multidimensional arrays and numerical operations, including mathematical and statistical functions.

Q7. What is the difference between loc and iloc?

loc selects data using index labels, while iloc selects data using integer positions.

Q8. How do you identify missing values in Pandas?

Use df.isna() to identify missing values and df.isna().sum() to count them by column.

Q9. What does groupby() do?

It groups records by one or more categories and allows calculations such as sums, averages, and counts for each group.

Q10. What is Exploratory Data Analysis?

EDA is the process of inspecting data using summaries, visualizations, and quality checks to understand patterns, relationships, and unusual observations.

Q11. What is the difference between mean and median?

The mean is the arithmetic average, while the median is the middle value in sorted data. The median is generally less affected by extreme values.

Q12. Why is data visualization important?

Visualization helps communicate trends, comparisons, distributions, and relationships in a way that can be easier to interpret than raw tables.

Q13. What is the difference between merge() and concat()?

merge() combines datasets using matching keys or columns, while concat() combines objects along a specified axis.

Q14. What is correlation?

Correlation measures the degree and direction of association between variables. It does not, by itself, establish causation.

Q15. How do you make analysis results reliable?

Validate data types, investigate missing values and duplicates, document assumptions, check calculations, and communicate limitations clearly.

26. Quick Revision Checklist

• Data analysis converts raw data into useful insights.
• The typical workflow includes defining a question, collecting, cleaning, exploring, analyzing, and communicating data.
• Pandas provides Series and DataFrame structures.
• NumPy supports efficient numerical calculations.
• Matplotlib creates charts and graphs.
• read_csv() and read_excel() load common data files.
• head(), info(), and describe() help inspect datasets.
• isna() and drop_duplicates() support data quality checks.
• Filtering and sorting help identify relevant records.
• groupby() summarizes data by category.
• Descriptive statistics include mean, median, minimum, maximum, and standard deviation.
• Visualizations help explain patterns and comparisons.
• Good analysis requires validation, documentation, and responsible data handling.

27. Conclusion

Introduction to Data Analysis Using Python provides the foundation for working with real-world datasets. By learning Pandas, NumPy, and Matplotlib, beginners can load data, inspect its structure, clean inconsistencies, perform calculations, group records, and communicate findings through visualizations.

The best way to develop these skills is through practice. Start with small datasets, ask clear questions, validate the results, and gradually work toward complete projects that combine data preparation, analysis, visualization, and written conclusions.

Explore Python Training: View the Python Training Course

Register for a Demo: Register for a Course Demo

Visit JustAcademy: Explore JustAcademy

Additional Learning Resource: Pandas Official Getting Started Guide

Additional Learning Resource: NumPy Beginner's Guide