Table of Contents

Haseeb

Haseeb

python tutorial

Complete Python Tutorial for Beginners

Python's simplicity, readability, and versatility make it an excellent choice for a wide range of programming tasks, from simple scripts to complex software development projects. Whether you're a beginner learning to code or an experienced developer looking to build advanced applications, Python has something to offer for everyone.

Getting Started

				
					print("Hello, World!")

This program prints the message “Hello, World!” to the console when executed. It’s often the first program beginners write when learning a new programming language, as it’s a simple way to verify that everything is set up correctly and the environment is ready for coding.

Python Syntax

				
					>>> print("Hello, World!")
Hello, World!

Understanding and adhering to Python syntax rules is essential for writing correct and readable code. Consistent and clear syntax improves code maintainability, readability, and collaboration among developers.

Python Comments

				
					# This is a single-line comment
x = 10  # Assigning the value 10 to the variable x

Comments play a crucial role in code readability, maintenance, and collaboration. They help other developers understand the purpose and functionality of code segments, making it easier to debug, modify, and extend Python programs.

Python Variables

				
					x = 10            # Integer variable
name = "John"     # String variable
is_true = True    # Boolean variable

Variables are fundamental building blocks in Python programming and are used extensively to store, manipulate, and represent data in code. Understanding how to use variables effectively is essential for writing clean, readable, and maintainable Python code.

Python Data Types

				
					# Numeric Types
x = 10                  # int
y = 3.14                # float
z = 5 + 2j              # complex

# Sequence Types
name = "John"           # str
numbers = [1, 2, 3, 4]  # list
coordinates = (10, 20)  # tuple

# Boolean Type
is_true = True          # bool

# Mapping Type
person = {'name': 'Alice', 'age': 30}  # dict

# Set Types
unique_numbers = {1, 2, 3, 4, 5}        # set
frozen_numbers = frozenset({1, 2, 3})   # frozenset

# None Type
empty_value = None      # None

# Printing the data types of variables
print(type(x))          # <class 'int'>
print(type(y))          # <class 'float'>
print(type(z))          # <class 'complex'>
print(type(name))       # <class 'str'>
print(type(numbers))    # <class 'list'>
print(type(coordinates))# <class 'tuple'>
print(type(is_true))    # <class 'bool'>
print(type(person))     # <class 'dict'>
print(type(unique_numbers)) # <class 'set'>
print(type(frozen_numbers)) # <class 'frozenset'>
print(type(empty_value)) # <class 'NoneType'>

Python is a dynamically typed language, meaning that variables do not have predefined types, and their types are determined dynamically based on the values assigned to them. Additionally, Python provides built-in functions such as type() to determine the data type of a variable or value.

Understanding Python’s data types is essential for effectively working with data, manipulating values, and writing efficient and readable code. Each data type has its own set of operations and methods for performing various tasks, so choosing the appropriate data type is crucial for writing efficient and bug-free Python programs.

Python Casting

				
					x = 3.14
y = int(x)  # y will be 3
x = 10
y = str(x)  # y will be "10"
x = 0
y = bool(x)  # y will be False

z = "Hello"
w = bool(z)  # w will be True

Casting allows you to manipulate and transform data in Python to suit your requirements. It’s essential to use casting functions appropriately to ensure compatibility and consistency in your code.

Python Strings

				
					a = '''Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
sed do eiusmod tempor incididunt
ut labore et dolore magna aliqua.'''
print(a)

Strings are immutable in Python, meaning their values cannot be changed after creation. However, you can create new strings by manipulating existing ones using various methods and operations. Understanding string manipulation is essential for working with text data and developing Python applications.

Python Operators

				
					#arthmetic opr
x = 10
y = 3
addition = x + y   # 13
division = x / y   # 3.3333333333333335
#assigment opr 
x = 5
x += 3   # x is now 8
#comparison oprx = 5
y = 3
result = x > y   # True
#logical opr 
x = True
y = False
result = x and y   # False

Understanding and using operators effectively is fundamental in programming, as they enable you to perform various tasks, such as arithmetic calculations, logical evaluations, and data manipulation.

Python Lists

				
					#list 
my_list = [1, 2, 3, 4, 5]
mixed_list = [1, 'hello', 3.14, True]
empty_list = []
#accessing items 
my_list = ['apple', 'banana', 'cherry', 'orange']
first_element = my_list[0]     # 'apple'
last_element = my_list[-1]      # 'orange'
sublist = my_list[1:3]          # ['banana', 'cherry']
#list opr 
list1 = [1, 2, 3]
list2 = [4, 5, 6]
concatenated_list = list1 + list2    # [1, 2, 3, 4, 5, 6]
repeated_list = list1 * 3            # [1, 2, 3, 1, 2, 3, 1, 2, 3]
length = len(list1)                  # 3
contains_2 = 2 in list1              # True

Lists are versatile data structures in Python and are widely used for storing and manipulating collections of items. Understanding lists and their operations is essential for effective Python programming.

Python Tuples

				
					#tuple 
my_tuple = (1, 2, 3, 4, 5)
mixed_tuple = (1, 'hello', 3.14, True)
single_element_tuple = (42,)  # Note the comma to indicate a single element tuple
empty_tuple = ()
#accessing items 
my_tuple = ('apple', 'banana', 'cherry', 'orange')
first_element = my_tuple[0]     # 'apple'
last_element = my_tuple[-1]      # 'orange'
sublist = my_tuple[1:3]          # ('banana', 'cherry')
#tuple opr 
tuple1 = (1, 2, 3)
tuple2 = (4, 5, 6)
concatenated_tuple = tuple1 + tuple2    # (1, 2, 3, 4, 5, 6)
repeated_tuple = tuple1 * 3            # (1, 2, 3, 1, 2, 3, 1, 2, 3)
length = len(tuple1)                  # 3
contains_2 = 2 in tuple1              # True

Tuples are often used to represent fixed collections of items where immutability is desired. They are commonly used for returning multiple values from functions, as dictionary keys (since they are immutable), and in cases where data integrity is important.

Python Sets

				
					#set 
my_set = {1, 2, 3, 4, 5}
mixed_set = {1, 'hello', 3.14, True}
empty_set = set()
#accessing items 
my_set = {1, 2, 3}
for item in my_set:
    print(item)
#sset opr 
set1 = {1, 2, 3}
set2 = {3, 4, 5}
union_set = set1 | set2              # Union of sets {1, 2, 3, 4, 5}
intersection_set = set1 & set2       # Intersection of sets {3}
difference_set = set1 - set2         # Difference of sets {1, 2}
symmetric_difference_set = set1 ^ set2   # Symmetric difference of sets {1, 2, 4, 5}

Sets are useful for eliminating duplicate elements from a collection, performing mathematical set operations, and testing for membership in a collection efficiently. However, since sets are unordered, they do not support indexing or slicing.

Python Dictionaries

				
					#dic 
my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}
empty_dict = {}
#accessing key values 
my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}
name = my_dict['name']     # 'John'
age = my_dict.get('age')   # 30
#modify dic 
my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}
my_dict['age'] = 35             # Modify the value of an existing key
my_dict['gender'] = 'Male'      # Add a new key-value pair
del my_dict['city']             # Delete a key-value pair
#dic methods 
my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}
keys = my_dict.keys()             # Get all keys
values = my_dict.values()         # Get all values
items = my_dict.items()           # Get all key-value pairs
my_dict.update({'age': 35})       # Update a key-value pair
my_dict.pop('city')               # Remove and return the value of a specific key
#dic iteration 
my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}
for key in my_dict:
    print(key, my_dict[key])      # Print keys and corresponding values

for value in my_dict.values():
    print(value)                  # Print values

for key, value in my_dict.items():
    print(key, value)             # Print key-value pairs

Dictionaries are versatile data structures in Python and are commonly used for storing and organizing data with meaningful keys. They provide a convenient way to represent mappings between keys and values, making them essential for many programming tasks.

Python If ... Else

				
					if condition:
    # Code block to execute if condition is True
else:
    # Code block to execute if condition is False


#example 

x = 10
if x > 5:
    print("x is greater than 5")
else:
    print("x is not greater than 5")

The if…else statement is fundamental in Python programming as it allows you to make decisions based on conditions, thus controlling the flow of your program.

While Loops

				
					while condition:
    # Code block to execute as long as the condition is true


#example 

i = 1
while i <= 5:
    print(i)
    i += 1

While loops are useful when you need to execute a block of code repeatedly until a certain condition is met. However, you should be cautious to avoid infinite loops and consider using for loops when the number of iterations is known in advance.

For Loops

				
					for item in sequence:
    # Code block to execute for each item in the sequence



#example 

fruits = ["apple", "banana", "cherry"]
for fruit in fruits:
    print(fruit)

For loops are commonly used in Python for iterating over sequences and performing repetitive tasks. They are versatile and can be applied to various data structures and scenarios, making them essential in many programming situations.

Python Functions

				
					def greet():
    print("Hello, world!")
greet()  # Output: Hello, world!


def greet(name):
    print("Hello,", name)

greet("John")  # Output: Hello, John



def add(x, y):
    return x + y

result = add(3, 5)  # result = 8

Functions are essential building blocks in Python programming, allowing you to write reusable and modular code. By breaking down your program into functions, you can improve readability, maintainability, and reusability of your code.

Python PIP 

				
					python -m pip install --upgrade pip


pip install package_name


pip list


pip install -r requirements.txt

PIP simplifies the process of installing and managing Python packages, making it easier for developers to leverage the vast ecosystem of libraries and tools available in the Python community.

Getting started with Numpy

NumPy is a powerful Python library for numerical computing that provides support for large, multi-dimensional arrays and matrices, along with a collection of mathematical functions to operate on these arrays efficiently. It is widely used in scientific computing, data analysis, machine learning, and more.

Creating Arrays

				
					import numpy as np

arr1 = np.array([1, 2, 3, 4, 5])  # 1D array
arr2 = np.array([[1, 2, 3], [4, 5, 6]])  # 2D array

zeros_arr = np.zeros((3, 3))  # 3x3 array filled with zeros
ones_arr = np.ones((2, 4))    # 2x4 array filled with ones


arr_range = np.arange(0, 10, 2)  # Array from 0 to 9 with step 2


arr_linspace = np.linspace(0, 10, 5)  # Array with 5 equally spaced values from 0 to 10

Let’s break down each method:

  1. Using np.array():

    • This method creates a NumPy array from a Python list or tuple.
    • For example, np.array([1, 2, 3, 4, 5]) creates a 1-dimensional array containing the elements [1, 2, 3, 4, 5].
    • Similarly, np.array([[1, 2, 3], [4, 5, 6]]) creates a 2-dimensional array with two rows and three columns.

  2. Using np.zeros() and np.ones():

    • np.zeros() creates an array filled with zeros of a specified shape.
    • np.ones() creates an array filled with ones of a specified shape.
    • For example, np.zeros((3, 3)) creates a 3×3 array filled with zeros, and np.ones((2, 4)) creates a 2×4 array filled with ones.

  3. Using np.arange():

    • This function creates an array with evenly spaced values within a specified range.
    • It takes start, stop, and step parameters.
    • For example, np.arange(0, 10, 2) creates an array from 0 to 9 with a step of 2, resulting in [0, 2, 4, 6, 8].

  4. Using np.linspace():

    • This function creates an array with evenly spaced values over a specified interval.
    • It takes start, stop, and num parameters, where num indicates the number of equally spaced values to generate.
    • For example, np.linspace(0, 10, 5) creates an array with 5 equally spaced values from 0 to 10, resulting in [0.0, 2.5, 5.0, 7.5, 10.0].

These methods provide flexibility in creating arrays with different shapes and values, making NumPy a powerful tool for numerical computing and data manipulation in Python.

Array Indexing

				
					import numpy as np

arr = np.array([1, 2, 3, 4, 5])
print(arr[0])      # Output: 1
print(arr[-1])     # Output: 5
print(arr[1:4])    # Output: [2, 3, 4]


import numpy as np

arr = np.array([1, 2, 3, 4, 5])
print(arr[:3])     # Output: [1, 2, 3]
print(arr[::2])    # Output: [1, 3, 5]



import numpy as np

arr = np.array([1, 2, 3, 4, 5])
mask = arr > 2
print(arr[mask])          # Output: [3, 4, 5]

indices = np.array([0, 2, 4])
print(arr[indices])       # Output: [1, 3, 5]

let’s discuss array indexing in NumPy without using code:

  1. Basic Indexing:

    • Basic indexing refers to accessing individual elements or slices of an array.
    • You can access elements using integer indices, where 0 represents the first element, 1 represents the second, and so on.
    • Negative indices can also be used, where -1 represents the last element, -2 represents the second-to-last element, and so forth.
    • Slices of arrays can be obtained using the colon : operator, allowing you to specify a range of indices to extract.

  2. Slicing:

    • Slicing is a powerful feature in NumPy that allows you to extract portions of an array.
    • It uses the colon : operator to specify the start, stop, and step values for the slice.
    • Omitting the start index defaults to the beginning of the array, omitting the stop index defaults to the end, and omitting the step defaults to 1.
    • Slices of arrays are views, meaning they refer to the original data rather than creating a copy.

  3. Advanced Indexing:

    • Advanced indexing allows for more complex methods of accessing array elements.
    • It includes boolean indexing and integer array indexing.
    • Boolean indexing involves creating a boolean mask, where True values indicate the elements to be selected.
    • Integer array indexing involves passing an array of indices to select specific elements from the original array.

Understanding array indexing in NumPy is fundamental for efficiently manipulating and extracting data from arrays. It provides flexibility and control over accessing elements, making it a powerful tool for array operations and data analysis.

Data Types

				
					import numpy as np

# Creating arrays with different data types
arr1 = np.array([1, 2, 3])                # Integers by default
arr2 = np.array([1.0, 2.5, 3.7])           # Floating-point numbers
arr3 = np.array([True, False, True])       # Booleans

# Specifying data types explicitly
arr_int32 = np.array([1, 2, 3], dtype=np.int32)
arr_float64 = np.array([1.0, 2.5, 3.7], dtype=np.float64)
arr_bool = np.array([True, False, True], dtype=np.bool)

# Printing array data types
print("Array 1 data type:", arr1.dtype)
print("Array 2 data type:", arr2.dtype)
print("Array 3 data type:", arr3.dtype)
print("Array with int32 data type:", arr_int32.dtype)
print("Array with float64 data type:", arr_float64.dtype)
print("Array with bool data type:", arr_bool.dtype)

Data types in NumPy represent the type of data stored in an array. Here are some common data types used in NumPy:

  1. bool: Boolean (True or False) stored as a byte
  2. int8, int16, int32, int64: Signed integers of 8, 16, 32, or 64 bits
  3. uint8, uint16, uint32, uint64: Unsigned integers of 8, 16, 32, or 64 bits
  4. float16, float32, float64: Floating-point numbers of 16, 32, or 64 bits
  5. complex64, complex128: Complex numbers represented by two 32-bit or 64-bit floats

Each data type has its own range of values that it can represent, as well as its own memory footprint. Choosing the appropriate data type for your array can help conserve memory and ensure compatibility with mathematical operations.

Additionally, NumPy supports structured data types, which allow you to define custom data types with multiple fields. This is useful for handling structured data such as CSV files or database records.

Understanding data types is important for working with NumPy arrays effectively, as it affects the behavior of array operations and memory usage.

Array Iterating

				
					import numpy as np

arr = np.array([[1, 2, 3], [4, 5, 6]])

# Iterate over each row
for row in arr:
    # Iterate over each element in the row
    for elem in row:
        print(elem)


import numpy as np

arr = np.array([[1, 2, 3], [4, 5, 6]])

# Iterate over each element using nditer
for elem in np.nditer(arr):
    print(elem)


import numpy as np

arr = np.array([[1, 2, 3], [4, 5, 6]])

# Iterate over each element and its index
for index, elem in np.ndenumerate(arr):
    print("Index:", index, "Element:", elem)

Array iterating in NumPy refers to the process of traversing through the elements of an array to perform operations or access values without using explicit loops in Python. It’s a fundamental aspect of working with arrays and is essential for tasks like data manipulation, calculations, and analysis.

When iterating over arrays, it’s important to understand the shape and dimensionality of the array. NumPy arrays can be multi-dimensional, and iterating over them requires consideration of each dimension.

Here are some key points about array iterating in NumPy:

  1. Element-wise Iteration: In most cases, array iteration involves visiting each element of the array. This can be achieved using loops, such as for loops in Python, or using NumPy’s built-in functions like nditer().

  2. Iterating over Rows and Columns: For multi-dimensional arrays, you can iterate over the rows and columns using nested loops. This allows you to access each element individually or perform row-wise or column-wise operations.

  3. Efficiency: NumPy provides efficient mechanisms for array iteration to improve performance. Functions like nditer() optimize the iteration process and can be faster than using traditional Python loops.

  4. Indexing and Element Access: During iteration, you can access individual elements of the array using indexing. This allows you to read or modify the values as needed.

  5. Additional Information: NumPy also provides functions like ndenumerate() for iterating with index information. This can be useful when you need both the value and the index of each element.

Overall, array iterating is a fundamental aspect of working with NumPy arrays, enabling you to manipulate data efficiently and perform various numerical computations. Understanding the different iteration methods and choosing the appropriate approach based on your specific task is important for writing efficient and readable code.

Joining Array

				
					import numpy as np

arr1 = np.array([[1, 2], [3, 4]])
arr2 = np.array([[5, 6]])

# Concatenate arr2 to arr1 along axis 0 (rows)
result = np.concatenate((arr1, arr2), axis=0)


import numpy as np

arr1 = np.array([[1, 2], [3, 4]])
arr2 = np.array([[5, 6]])

# Vertically stack arr2 on top of arr1
result_vstack = np.vstack((arr1, arr2))

# Horizontally stack arr2 next to arr1
result_hstack = np.hstack((arr1, arr2.T))  # Transpose arr2 to match dimensions


import numpy as np

arr1 = np.array([1, 2, 3])
arr2 = np.array([4, 5, 6])

# Stack arr2 on top of arr1 along a new axis
result_stack = np.stack((arr1, arr2))

Joining arrays in NumPy involves combining multiple arrays into a single array along a specified axis. This operation is essential for manipulating and organizing data efficiently. Here’s an explanation of the main methods for joining arrays:

  1. numpy.concatenate(): This function combines arrays along a specified axis. By default, it concatenates arrays along axis 0 (rows), but you can specify a different axis if needed. It is useful for joining arrays with the same shape along an existing axis.

  2. numpy.vstack() and numpy.hstack(): These functions stack arrays vertically (vstack) or horizontally (hstack). vstack stacks arrays vertically, combining them along axis 0, while hstack stacks arrays horizontally, combining them along axis 1. They are convenient for combining arrays with compatible shapes.

  3. numpy.stack(): This function stacks arrays along a new axis. It creates a new axis to accommodate the arrays being stacked, allowing for the combination of arrays with different shapes. You specify the axis along which the new axis is inserted.

These methods provide flexibility for combining arrays in different ways, whether it’s concatenating arrays along existing axes or stacking arrays to create new axes. They are fundamental operations in NumPy for data manipulation and array manipulation tasks.

Array Splitting

				
					import numpy as np

# Create an array
arr = np.arange(10)
print("Original array:")
print(arr)

# Split the array into three equal-sized sub-arrays
sub_arrays = np.split(arr, 3)
print("\nSplit array into 3 sub-arrays:")
for sub_arr in sub_arrays:
    print(sub_arr)

# Split the array at specified indices
indices = [3, 7]
sub_arrays_indices = np.split(arr, indices)
print("\nSplit array at indices 3 and 7:")
for sub_arr in sub_arrays_indices:
    print(sub_arr)

# Split the array into unequal-sized sub-arrays
unequal_sub_arrays = np.array_split(arr, 4)
print("\nSplit array into unequal-sized sub-arrays:")
for sub_arr in unequal_sub_arrays:
    print(sub_arr)

Splitting arrays in NumPy involves dividing a single array into multiple smaller arrays along a specified axis. This operation is useful for organizing and manipulating data efficiently. Here’s an explanation of the main methods for splitting arrays:

  1. numpy.split(): This function splits an array into multiple sub-arrays along a specified axis. You provide the array to split and the number of splits or the indices at which to split the array. It returns a list of sub-arrays.

  2. numpy.array_split(): Similar to numpy.split(), this function splits an array into multiple sub-arrays along a specified axis. However, it allows for uneven splits if the number of splits does not evenly divide the size of the array.

  3. numpy.hsplit() and numpy.vsplit(): These functions split an array horizontally (hsplit) or vertically (vsplit). hsplit divides the array along axis 1 (columns), while vsplit divides the array along axis 0 (rows). They are convenient for splitting arrays into smaller chunks based on the shape of the array.

  4. numpy.dsplit(): This function splits an array along the third axis (depth). It is useful for splitting 3-dimensional arrays into smaller sub-arrays along the depth axis.

These methods provide flexibility for splitting arrays in different ways, whether it’s splitting arrays into equal-sized chunks or dividing them based on specific dimensions. They are fundamental operations in NumPy for data manipulation and array manipulation tasks.

Searching Arrays

				
					import numpy as np

# Creating a sample array
arr = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

# argmax() and argmin()
print("Index of maximum element:", np.argmax(arr))
print("Index of minimum element:", np.argmin(arr))

# where()
print("Indices where elements are greater than 5:", np.where(arr > 5))

# nonzero()
print("Indices of non-zero elements:", np.nonzero(arr))

# searchsorted()
sorted_arr = np.sort(arr)
print("Indices to maintain sorted order:", np.searchsorted(sorted_arr, [2, 5, 8]))

# isin()
print("Elements present in the array:", np.isin(arr, [2, 5, 8]))

# extract()
print("Elements greater than 5:", np.extract(arr > 5, arr))

Searching arrays in NumPy can be done using various functions. Here are some common ones:

  1. argmax() and argmin(): These functions return the indices of the maximum and minimum elements in the array, respectively.

  2. where(): This function returns the indices of elements in the array where a specified condition is satisfied.

  3. nonzero(): This function returns the indices of non-zero elements in the array.

  4. searchsorted(): This function finds the indices where elements should be inserted to maintain the array’s sorted order.

  5. isin(): This function checks for the presence of elements from one array in another array.

  6. extract(): This function returns the elements of an array that satisfy a condition.

These functions provide powerful tools for searching and extracting information from NumPy arrays.

Sorting Arrays

				
					import numpy as np

# Creating an array
arr = np.array([3, 1, 6, 2, 8, 4])

# Sorting the array and storing the result in a new array
sorted_arr = np.sort(arr)
print("Sorted array:", sorted_arr)

# Sorting the array in-place
arr.sort()
print("Array sorted in-place:", arr)

Sorting arrays in NumPy arranges the elements of the array in ascending order by default. The np.sort() function returns a new sorted array without modifying the original array. If you want to sort the array in-place, you can use the sort() method of the array object itself.

For example, given an array [3, 1, 6, 2, 8, 4], sorting it will result in [1, 2, 3, 4, 6, 8]. This operation is useful for various data manipulation and analysis tasks, such as finding the minimum or maximum values, identifying outliers, or preparing data for visualization or further analysis.

Pandas Getting Started

Certainly! Pandas is a powerful data analysis and manipulation library in Python that provides easy-to-use data structures and functions for working with structured data. Here’s a breakdown of its key components and concepts:

  1. DataFrame: The central data structure in Pandas is the DataFrame, which is a two-dimensional labeled data structure similar to a spreadsheet or SQL table. It consists of rows and columns, where each column can have a different data type. DataFrames allow you to store and manipulate data efficiently.

  2. Series: A Series is a one-dimensional labeled array capable of holding data of any type. It’s like a single column of a DataFrame. Series are used to represent one-dimensional data structures and are often created from lists or arrays.

  3. Indexing and Slicing: Pandas provides powerful indexing and slicing capabilities to access specific rows, columns, or elements in a DataFrame. You can use labels (.loc[]) or integer-based indices (.iloc[]) to access data. Indexing and slicing allow you to select subsets of data for analysis or manipulation.

  4. Data Manipulation: Pandas offers a wide range of functions for data manipulation, including adding or removing columns, handling missing data, filtering rows based on conditions, grouping data, and merging or joining multiple DataFrames. These operations enable you to clean, transform, and reshape your data as needed.

  5. Descriptive Statistics: Pandas provides functions to compute descriptive statistics for your data, such as mean, median, standard deviation, minimum, maximum, and quantiles. These statistics help you understand the distribution and characteristics of your data.

  6. Data Visualization: While Pandas itself does not provide visualization capabilities, it seamlessly integrates with popular data visualization libraries like Matplotlib and Seaborn. You can easily convert your DataFrame into a format compatible with these libraries to create insightful plots and charts for data visualization.

  7. Handling Missing Data: Missing data is a common issue in real-world datasets. Pandas provides functions to handle missing data, including dropping rows or columns with missing values (dropna()), filling missing values with a specified value (fillna()), and identifying missing values (isna()).

  8. Grouping and Aggregation: Pandas allows you to group data based on one or more columns and perform aggregation functions like sum, mean, count, etc., on the grouped data. Grouping and aggregation are useful for analyzing data at different levels of granularity.

  9. Reading and Writing Data: Pandas supports reading data from various file formats, including CSV, Excel, SQL databases, JSON, and HTML, using functions like read_csv(), read_excel(), read_sql(), etc. Similarly, you can write DataFrame back to these formats using corresponding functions.

Overall, Pandas is an essential tool for data analysis and manipulation tasks in Python, offering a wide range of functionalities to work with structured data effectively.

Pandas Series

				
					import pandas as pd

# Create a Series from a list
data = [10, 20, 30, 40, 50]
series_from_list = pd.Series(data)
print("Series from list:")
print(series_from_list)

# Create a Series from a dictionary
data = {'a': 10, 'b': 20, 'c': 30, 'd': 40, 'e': 50}
series_from_dict = pd.Series(data)
print("\nSeries from dictionary:")
print(series_from_dict)

# Create a Series with custom indices
data = [10, 20, 30, 40, 50]
custom_index = ['A', 'B', 'C', 'D', 'E']
series_with_custom_index = pd.Series(data, index=custom_index)
print("\nSeries with custom indices:")
print(series_with_custom_index)

A Pandas Series is a one-dimensional labeled array capable of holding data of any type. It is similar to a one-dimensional array, list, or column in a spreadsheet. However, unlike a simple Python list, a Pandas Series can have custom row indices, making it more versatile and powerful for data manipulation and analysis.

Here are some key characteristics of Pandas Series:

  1. Homogeneous Data: Series can contain data of any type (integer, float, string, etc.), but all the elements within a single Series must have the same data type.

  2. Labeled Indices: Each element in a Series is associated with a label or index, which can be explicitly defined or automatically generated by Pandas. Indices can be integers, strings, or any other immutable type.

  3. Size Immutability: Once a Series is created, its size (number of elements) remains fixed. However, you can modify individual elements or the entire Series using various methods provided by Pandas.

  4. Vectorized Operations: Series support vectorized operations, meaning you can perform element-wise operations on entire Series without using explicit loops. This makes data manipulation more efficient and concise.

  5. Flexible Indexing: Series support both integer-based and label-based indexing. You can access elements using integer positions (similar to Python lists) or labels (similar to dictionaries).

  6. Named Series: Series can have a name attribute, which provides a descriptive label for the Series. This can be useful for documentation and organization of data.

  7. Similar to NumPy Arrays: Under the hood, Series are built on top of NumPy arrays and share many similarities with them. However, Series provide additional functionalities tailored for data analysis and manipulation tasks.

In summary, Pandas Series are versatile data structures that provide a convenient way to work with one-dimensional labeled data in Python. They form the building blocks of more complex data structures like DataFrames and are widely used in data analysis, data visualization, and other data-related tasks.

Pandas DataFrames

				
					import pandas as pd

# Creating a DataFrame from a dictionary
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David'],
        'Age': [25, 30, 35, 40],
        'City': ['New York', 'Los Angeles', 'Chicago', 'Houston']}

df = pd.DataFrame(data)

print(df)

Pandas DataFrames are tabular data structures in Python that consist of rows and columns. They are similar to Excel spreadsheets or SQL tables, providing a way to store and manipulate structured data.

Here’s an overview of key concepts related to Pandas DataFrames:

  1. Rows and Columns: DataFrames are organized into rows and columns. Each row typically represents an observation or sample, while each column represents a variable or feature.

  2. Indexing: DataFrames have an index, which is used to uniquely identify each row. By default, the index is a sequence of integers starting from 0, but you can specify a custom index.

  3. Data Types: Each column in a DataFrame can have a different data type, such as integer, float, string, or datetime.

  4. Creation: DataFrames can be created from various data sources, including dictionaries, lists of dictionaries, NumPy arrays, and external files like CSV or Excel files.

  5. Manipulation: You can manipulate DataFrames in various ways, such as selecting subsets of data, adding or removing columns, merging or joining with other DataFrames, and performing calculations or transformations on the data.

  6. Operations: DataFrames support various operations, such as arithmetic operations, aggregation functions (like mean, sum, count), sorting, filtering, and grouping.

  7. Visualization: Pandas provides built-in methods for data visualization, allowing you to create plots and charts directly from your DataFrame data.

Overall, Pandas DataFrames are powerful tools for data analysis and manipulation in Python, offering a flexible and intuitive way to work with structured data.

Pandas Read CSV

				
					import pandas as pd

# Reading CSV file into a DataFrame
df = pd.read_csv("example.csv")

# Displaying the DataFrame
print(df)

Reading CSV files into Pandas DataFrames involves using the pd.read_csv() function. This function reads the data from a CSV file and creates a DataFrame, which is a two-dimensional labeled data structure with rows and columns, similar to a spreadsheet or SQL table.

When reading a CSV file, Pandas infers the data types of each column based on the data present in the file. It also assigns column names automatically if the file contains a header row. If not, you can specify the column names using the names parameter.

By default, pd.read_csv() assumes that the first row of the CSV file contains the column names. However, if your file doesn’t have a header row, you can set header=None and specify column names using the names parameter.

Additionally, Pandas provides several parameters to customize the CSV reading process, such as specifying the delimiter, handling missing values, skipping rows or columns, and parsing dates.

Once the CSV file is read into a DataFrame, you can perform various data manipulation and analysis tasks, such as filtering rows, selecting columns, aggregating data, visualizing data, and much more, using Pandas’ extensive functionality.

Analyzing DataFrames

				
					import pandas as pd

# Read data from CSV file into a DataFrame
df = pd.read_csv('data.csv')

# Display the first few rows of the DataFrame
print(df.head())

# Summary statistics
print(df.describe())

# Check for missing values
print(df.isnull().sum())

# Filter data based on conditions
filtered_data = df[df['column_name'] > 50]

# Grouping and aggregation
grouped_data = df.groupby('column_name').agg({'numerical_column': 'mean'})

# Data visualization (requires matplotlib or seaborn)
import matplotlib.pyplot as plt

# Create a histogram
df['numerical_column'].hist()
plt.title('Histogram of Numerical Column')
plt.xlabel('Value')
plt.ylabel('Frequency')
plt.show()

# Merge DataFrames
df1 = pd.DataFrame({'key': ['A', 'B', 'C'], 'value': [1, 2, 3]})
df2 = pd.DataFrame({'key': ['A', 'B', 'D'], 'value': [4, 5, 6]})
merged_df = pd.merge(df1, df2, on='key', how='inner')

# Reshape Data
stacked_df = df.stack()

# Statistical Analysis
correlation = df['column1'].corr(df['column2'])

# Time Series Analysis
df['date_column'] = pd.to_datetime(df['date_column'])
resampled_data = df.resample('W').sum()

# Display the final DataFrame
print(merged_df)

Analyzing DataFrames involves various operations to understand and extract insights from the data they contain. Here are some common tasks involved in analyzing DataFrames:

  1. Viewing Data: Displaying the first few rows of the DataFrame to get an overview of the data.

  2. Summary Statistics: Calculating summary statistics such as mean, median, standard deviation, minimum, and maximum values for numerical columns.

  3. Data Cleaning: Handling missing or duplicate values, converting data types, and renaming columns.

  4. Filtering Data: Selecting rows or columns based on specific conditions.

  5. Grouping and Aggregation: Grouping data based on one or more columns and performing aggregate functions (e.g., sum, mean, count) on grouped data.

  6. Data Visualization: Creating visualizations such as histograms, scatter plots, and bar charts to explore relationships and patterns in the data.

  7. Merging and Joining: Combining multiple DataFrames based on common columns or indices.

  8. Reshaping Data: Pivoting, melting, or stacking DataFrames to transform the data structure.

  9. Statistical Analysis: Conducting hypothesis tests, correlations, and regression analysis to understand relationships between variables.

  10. Time Series Analysis: Analyzing time-based data, including calculating rolling averages, resampling, and plotting time series.

These tasks can be performed using various methods and functions available in libraries like pandas, numpy, and matplotlib/seaborn for data manipulation, analysis, and visualization in Python.

Cleaning Data

				
					import pandas as pd

# Sample DataFrame with missing values and duplicates
data = {
    'Name': ['John', 'Jane', 'Paul', 'John', 'Emma'],
    'Age': [30, 25, None, 30, 28],
    'Gender': ['Male', 'Female', 'Male', 'Male', 'Female'],
    'City': ['New York', 'Los Angeles', 'Chicago', 'New York', 'Chicago']
}

df = pd.DataFrame(data)

# Handling missing values
df.dropna(inplace=True)  # Drop rows with missing values

# Removing duplicates
df.drop_duplicates(inplace=True)  # Remove duplicate rows

# Correcting data types
df['Age'] = df['Age'].astype(int)  # Convert Age column to integer data type

# Standardizing text data
df['City'] = df['City'].str.lower()  # Convert City names to lowercase

# Encoding categorical variables
df = pd.get_dummies(df, columns=['Gender'])  # One-hot encoding for Gender column

print(df)

Cleaning data is an essential step in the data analysis process. It involves identifying and handling missing values, removing duplicates, and correcting inconsistencies or errors in the data. Here’s an overview of common techniques used for cleaning data:

  1. Handling Missing Values:

    • Identify missing values using functions like isnull() or isna().
    • Decide how to handle missing values:
      • Drop rows or columns with missing values using dropna().
      • Impute missing values using methods like mean, median, or interpolation using fillna().

  2. Removing Duplicates:

    • Identify duplicate rows using duplicated() function.
    • Remove duplicate rows using drop_duplicates() function.

  3. Correcting Data Types:

    • Ensure data types are correct for analysis.
    • Convert data types using functions like astype() or to_datetime().

  4. Standardizing Text Data:

    • Standardize text data by converting to lowercase, removing leading/trailing whitespaces, or using regular expressions.
    • Use functions like str.lower(), str.strip(), or str.replace().

  5. Handling Inconsistent Data:

    • Identify and correct inconsistent data, such as different representations of the same category or data entry errors.
    • Use string methods or regular expressions to clean inconsistent data.

  6. Scaling and Normalizing Numerical Data:

    • Scale numerical data to a common range or normalize it to have a mean of 0 and standard deviation of 1.
    • Use techniques like Min-Max scaling or Z-score normalization.

  7. Encoding Categorical Variables:

    • Convert categorical variables into numerical format using techniques like one-hot encoding or label encoding.
    • Use functions like get_dummies() or LabelEncoder().

  8. Handling Outliers:

    • Identify and handle outliers in the data.
    • Remove outliers or apply transformations to reduce their impact on the analysis.

  9. Data Validation:

    • Validate data against predefined rules or constraints.
    • Check for logical inconsistencies or unexpected values.

  10. Data Transformation:

    • Perform transformations such as log transformation or box-cox transformation to make data more normally distributed.
    • Use functions like np.log() or stats.boxcox().

By following these techniques, you can ensure that your data is clean, consistent, and ready for analysis.

Matplotlib Getting Started

				
					import matplotlib.pyplot as plt

# Sample data
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]

# Create a line plot
plt.plot(x, y)

# Add labels and title
plt.xlabel('X-axis')
plt.ylabel('Y-axis')
plt.title('Simple Line Plot')

# Show the plot
plt.show()

Certainly! Matplotlib is a versatile plotting library in Python that allows you to create various types of plots and visualizations. Here’s a breakdown without using code:

  1. Installation: Before you can use Matplotlib, you need to install it. You can do this using pip, the Python package manager, by running pip install matplotlib in your command prompt or terminal.

  2. Importing: Once installed, you import Matplotlib into your Python script or environment. It’s common to import it with the alias plt, which makes it easier to refer to later in your code.

  3. Basic Plotting: The core functionality of Matplotlib revolves around creating plots. You can start by creating simple line plots, scatter plots, histograms, bar plots, etc. These plots are created by providing data to Matplotlib functions like plt.plot(), plt.scatter(), plt.hist(), plt.bar(), etc.

  4. Customization: Matplotlib allows extensive customization of plots. You can change colors, line styles, markers, add titles, labels to axes, create legends, adjust grid lines, and more. This customization helps in making your plots visually appealing and informative.

  5. Other Types of Plots: Besides the basic plots, Matplotlib supports a wide range of plot types. You can create box plots, pie charts, violin plots, 3D plots, contour plots, etc., depending on the nature of your data and the insights you want to convey.

  6. Saving Plots: Once you’ve created your plot, you can save it as an image file in different formats such as PNG, PDF, SVG, etc. This is useful for sharing your visualizations with others or incorporating them into reports or presentations.

  7. Exploration and Documentation: Matplotlib is a vast library with many features and capabilities. Exploring the official documentation, tutorials, and examples provided by the Matplotlib community can help you learn more about its functionalities and how to use them effectively.

Overall, Matplotlib is a powerful tool for data visualization in Python, widely used by data scientists, researchers, engineers, and analysts to explore and communicate insights from their data.

Matplotlib Pyplot

				
					import matplotlib.pyplot as plt
import numpy as np

xpoints = np.array([0, 6])
ypoints = np.array([0, 250])

plt.plot(xpoints, ypoints)
plt.show()
pyplot

Matplotlib’s pyplot module is a powerful tool for creating visualizations in Python. It provides a MATLAB-like interface for generating plots quickly and easily. Here’s a rundown of some key concepts:

  1. Plotting Functions: pyplot offers a variety of functions for creating different types of plots, including line plots, scatter plots, bar plots, histograms, and more. Each plot type has a corresponding function like plot(), scatter(), bar(), hist(), etc.

  2. Customization: You can customize nearly every aspect of your plots, including colors, line styles, markers, labels, titles, and axes. This allows you to tailor your visualizations to convey your data effectively.

  3. Data Representation: Matplotlib works with data represented as NumPy arrays, Python lists, or Pandas DataFrames. You can plot data directly from these data structures.

  4. Subplots: You can create multiple plots within the same figure using subplots. This is useful for comparing different datasets or visualizing multiple aspects of the same data.

  5. Annotations and Text: Matplotlib allows you to annotate your plots with text, arrows, shapes, and more. Annotations can help highlight important features or add context to your visualizations.

  6. Saving Plots: Once you’ve created a plot, you can save it to various file formats such as PNG, PDF, SVG, etc., using the savefig() function. This allows you to use your plots in presentations, reports, or publications.

  7. Integration with Jupyter Notebooks: Matplotlib works seamlessly with Jupyter Notebooks, making it easy to create interactive visualizations alongside your code and analysis.

These are just some of the fundamental concepts of using Matplotlib’s pyplot module. As you become more familiar with the library, you’ll discover additional features and techniques for creating insightful and visually appealing plots.

Matplotlib Scatter

				
					import matplotlib.pyplot as plt
import numpy as np

x = np.array([5,7,8,7,2,17,2,9,4,11,12,9,6])
y = np.array([99,86,87,88,111,86,103,87,94,78,77,85,86])

plt.scatter(x, y)
plt.show(
scatter plot

A scatter plot in Matplotlib is a type of plot used to visualize the relationship between two numerical variables. It’s particularly useful for showing the distribution and correlation between the variables.

In a scatter plot, each data point is represented as a dot, positioned according to its values on the x and y axes. If there is a pattern or trend in the data, it can often be discerned visually.

For example, if you have two variables, such as the number of hours studied and the exam scores obtained by a group of students, you can create a scatter plot to see if there’s any correlation between the two. If the points cluster around a line with a positive slope, it suggests a positive correlation (i.e., as one variable increases, the other tends to increase). If the points cluster around a line with a negative slope, it suggests a negative correlation (i.e., as one variable increases, the other tends to decrease). If the points are scattered randomly, there may be no correlation.

Scatter plots are also useful for identifying outliers or anomalies in the data and for visualizing clusters or groups within the data.

Overall, scatter plots are a simple yet powerful tool for exploring and understanding relationships between variables in your data.

Matplotlib Histograms

				
					import matplotlib.pyplot as plt

# Sample data
data = [22, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85]

# Create histogram
plt.hist(data, bins=5, color='skyblue', edgecolor='black')

# Add labels and title
plt.xlabel('Age')
plt.ylabel('Frequency')
plt.title('Age Distribution')

# Show plot
plt.show()
histogram

Histograms in Matplotlib are used to visualize the distribution of a single numerical variable by dividing it into bins and displaying the frequency or count of data points within each bin. Histograms are particularly useful for understanding the central tendency, spread, and shape of the data.

Here’s how histograms work:

  1. Binning: The range of the data is divided into intervals or bins along the x-axis. Each bin represents a range of values.

  2. Counting: The number of data points falling within each bin is counted. This count is represented on the y-axis.

  3. Plotting: The counts are then plotted as bars, with the height of each bar corresponding to the frequency of data points in the bin.

Histograms provide insights into the distribution of the data, including whether it’s symmetric, skewed left or right, bimodal, or uniform. They also help identify the presence of outliers or unusual patterns in the data.

For example, if you have a dataset containing the ages of people in a population, you can create a histogram to visualize the age distribution. The histogram will show how many people fall into different age ranges, such as 0-10, 11-20, 21-30, and so on.

Overall, histograms are a valuable tool for exploratory data analysis and are often used to summarize and visualize the distribution of numerical data in a dataset.

Matplotlib Pie Charts

				
					import matplotlib.pyplot as plt
import numpy as np

y = np.array([35, 25, 25, 15])
mylabels = ["Apples", "Bananas", "Cherries", "Dates"]

plt.pie(y, labels = mylabels)
plt.legend()
plt.show()
pie chart

Pie charts are circular statistical graphics used to illustrate numerical proportions. Each slice of the pie represents a proportion of the whole, typically expressed as a percentage. They’re useful for showing the relative sizes of different categories in a dataset

This code creates a simple pie chart using Matplotlib. Here’s what each part of the code does:

  • import matplotlib.pyplot as plt: Imports the Matplotlib library under the alias plt.
  • import numpy as np: Imports the NumPy library under the alias np.
  • y = np.array([35, 25, 25, 15]): Creates a NumPy array with the values [35, 25, 25, 15]. These values represent the sizes of the slices in the pie chart.
  • mylabels = ["Apples", "Bananas", "Cherries", "Dates"]: Defines labels for each slice of the pie chart.
  • plt.pie(y, labels = mylabels): Creates a pie chart with the data provided in y and labels each slice according to the mylabels list.
  • plt.legend(): Adds a legend to the pie chart, indicating the meaning of each slice.
  • plt.show(): Displays the pie chart.

This pie chart visualizes the distribution of different types of fruits, where each slice represents a different fruit category, and the size of each slice represents the proportion of that category in the total.

Scroll to Top