Numpy

What is NumPy?

• NumPy (Numerical Python) is an open-source library used for numerical computing. It provides support for arrays, matrices, and a wide collection of mathematical functions to perform operations on these arrays efficiently.

Key Features of NumPy:

• Efficient Array Operations: NumPy arrays are more efficient for large-scale mathematical operations compared to Python lists.
• Multidimensional Arrays: Supports arrays with multiple dimensions (like matrices, 3D arrays, etc.).
• Vectorized Operations: Allows for element-wise operations on arrays, making calculations faster and more concise.
• Broadcasting: NumPy automatically handles operations on arrays of different shapes and sizes.

Installing NumPy:

pip install numpy

Creating NumPy Arrays:

• 1D Array:

import numpy as np

arr = np.array([1, 2, 3, 4])

. 2D Array (Matrix):

arr_2d = np.array([[1, 2, 3], [4, 5, 6]])

Array with Zeros:

zeros = np.zeros((3, 3)) # 3x3 array of zeros

Array with Ones:

ones = np.ones((2, 3)) # 2x3 array of ones

Array with Range:

range_arr = np.arange(10) # 0 to 9

5. Array Operations:

• Element-wise Operations: These operations work on each element individually.

arr = np.array([1, 2, 3])

result = arr * 2 # Output: [2, 4, 6]

Mathematical Functions:

np.sqrt(arr) # Square root of each element

np.exp(arr) # Exponential of each element

np.log(arr) # Natural log of each element

6. Array Indexing and Slicing:

Indexing:

arr = np.array([10, 20, 30, 40])

arr[1] # Output: 20

Slicing:

arr[1:3] # Output: [20, 30]

arr[:3] # Output: [10, 20, 30]

7. Reshaping Arrays:

• You can change the shape of an array without changing its data.

arr = np.array([1, 2, 3, 4, 5, 6])

reshaped = arr.reshape(2, 3) # 2x3 matrix

8. Broadcasting:

• Broadcasting allows NumPy to perform operations on arrays of different shapes. For example:

arr = np.array([1, 2, 3])

matrix = np.array([[1], [2], [3]])

result = arr + matrix # Broadcasting adds arr to each row of matrix

9. Common Array Methods:

• Shape of Array:

arr.shape

Size (number of elements):

arr.size

Sum of Array Elements:

arr.sum()

Mean and Standard Deviation:

arr.mean() # Mean

arr.std() # Standard deviation

10. Matrix Operations:

Dot Product (Matrix Multiplication):

matrix1 = np.array([[1, 2], [3, 4]])

matrix2 = np.array([[5, 6], [7, 8]])

result = np.dot(matrix1, matrix2) # Matrix multiplication

Transpose of a Matrix:

matrix.T

11. Random Number Generation:

• Random Array:

random_arr = np.random.rand(3, 3) # 3x3 array of random floats between 0 and 1

Random Integers:

random_ints = np.random.randint(0, 10, size=(3, 3)) # Random integers from 0 to 9

12.Linear Algebra Functions:

• Inverse of a Matrix:

inv_matrix = np.linalg.inv(matrix)

Eigenvalues and Eigenvectors:

eigvals, eigvecs = np.linalg.eig(matrix)

13. Array Manipulation:

• Stacking Arrays:

stacked = np.vstack((arr1, arr2)) # Stack arrays vertically

stacked = np.hstack((arr1, arr2)) # Stack arrays horizontally

Splitting Arrays:

np.split(arr, 3) # Split into 3 parts

14. Advanced Functions:

• Concatenate Arrays:

np.concatenate((arr1, arr2), axis=0) # Concatenate along axis 0 (vertically)

Unique Elements in an Array:

np.unique(arr) # Find unique elements

Best Practices:

• Use NumPy arrays instead of Python lists for large numerical datasets to improve performance.
• NumPy functions are optimized in C, so they are faster than regular Python loops for mathematical computations.

Excercise :

Here’s a list of popular Python libraries, categorized by their usage:

Data Manipulation and Analysis

• Pandas: Data manipulation and analysis, especially for tabular data.
• NumPy: Efficient numerical operations and array handling.
• Dask: Scalable data processing with parallel computing.
• Polars: High-performance DataFrame library for fast data manipulation.

2. Data Visualization

• Matplotlib: Basic 2D plotting and charting library.
• Seaborn: Statistical data visualization, built on Matplotlib.
• Plotly: Interactive plots and dashboards.
• Bokeh: Interactive web-based visualizations.
• Altair: Declarative statistical visualization library.
• Dash: Framework for building analytical web applications.

3. Machine Learning and AI

• Scikit-learn: Machine learning tools for classification, regression, and clustering.
• TensorFlow: Deep learning library by Google.
• PyTorch: Deep learning framework by Facebook.
• Keras: High-level API for building neural networks (works with TensorFlow).
• XGBoost: Gradient boosting framework for optimized machine learning models.
• LightGBM: Fast and efficient gradient boosting.
• Hugging Face Transformers: Pre-trained models for NLP tasks.

4. Data Science and Statistics

• SciPy: Advanced scientific computing and optimization.
• Statsmodels: Statistical modeling and hypothesis testing.
• PyMC3: Bayesian statistical modeling and probabilistic programming.
• SymPy: Symbolic mathematics and algebra.

5. Web Development

• Flask: Lightweight web framework.
• Django: High-level web framework for robust web applications.
• FastAPI: Modern, fast (high-performance) framework for building APIs.
• Bottle: Minimalist web framework.
• Tornado: Scalable and asynchronous web framework.

6. Big Data and Databases

• PySpark: Interface for Apache Spark for big data processing.
• SQLAlchemy: Database toolkit and ORM for Python.
• Pymongo: MongoDB driver for Python.
• Elasticsearch-py: Python client for Elasticsearch.

7. Natural Language Processing (NLP)

• NLTK: Natural Language Toolkit for text processing.
• spaCy: Industrial-strength NLP library.
• Gensim: Topic modeling and document similarity.
• TextBlob: Simple text processing.
• Flair: State-of-the-art NLP made easy.

8. Image Processing and Computer Vision

• OpenCV: Computer vision and image processing.
• Pillow (PIL): Image manipulation and processing.
• scikit-image: Advanced image processing.
• Tesseract: OCR (optical character recognition) library.

9. Web Scraping

• BeautifulSoup: Parsing HTML and XML documents.
• Scrapy: Advanced web scraping and crawling.
• Selenium: Web automation and scraping using a browser.
• Requests-HTML: Web scraping with modern JavaScript support.

10. Network and API Interaction

• Requests: HTTP requests library.
• httpx: Modern alternative to Requests with async capabilities.
• Socket: Core library for network programming.
• Twisted: Event-driven networking engine.

11. Automation and Scripting

• PyAutoGUI: Automating mouse and keyboard tasks.
• Celery: Asynchronous task queue/job queue.
• Paramiko: SSH and SFTP library.

12. Testing

• Pytest: Advanced testing framework.
• unittest: Built-in testing library.
• Nose2: Extensible testing framework.
• Hypothesis: Property-based testing.

13. File Handling and PDF

• OS and shutil: Built-in libraries for file handling.
• PyPDF2: PDF file manipulation.
• Fitz (PyMuPDF): Extract text and images from PDFs.
• python-docx: Working with Microsoft Word documents.

14. Cybersecurity and Cryptography

• Cryptography: Cryptographic recipes and primitives.
• PyCrypto: Cryptographic services.
• Scapy: Network packet manipulation.
• Hashlib: Built-in library for secure hashing.

15. DevOps and Cloud

• Boto3: AWS SDK for Python.
• Kubernetes Python Client: Managing Kubernetes clusters.
• Fabric: Remote server automation and deployment.
• Ansible: Python-based automation.

16. Game Development

• Pygame: 2D game development.
• Panda3D: 3D game engine.
• Godot Python API: Scripting for the Godot game engine.

17. Audio and Video Processing

• MoviePy: Video editing.
• pydub: Audio processing and editing.
• PyMedia: Multimedia processing.
• ffmpeg-python: Wrapping FFmpeg for video and audio tasks.

18. Data Encryption and Security

• Cryptography: Encrypt and decrypt sensitive data.
• PyCryptoDome: Updated cryptography toolkit.