DASK

DASK

Rohit Singh Rohit Singh

Rohit Singh

Associate Project Manager @ HuQuo

发布日期: 2024年9月6日
+ 关注

Dask is an open-source Python library for parallel computing. Dask scales Python code from multi-core local machines to large distributed clusters in the cloud. Dask provides a familiar user interface by mirroring the APIs of other libraries in the PyData ecosystem including: Pandas, scikit-learn and NumPy.

Dask Arrays allow scientists and researchers to perform intuitive and sophisticated operations on large datasets but use the familiar NumPy API and memory model. One Dask array is simply a collection of NumPy arrays on different computers. Dask is used by retail, financial, governmental organizations, as well as life science and geophysical institutes.

Dask has two parts:

  1. Big data collections (high level and low level)
  2. Dynamic task scheduling

Limitations of dask

There are certain limitations in dask.

  1. Dask cannot parallelize within individual task
  2. As a distributed-computing framework, dask enables remote execution of arbitrary code. So dask workers should be hosted within trusted network only.

要查看或添加评论，请登录

Rohit Singh的更多文章

  • Scope management
    2025年3月27日

    Scope management

    Project scope refers to the detailed description of the deliverables, objectives, tasks, and goals that need to be…

  • Selenium WebDriver
    2025年3月26日

    Selenium WebDriver

    Selenium WebDriver is a powerful Automation tool widely used for web application testing. It provides a programming…

  • Robot Framework
    2025年3月25日

    Robot Framework

    Robot Framework is an open-source test automation framework for acceptance testing and acceptance test-driven…

  • Azure Active Directory
    2025年3月24日

    Azure Active Directory

    Azure Active Directory (Azure AD), now known as Microsoft Entra ID, is a cloud-based identity and access management…

  • Matillion
    2025年3月22日

    Matillion

    Matillion is a cloud-native data integration platform that simplifies and accelerates the ELT (Extract, Load…

  • Azure Blob storage
    2025年3月21日

    Azure Blob storage

    Blob storage is a type of cloud storage for unstructured data, like images, videos, or documents, where data is stored…

  • BI Testing
    2025年3月20日

    BI Testing

    BI testing, or Business Intelligence testing, verifies and validates the accuracy and reliability of insights delivered…

  • Amazon Elastic Container Service (Amazon ECS)
    2025年3月19日

    Amazon Elastic Container Service (Amazon ECS)

    Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service that simplifies the…

  • User Acceptance Testing (UAT)
    2025年3月18日

    User Acceptance Testing (UAT)

    User Acceptance Testing (UAT) is a crucial phase in software testing where the software is tested in a real-world…

  • Software Development Engineer in Test (SDET)
    2025年3月17日

    Software Development Engineer in Test (SDET)

    Software Development Engineer in Test (SDET) is a developer with the primary responsibility for the development of…

    1 条评论

社区洞察

其他会员也浏览了