Pangeo: Earth Science Who Am I? What Problem are We Trying to Solve? How Dask Helps Why We Chose Dask Originally Pain Points Technology around Dask Pangeo: Earth Science ...

API Datasets Utilities API Dask APIs generally follow from upstream APIs: Arrays follows NumPy DataFrames follows Pandas Bag follows map/filter/groupby/reduce common in ...

Diagnostics (local) Progress Bar Profiler ResourceProfiler CacheProfiler Example Custom Callbacks API Diagnostics (local) Profiling parallel code can be challenging, but...

Adaptive Deployments Policies Adaptive class interface Marathon: an example Subclassing Adaptive Adaptive Deployments It is possible to grow and shrink Dask clusters based ...

Worker Resources Example Resources are applied separately to each worker process Resources are Abstract Resources with collections Worker Resources Access to scarce resourc...

Configuration Access Configuration Specify Configuration YAML files Environment Variables Defaults Directly within Python Updating Configuration Manipulating configuration di...

GPUs Custom Computations High Level Collections DataFrames Arrays Scikit-Learn Setup Restricting Work Specifying GPUs per Machine Work in Progress GPUs Dask works with...

Setup Setup This page describes various ways to set up Dask on different hardware, eitherlocally on your own machine or on a distributed cluster. If you are justgetting started...

Diagnostics (distributed) Dashboard Progress bar External Documentation API Diagnostics (distributed) The Dask distributed scheduler provides live feedback in twoforms: ...

Citations Papers about parts of Dask Citations Dask is developed by many people from many institutions. Some of thesedevelopers are academics who depend on academic citations ...