cuDF: RAPIDS GPU-Accelerated Dataframe Library

Modern data science demands interactive exploration and analysis of very large volumes of data. Data scientists rely on Python for solving problems of all scales, but the biggest workloads today run up against the limits of CPU performance and scalability. The massive parallelism, high memory bandwidth and high-speed interconnect of the latest GPUs provide a solution.

RAPIDS is a suite of open-source software libraries that enables end-to-end data science and analytics pipelines to run entirely on GPUs, providing end-to-end speedups of 50x or more. cuDF is a DataFrame manipulation library based on Apache Arrow, a columnar in-memory data format. cuDF provides a familiar pandas-like Python API to accelerate loading, filtering, and manipulation of data for model training data preparation.

cuDF accelerates data preparation workloads by keeping datasets in GPU memory and applying high-performance data-parallel CUDA C++ algorithms for everything from file parsing (CSV, Parquet, ORC, etc), decompression and loading, to selection, sorting, joining, grouping, and aggregation. Combined with the RAPIDS cuML machine learning library and the cuGraph GRAPH analytics library, data scientists can achieve impressive end-to-end speedups and perform interactive ETL and training on very large data sets.

In this talk I will provide an overview of RAPIDS with a focus on the features, functionality and software architecture of cuDF. I’ll share some performance results along with details of how cuDF combines Python and Apache Arrow with Numba, CUDA, and Dask to achieve high performance and scalability on massive workloads.