New on the Open Source blog: Using the FSx for Lustre CSI Driver with HAQM EKS

From time to time the AWS Storage Blog will point you to storage related blog posts from other AWS blog channels that we think you will find interesting or helpful for your storage use cases. We wanted to share a post from the Open Source Blog that highlights a walk-through of using HAQM FSx for Lustre Container Storage Interface (CSI) Driver with HAQM EKS.

The HAQM Elastic Container Service for Kubernetes (HAQM EKS) team has been busy building CSI drivers for all our storage solutions, including HAQM FSx for Lustre: a fully-managed file system which has integrations with S3 and is optimized for compute-intensive workloads such as high-performance computing (HPC) and machine learning. Because the AWS FSx CSI Driver is potentially useful to any Kubernetes user, we’ve donated it to Kubernetes SIG-AWS. This blog post focuses on deploying the AWS FSx CSI driver to an HAQM EKS cluster.

You can now dynamically provision HAQM FSx for Lustre filesystems for your high performance computing workloads, and have a container automatically connected into this filesystem. FSx for Lustre provides a high-performance file system optimized for fast processing of workloads such as machine learning, high performance computing (HPC), video processing, financial modeling, and electronic design automation (EDA). These workloads commonly require data to be presented via a fast and scalable file system interface, and typically have data sets stored on long-term data stores like HAQM S3.

You can also use dynamic provisioning to consume the same persistent volume claim from multiple pods from different nodes. If you need to start your containers with a dataset automatically available, for example loading a machine learning training data set, you can even connect your StorageClass to an existing HAQM S3 bucket by using Dynamic Provisioning with Data Repository. Other examples can be found in the FSx for Lustre CSI Driver documentation.

To learn more, see the full AWS Open Source Blog post.