AWS Machine Learning Blog
Author: Shreyas Subramanian
Shreyas Subramanian is a Principal data scientist and helps customers by using Machine Learning to solve their business challenges using the AWS platform. Shreyas has a background in large scale optimization and Machine Learning, and in use of Machine Learning and Reinforcement Learning for accelerating optimization tasks.
Define and run Machine Learning pipelines on Step Functions using Python, Workflow Studio, or States Language
May 2024: This post was reviewed and updated for accuracy. You can use various tools to define and run machine learning (ML) pipelines or DAGs (Directed Acyclic Graphs). Some popular options include AWS Step Functions, Apache Airflow, KubeFlow Pipelines (KFP), TensorFlow Extended (TFX), Argo, Luigi, and HAQM SageMaker Pipelines. All these tools help you compose […]
Build reusable, serverless inference functions for your HAQM SageMaker models using AWS Lambda layers and containers
July 2023: This post was reviewed for accuracy. Please refer to Deploying ML models using SageMaker Serverless Inference, a new inference option that enables you to easily deploy machine learning models for inference without having to configure or manage the underlying infrastructure. In AWS, you can host a trained model multiple ways, such as via […]
Solving numerical optimization problems like scheduling, routing, and allocation with HAQM SageMaker Processing
July 2023: This post was reviewed for accuracy. In this post, we discuss solving numerical optimization problems using the very flexible HAQM SageMaker Processing API. Optimization is the process of finding the minimum (or maximum) of a function that depends on some inputs, called design variables. This pattern is relevant to solving business-critical problems such […]
Running on-demand, serverless Apache Spark data processing jobs using HAQM SageMaker managed Spark containers and the HAQM SageMaker SDK
July 2023: This post was reviewed for accuracy. Apache Spark is a unified analytics engine for large scale, distributed data processing. Typically, businesses with Spark-based workloads on AWS use their own stack built on top of HAQM Elastic Compute Cloud (HAQM EC2), or HAQM EMR to run and scale Apache Spark, Hive, Presto, and other […]
Building a deep neural net–based surrogate function for global optimization using PyTorch on HAQM SageMaker
July 2023: This post was reviewed for accuracy. Optimization is the process of finding the minimum (or maximum) of a function that depends on some inputs, called design variables. Customer X has the following problem: They are about to release a new car model to be designed for maximum fuel efficiency. In reality, thousands of […]
Bring your own hyperparameter optimization algorithm on HAQM SageMaker
July 2023: This post is outdated. We recommend referring to HAQM SageMaker Automatic Model Tuning now supports three new completion criteria for hyperparameter optimization for the latest solution. In this blog post, we’ll discuss how to implement custom, state-of-the-art hyperparameter optimization (HPO) algorithms to tune models on HAQM SageMaker. HAQM SageMaker includes a built-in HPO […]