HAQM Augmented AI FAQs

Many machine learning applications require humans to review low confidence predictions to ensure the results are correct. For example, extracting information from scanned mortgage application forms can require human review in some cases due to low-quality scans or poor handwriting. But building human review systems can be time consuming and expensive because it involves implementing complex processes or “workflows”, writing custom software to manage review tasks and results, and in many cases, managing large groups of reviewers.

HAQM A2I makes it easy to build and manage human reviews for machine learning applications. HAQM A2I provides built-in human review workflows for common machine learning use cases, such as content moderation and text extraction from documents, which allows predictions from HAQM Rekognition and HAQM Textract to be reviewed easily. You can also create your own workflows for ML models built on HAQM SageMaker or any other tools. Using HAQM A2I, you can allow human reviewers to step in when a model is unable to make a high confidence prediction or to audit its predictions on an ongoing basis.

HAQM A2I provides a managed experience where you can set up an entire human review workflow in a few easy steps. To get started with HAQM A2I, sign in to your AWS Console, and navigate to the HAQM SageMaker console. From there, select Human review workflows under Augmented AI. First, as a part of the human review workflow, you provide a pointer to the S3 bucket where the review results should be stored. Next, you select the appropriate task type and define conditions when a human review should be triggered. HAQM A2I provides pre-built workflows where you only need to enter a few choices and provide instructions on how your objects should be reviewed by humans. Alternatively, you can create your own custom workflow and use your own custom review templates. Once created, the workflow can be used directly in your applications using a generated unique identifier for this workflow.

With A2I, you can define what is an acceptable prediction confidence for your business problem. You can define business rules for the machine learning predictions, based on which a human review is triggered. For HAQM Rekognition image moderation tasks, you can use the confidence score that HAQM Rekognition provides for each label it outputs to trigger human review. For HAQM Textract tasks, you can trigger a human review when specific form keys are missing or when form key detection confidence is low. You can also trigger a human review if, after evaluating all form keys in the text, confidence is lower than your required threshold for any form key. For your own custom workflow, you can write the code for business conditions in AWS Lambda or directly in your client application.

With HAQM A2I, you can choose from three workforce options: (1) HAQM Mechanical Turk; (2) Third party data labeling service providers available through the AWS Marketplace; and (3) Your own employees. See the HAQM A2I developer guide for more information.

Please see the HAQM A2I pricing page for the current pricing information.

The AWS Region Table lists all the AWS regions where HAQM A2I is currently available.

Yes, HAQM Augmented AI service providers can process customer confidential data. The Standard Service Agreement between AWS customers and the third-party service provider contains clauses to protect your confidential information. Please review those terms before sharing any confidential information with the service provider. The terms are located on the listing page for the service provider on AWS Marketplace.

In light of the rapidly evolving impact of COVID-19, some service providers have implemented a remote work policy for the health and safety of their employees temporarily. During this time, security standards including SOC 2 compliance and additional security controls outlined in the below FAQ may not be applicable to the affected service providers. Impacted service providers have updated their AWS Marketplace listings to reflect this, and will not process customer data from remote work environments without explicit customer consent.

Human review service providers are required to go through SOC 2 compliance and certification on an annual basis. The SOC 2 report is a description of the service provider’s control environment based on the American Institute of Certified Public Accountants (AICPA) Trust Services Criteria - Security, Availability, Processing Integrity, Confidentiality, and Privacy.

In addition to SOC 2, service providers are required to maintain these additional security controls to secure customer data in their environments.

Technology Controls:
Service providers are required to utilize the appropriate software to block any attempts to download or copy files/data from their system and prevent unauthorized access to their systems. Service providers are also required to prohibit their workforce from storing or copying customer task-related data outside of service provider secure environments.

Network Security Controls:
Service providers must prohibit remote access to customer's task-related data. Further, peer-to-peer file sharing software is blocked on the provider's network.

Employee Controls:
Service providers are required to ensure they have Non-Disclosure Agreements (NDAs) with their employees. Service providers are required to adopt stringent policies to prevent prevent employees from copying or moving customer task related data from providers secure environment including controls for paper, USBs, mobile phones, or other media.

Physical Access Controls:
Service providers are required to maintain physical access control measures to prevent unauthorized access to their production site. These may include bio-metric authentication, employee badge identification, visual verification of employees by security guards, etc.

AWS requests that service providers furnish their SOC 2 certification reports prior to being listed in the marketplace and confirms:

Authenticity (if service provider auditor is certified by the AICPA);

Report period (SOC 2 certification validity date); and

Production site verification (the physical site where the service provider workforce will work on HAQM Augmented AI tasks).

The security standards from every service provider are reviewed annually to ensure they meet the mandatory requirements.