Query structured data from HAQM Q Business using HAQM QuickSight integration

HAQM Q Business is a generative AI-powered assistant that can answer questions, provide summaries, generate content, and securely complete tasks based on data and information in your enterprise systems. Although generative AI is fueling transformative innovations, enterprises may still experience sharply divided data silos when it comes to enterprise knowledge, in particular between unstructured content (such as PDFs, Word documents, and HTML pages), and structured data (real-time data and reports stored in databases or data lakes). Both categories of data are typically queried and accessed using separate tools, from in-product browse and search functionality for unstructured data, to business intelligence (BI) tools like HAQM QuickSight for structured content.

HAQM Q Business offers an effective solution for quickly building conversational applications over unstructured content, with over 40 data connectors to popular content and storage management systems such as Confluence, SharePoint, and HAQM Simple Storage Service (HAQM S3), to aggregate enterprise knowledge. Customers are also looking for a unified conversational experience across all their knowledge repositories, regardless of the format the content is stored and organized as.

On December 3, 2024, HAQM Q Business announced the launch of its integration with QuickSight, allowing you to quickly connect your structured sources to your HAQM Q Business applications, creating a unified conversational experience for your end-users. The QuickSight integration offers an extensive set of over 20 structured data source connectors, including HAQM Redshift, PostgreSQL, MySQL, and Oracle, enabling you to quickly expand the conversational scope of your HAQM Q Business assistants to cover a wider range of knowledge sources. For the end-users, answers are returned in real time from your structured sources, combined with other relevant information found in unstructured repositories. HAQM Q Business uses the analytics and advanced visualization engine in QuickSight to generate accurate and simple-to-understand answers from structured sources.

In this post, we show you how to configure the QuickSight connection from HAQM Q Business and then ask questions to get real-time data and visualizations from QuickSight for structured data in addition to unstructured content.

Solution overview

The QuickSight feature in HAQM Q Business is available on the HAQM Q Business console as well as through HAQM Q Business APIs. This feature is implemented as a plugin within HAQM Q Business. After it’s enabled, this plugin will behave differently than other HAQM Q Business plugins—it will query QuickSight automatically for every user prompt, looking for relevant answers.

For AWS accounts that aren’t subscribed to QuickSight already, the HAQM Q Business admin completes the following steps:

1. Create a QuickSight account.
2. Connect your database in QuickSight to create a dataset.
3. Create a topic in QuickSight, which is then used to make it searchable from your HAQM Q Business application.

When the feature is activated, HAQM Q Business will use your unstructured data sources configured in HAQM Q Business, as well as your structured content available using QuickSight, to generate a rich answer that includes narrative and visualizations. Depending on the question and data in QuickSight, HAQM Q Business may generate one or more visualizations as a response.

Prerequisites

You should have the following prerequisites:

• An AWS account where you can follow the instructions in this post.
• AWS IAM Identity Center set up to be used with HAQM Q Business. For more information, see Configure HAQM Q Business with AWS IAM Identity Center trusted identity propagation.
• At least one HAQM Q Business Pro user that has admin permissions to set up and configure HAQM Q Business. For pricing information, see HAQM Q Business pricing.
• An IAM Identity Center group that will be assigned the QuickSight Admin Pro role, for users who will manage and configure QuickSight.
• If a QuickSight account exists, then it needs to be in the same AWS account and AWS Region as HAQM Q Business, and configured with IAM Identity Center.
• A database that is installed and can be reached from QuickSight to load structured data (or you could create a dataset by uploading a CSV or XLS file). The database also needs credentials to create tables and insert data.
• Sample structured data to load into the database (along with insert statements).

Create an HAQM Q Business application

To use this feature, you need to have an HAQM Q Business application. If you don’t have an existing application, follow the steps in Discover insights from HAQM S3 with HAQM Q S3 connector to create an application along with an HAQM S3 data source. Upload the non-structured document(s) to HAQM S3 and sync the data source.

Create and configure a new QuickSight account

You can skip this section if you already have an existing QuickSight account. To create a QuickSight account, complete the following steps:

1. On the HAQM Q Business console, navigate to your application.
2. Choose HAQM QuickSight in the navigation pane.

3. Choose Create QuickSight account.

4. Under QuickSight account information, enter your account name and an email for account notifications.
5. Under Assign QuickSight Admin Pro roles, choose the IAM Identity Center group you created as a prerequisite.
6. Choose Next.

7. Under Service access, select Create and use a new service role.
8. Choose Authorize.

This will create a QuickSight account, assign the IAM Identity Center group as QuickSight Admin Pro, and authorize HAQM Q Business to access QuickSight.

You will see a dashboard with details for QuickSight. Currently, it will show zero datasets and topics.

9. Choose Go to QuickSight.

You can now proceed to the next section to prepare your data.

Configure an existing QuickSight account

You can skip this section if you followed the previous steps and created a new QuickSight account.

If your current QuickSight account is not on IAM Identity Center, consider using a different AWS account without a QuickSight subscription for the purpose of testing this feature. From that account, you create an HAQM Q Business application on IAM Identity Center and go through the QuickSight integration setup steps on the HAQM Q Business console that will create the QuickSight account for you in IAM Identity Center. Remember to delete that new QuickSight account and HAQM Q Business application after your testing is done to avoid further billing.

Complete the following steps to set up the QuickSight connector from HAQM Q Business for an existing QuickSight account:

1. On the HAQM Q Business console, navigate to your application.
2. Choose HAQM QuickSight in the navigation pane.

3. Choose Authorize QuickSight answers.

4. Under Assign QuickSight Admin Pro roles, choose the IAM Identity Center group you created as a prerequisite.
5. Under Service Access, select Create and use a new service role.
6. Choose Save.

You will see a dashboard with details for QuickSight. If you already have a dataset and topics, they will show up here.

You’re now ready to add a dataset and topics in the next section.

Add data in QuickSight

In this section, we create an HAQM Redshift data source. You can instead create a data source from the database of your choice, use files in HAQM S3, or perform a direct upload of CSV files and connect to it. Refer to Creating a dataset from a database for more details.

To configure your data, complete the following steps:

1. Create a new dataset with HAQM Redshift as a data source.

Configuring this connection offers multiple choices; choose the one that best fits your needs.

2. Create a topic from the dataset. For more information, see Creating a topic.

3. Optionally, create dashboards from the topic. If created, HAQM Q Business can use them.

Ask queries to HAQM Q Business

To start chatting with HAQM Q Business, complete the following steps:

1. On the HAQM Q Business console, navigate to your application.
2. Choose HAQM QuickSight in the navigation pane.

You should see the datasets and topics populated with values.

3. Choose the link under Deployed URL.

We uploaded AWS Cost and Usage Reports for a specific AWS account in QuickSight using HAQM Redshift. We also uploaded HAQM service documentation into a data source using HAQM S3 into HAQM Q Business as unstructured data. We will ask questions related to our AWS costs and show how HAQM Q Business answers questions from both structured and unstructured data.

The following screenshot shows an example question that returns a response from only unstructured data.

The following screenshot shows an example question that returns a response from only structured data.

The following screenshot shows an example question that returns a response from both structured and unstructured data.

The following screenshot shows an example question that returns multiple visualizations from both structured and unstructured data.

Clean up

If you no longer want to use this HAQM Q Business feature, delete the resources you created to avoid future charges:

1. Delete the HAQM Q Business application:
   1. On the HAQM Q Business console, choose Applications in the navigation pane.
   2. Select your application and on the Actions menu, choose Delete.
   3. Enter delete to confirm and choose Delete.

The process can take up to 15 minutes to complete.

2. Delete the S3 bucket:
   1. Empty your S3 bucket.
   2. Delete the bucket.
3. Delete the QuickSight account:
   1. On the HAQM QuickSight console, choose Manage HAQM QuickSight.
   2. Choose Account setting and Manage.
   3. Delete the account.
4. Delete your IAM Identity Center instance.

Conclusion

In this post, we showed how to include answers from your structured sources in your HAQM Q Business applications, using the QuickSight integration. This creates a unified conversational experience for your end-users that saves them time, helps them make better decisions through more complete answers, and improves their productivity.

At AWS re:Invent 2024, we also announced a similar unified experience enabling access to insights from unstructured data sources in HAQM Q in QuickSight powered by HAQM Q Business.

To learn about the new capabilities HAQM Q in QuickSight provides, see QuickSight Plugin.

To learn more about HAQM Q Business, refer to the HAQM Q Business User Guide.

To learn more about configuring a QuickSight dataset, see Manage your HAQM QuickSight datasets more efficiently with the new user interface.

QuickSight also offers querying unstructured data. For more details, refer to Integrate unstructured data into HAQM QuickSight using HAQM Q Business.

About the authors

Jiten Dedhia is a Sr. AIML Solutions Architect with over 20 years of experience in the software industry. He has helped Fortune 500 companies with their AIML/Generative AI needs.

Jean-Pierre Dodel is a Principal Product Manager for HAQM Q Business, responsible for delivering key strategic product capabilities including structured data support in Q Business, RAG. and overall product accuracy optimizations. He brings extensive AI/ML and Enterprise search experience to the team with over 7 years of product leadership at AWS.