AWS Storage Blog
Tag: HAQM Simple Storage Service (HAQM S3)
Build a data lake for streaming data with HAQM S3 Tables and HAQM Data Firehose
Businesses are increasingly adopting real-time data processing to stay ahead of user expectations and market changes. Industries such as retail, finance, manufacturing, and smart cities are using streaming data for everything from optimizing supply chains to detecting fraud and improving urban planning. The ability to use data as it is generated has become a critical […]
Optimizing HAQM FSx for Lustre storage consumption using automatic data tiering with HAQM S3
Managing high-performance file storage can be a significant operational and cost challenge for many organizations, especially those running compute-intensive workloads such as high-performance computing (HPC) or data analytics. This is particularly true for organizations with existing data lakes on HAQM S3 who need POSIX-compliant, high-performance file system access. HAQM FSx for Lustre provides a scalable, […]
Integrating custom metadata with HAQM S3 Metadata
Organizations of all sizes face a common challenge: efficiently managing, organizing, and retrieving vast amounts of digital content. From images and videos to documents and application data, businesses are inundated with information that needs to be stored securely, accessed quickly, and analyzed effectively. The ability to extract, manage, and use metadata from this content is […]
Design patterns for multi-tenant access control on HAQM S3
Large organizations and software as a service (SaaS) platforms often share storage resources across multiple users, groups, or tenants. The design pattern chosen to implement this shared storage can significantly impact how access permissions are managed at scale. This decision is key because it directly affects platforms’ security and ease of scale. A well thought […]
Optimizing data transfers for high throughput life science instruments using AWS DataSync
Healthcare and life sciences (HCLS) customers are generating more data than ever as they integrate the use of omics data with applications in drug discovery, clinical development, molecular diagnostics, and population health. The rate and volume of data that HCLS laboratories generate are a reflection of their lab instrumentation and day-to-day lab operations. Efficiently moving […]
Archiving relational databases to HAQM S3 Glacier storage classes for cost optimization
Many customers are growing their data footprints rapidly, with significantly more data stored in their relational database management systems (RDBMS) than ever before. Additionally, organizations subject to data compliance including the Health Insurance Portability and Accountability Act (HIPAA), the Payment Card Industry Data Security Standard (PCI-DSS) and General Data Protection Regulation (GDPR) are often required […]
Cost-optimized log aggregation and archival in HAQM S3 using s3tar
According to a study by the International Data Corporation (IDC), the global datasphere is expected to grow from 33 zettabytes (ZB) in 2018 to 175 ZB by 2025, a staggering five-fold increase. Organizations that leverage distributed architectures generate a significant portion of their data footprint from observability data, including application logs, metrics, and traces, which […]
Backing up Oracle databases to HAQM S3 at scale
In today’s data-driven world, safeguarding critical information stored in Oracle databases is crucial for enterprises. Companies struggle to efficiently backing up vast amounts of data from hundreds of databases powering enterprise resource planning (ERP) systems and critical applications. These backups must be secure, durable, and easily restorable to ensure business continuity, guard against ransomware, and […]
Adapting to change with data patterns on AWS: The “extend” cloud data pattern
As part of my re:Invent 2024 Innovation Talk, I shared three data patterns that many of our largest AWS customers have adopted. This article focuses on “Extend” which is an emerging data pattern. You can also watch this four-minute video clip on the Extend data pattern if interested. Many companies find great success with the […]
Adapting to change with data patterns on AWS: The “aggregate” cloud data pattern
As part of my re:Invent 2024 Innovation talk, I shared three data patterns that many of our largest AWS customers have adopted. This article focuses on the “Aggregate” cloud data pattern, which is the most commonly adopted across AWS customers. You can also watch this six-minute video clip on the Aggregate data pattern for a […]