Demystifying the use of the Parquet file format for time series
In the world of data, the Parquet format plays an important role and it might be tempting to use it for storing time series.
Stay updated with breaking news from Ted Dunning. Get real-time updates on events, politics, business, and more. Visit us for reliable news and exclusive interviews.
In the world of data, the Parquet format plays an important role and it might be tempting to use it for storing time series.
DataOps (data operations) brings together DevOps teams with data engineers and data scientists to provide the tools, processes, and skills to enable the data-driven enterprise.
Finding, accessing, and using data effectively is one of the biggest issues companies face. Here’s a new approach to solving the challenge.
shutterstock When you work with large-scale data, it’s important to “…know when to hold ‘em, know when to fold ‘em…”. Most modern businesses rely on large-scale data, so it’s natural to focus on the best ways to quickly ingest and store huge amounts of data. But people often overlook how to delete data in very large-scale systems. It may sound simple, but it becomes a non-trivial task when data sets are in the tens of terabytes to petabyte or even exabyte scale. The challenge of deleting large-scale data is a widespread issue that can have significant conse...
shutterstock Unlike Las Vegas, what happens at the edge doesn’t stay at the edge. And that's half the challenge. People commonly think of edge computing as a glorified form of data acquisition or a local digital process control. In reality, edge is a lot more than both of those. It’s true that edge involves many data sources, usually at geo-distributed locations. But keep in mind, it’s the aggregate of that data that holds the key to value and insights. Analysis of the combined data is carried out at core data centers, and actions guided by the resulting insights often ne...