Internet of Things: The Era of Even Bigger Data
For many years, people have been talking about the accelerating volume and velocity of data. Open any technology publication and you’ll see stories offering advice on how companies can tame their big data and clean up their data lake. In fact, some of those stories are authored by my colleagues! But another dimension to this ever-growing volume of data is the Internet of Things (IoT).
According to Wikipedia:
The internet of things (IoT) is the network of physical devices, vehicles, buildings and other items—embedded with electronics, software, sensors, actuators, and network connectivity that enable these objects to collect and exchange data
Think about it like this: do your customers use your mobile app? That is a data-generating sensor. They drive your car? Full of sensors producing data. Jet engine: massive data production in operation. They walk through your store? Where were they, what did they look at and touch? Every thing nowadays can host sensors that generate data and teach you things about your products, customers, and services. This is a true data fire hose so your organization really needs to embrace data-driven otherwise you will be hard pressed to find signal in this massive data noise.
Although IoT may be a new(ish) concept to many of you, it’s actually been around for a while. For over 20 years, researchers in universities worldwide have been investigating how they can put sensors on objects and IP addresses on things. A classic example that you may recognize is the concept of a smart fridge. It’s a refrigerator that connects to the Internet, alerts you when you’ve run out of milk, and orders the milk for you. Pretty cool, right?
Until fairly recently, IoT stayed mainly in the halls of academia. But all that has changed. Today, Internet of Things is very much a part of our everyday lives. Think about it. We have thermostats that can connect to the internet. They collect information about you and apply it to do things like automatically enable and disable the thermostat based on your behavior or on the basis of the weather.
Let me share another example. A few months back, I visited a technology firm in Silicon Valley. In their cafeteria, they had a soda machine with a menu, much like you see in many restaurants today. I picked cola with lemon. The physical machine has a cartridge inside that mixed up my cola with lemon. But that soda machine is also an IoT device. It has a device connected to a server that collects data. It recorded that I picked cola mixed with lemon, on a specific date and time. And it merged that data with data it collects from all its machines around the world. I’m certain that the company that owns the machine uses the data to drive innovation (e.g., which mixes are popular where and when – and which mix cartridges should we push).
Seems innocent enough, right? But things actually get a bit scarier. Many companies are now adding sensors to their machines that go even farther. They’re adding cameras on the device that see you arrive. They perform face recognition so that they can present you with the most popular options based on your latest choices. These machines combine data with sensors to make the experience personal. You’re no longer a demographic, but an individual standing in front of the IoT device. And it recognizes you. You think you’re ordering a drink, but the company behind the machine is collecting data about you – your preferences, the time of day you visit the machine, your photo. And they collect this data without you realizing it. So the real question becomes how are they using, transferring, and securing this data. How is your data being used – for what purpose and in what way?
And think about the volume of data that IoT adds to the world. It makes the “big dataâ€ we’ve been talking about for all these years really just “relatively big data.â€ Internet of Things introduces an enormously higher volume and frequency of big data. It’s not just generated by transactions – it’s generated by machines themselves. These machines are always on, and they are constantly generating a flood of data. And how we deal with that flood requires an entirely new set of requirements.
Now, we need to consider:
- How do we interpret it?
- Is it quality information?
- What is signal vs noise?
- Is it secure?
- How do we manage the volume itself?
- Where do you store it?
- What about privacy and data protection?
All of these considerations are ones we consider today, but in the world of IoT, we need to think about them in an entirely different way. True, data-driven companies are embracing IoT and using it to drive innovation. They are being extreme in how they use data to be truly data-driven. Traditional companies are jumping on the IoT bandwagon as well – but many times, they struggle to deal with the data deluge it creates. They don’t have the right systems, the right people, or the right interpretation to use it for true competitive edge.
So I ask you – if your organization is talking about IoT, are you also talking about the extreme ways in which data can drive your business? Are you considering the sheer volume of data that Internet of Things generate, and how you’ll use it to your advantage?
If you’re not, you should be.
Stan is the co-founder and CTO at Collibra and leads the global product organization. He’s responsible for product management and UX, Collibra’s Center of Excellence, and Collibra University, Collibra’s online learning platform. Prior to founding the company he was a senior researcher at the Vrije Universiteit of Brussels, a leading semantic research center in Europe, performing application-oriented research in semantics.