Everyone tends to think of Big Data as only being a challenge in Big Organizations. No more. Small and midsize organizations face their own set of big data challenges. Even mobile, remote, and branch offices of large organizations face big data challenges like these smaller, autonomous organizations. To avoid being caught unprepared, there are three specific areas these smaller shops should flag as areas for data growth.
Big Data Drivers in Smaller Shops
Many smaller shops – whether they be autonomous organizations or remote enterprise offices – know their current sources of data growth. These sources generally fall into the following four categories:
In most cases, they know about them, plan for them, and can manage them. However, as technology costs continue to drop, new drivers for data growth loom. To avoid being caught unprepared, smaller shops should flag these areas as potential new sources for explosive data growth.
Application and User Logs
Organizations increasingly want to log application and user activities for multiple purposes. Performing real time and historical analysis on these logs represent the motivation for capturing and storing them.
Proactively detecting and preventing ransomware represents one of the primary reasons for capturing and analyzing all these logs. Only by monitoring and analyzing application, file, and user activity can organizations proactively detect and defeat ransomware attacks.
Capturing and storing all these logs from all these sources generates lots of data which must be stored. This puts the onus on these smaller shops to store all this data.
Machine Generated Data
While akin to application and user logs, the source of machine generated data differs slightly. This data originates from sensors in “dumb” appliances or equipment that organizations deploy in their physical infrastructure.
This data may come from a sensor on any device that powers on. It most typically comes from building infrastructure such as furnaces, thermostats, and light switches. However, it may include appliances or devices such as refrigerators, microwaves, stoves, computers, and laptops.
Here again, organizations may want to capture this data to identify faulty or failing equipment. They may also use it to better understand how this equipment gets used and optimize its use. For instance, if a room is lit and heated but no one is in the room, an organization may programmatically turn off the lights and lower the room’s temperature.
All these sensors generate data these smaller shops may again need to manage and store.
Video surveillance represents perhaps the biggest new driver of data growth. More small shops use video surveillance technology for numerous reasons, though security is the primary driver.
Video surveillance differs from the other sources in that it generates large amounts of data so quickly. Logs and sensor data generate data but rarely at the rate that video surveillance does.
Granted, the amount of data generated and stored by video surveillance technology depends upon multiple variables. These include the number of cameras, the camera resolution, and retention policies for the video, among others. Further, how frequently organizations access this data and use it for tasks for analytics impacts how they should store it.
A Head’s Up
I share these new Big Data challenges that smaller shops may face for two reasons.
First, to give them a head’s up as to the big data challenges that these applications will potentially create. Many already have one or more of these applications in place that generate one or more of these types of data. If the big data challenges these applications may create have not impacted you, great!
However, know that any changes in your environment can impact you in two ways. It may increase how much data you need to store. Alternatively, there may be new performance demands for faster access to data already stored.
Enterprise Features on Small Shop Solutions
In either scenario, this second point comes into play. New storage solutions exist that grant smaller shops access to features once reserved for enterprise solutions. These include cloud connectivity, high availability, high performance, high reliability, and scalability, among others. They even include features such as simplicity of management that enterprises may not even offer.
These solutions specifically target smaller shops that need simpler management, cloud connectivity, higher performance, or some combination thereof. Further, they deliver these solutions at a price point that should fall within the budgets of smaller shops. So regardless if you must deal with logs, sensor data, video surveillance, simple file growth, or all the above, know that cost-effective solutions now exist to meet these new challenges of smaller shops.