There are a number of ways in which companies use big data, but a study released in the summer of 2015 by the SANS Institute reports that one of the most common uses for big data is security –specifically log management. Over 200 professionals in security and compliance participated in the survey, and more than half report that their companies have deployed big data for this purpose. Furthermore, of the respondents who said their companies plan on deploying big data in the next two years, nearly 58 percent said that they anticipate log management will be a priority.
A Natural Use Case for Big Data
Big Data is a natural fit for log management. Think, for a moment, about the Three Vs of big data: Volume, variety and velocity. Now, consider intrusion alerts, log data and other types of security events that occur every day in the world of data security. Information security generates a high volume of data and a wide variety of data that comes in at a high velocity. It is not uncommon for large companies to process billions of security events every single day. The information is pulled from any number of endpoints and every log file looks different.
Without big data deployments, teams of analysts must spend a great deal of time compiling data before they can even think about analyzing even a single security event. With big data in place, however, the process becomes instantly scalable, efficient, and cost effective. What takes weeks for analysts to do manually can be done in virtual real time. Big data platforms also make it easier to spot patterns in security data that might otherwise have gone unnoticed in a traditional – and limited – rule-based methodology.
What Types of Secure Information are Companies Storing?
The SANS Institute survey also revealed the types of secure data that companies are storing. Of the respondents who indicated their companies use big data, 73 percent said it is used to store identifiable information, 64 percent said they use it to hold employee records, 59 percent reported an intellectual property component, 53 percent reported payment card information storage, and 40 percent said they use it for information related to national security. All of this indicates that organizations turn to and trust big data platforms with their most sensitive of information.
What does this shift mean for big data job seekers? Tech professionals with big data skills have found a new niche in information security and vice-versa. While both big data and information security skills have been in demand for quite some time, this marriage signals a shift in hiring needs for many employers. They will need to source highly skilled big data professionals who can not only analyze data in a “traditional” sense, but also those who can help them collect and store their most sensitive data and analyze security information to help keep that data secure.
If you are a big data analytics professional looking for new opportunities to grow your career in data security, contact Onboard Recruitment Advisers today. We specialize in analytics and big data positions in a variety of fields, and we will work with you to connect you to a position that will help you reach your long-term career goals.