Data driven world and networking

Hi Redditors!

I'm (yet again) wondering a lot of things related to how our professional world currently works...

I've been reading a lot lately about "data driven companies" as in, companies that use data to make decisions, but not in the regular sense (as in, consultant reports of stats, or departament reports on effectiveness), basically the approach is to process huge amounts of data (big data) via machine learning algorithms, to make smarter predictions or to "let the data and algorithms make such decisions for us"

See for instance Telefonica Group (major european and latin american telco) releasing "Aurora" to integrate every layer of this company by means of data. I'm not a data scientist, but I do find all of this fascinating (and quite complicated to be honest).

Now in networking we do use data to make decisions, information about traffic stats, peaks, CPU, type of traffic, complaints, etc. Drive us to make decisions about the future and about what may be wrong and what may be ok, but this is all "a manual approach". So I want us to focus on the "automated" part of this, using algorithms to solve problems.

Is anyone working (or has worked) in a networking project related to big data, data science. Or has anyone integrated this kind of approach into networking to come with neat ways to solve problems. If so, could you share what you're doing? what's the purpose, approach, what were you trying to solve exactly? I'm trying to get ideas on this.

What are your opinions on this field and networking? I once attended a conference where one of the persons who talked, said they were using big data and machine learning to predict failures, they basically parsed all the logs of all the equipment they had and trained the system based on previous problems, seems the system managed to predict with a high amount of confidence when a network outage was about to happen and where. I personally think this will potentially make our lives easier (and harder with all the new required skill set just to get in).