10 Big Data Technology Trends

Machine Learning

When people thinking of artificial intelligence laying waste to human intellect, their first thought routinely shifts to an event such as chess supercomputer Deep Blue’s landmark victory over Garry Kasparov in 1997. However, while this pivotal moment in artificial intelligence was worthy of its praise, it was mostly a result of a powerful computer brute-forcing its way into winning positions; that is, the computer wasn’t really learning in the way that modern AI is. For an example of machine learning, Watson’s demolition in a game of Jeopardy was the result of machine learning. Similarly, Lee Sedol, a Go player who is generally recognised as the best Go player in the world, was recently defeated by Google’s DeepMind program called AlphaGo. What makes this event special is that Go was considered to be a game that was more ‘human’ in nature — a game that required abstract thought and positional nuance.

Data Visualisation

Data visualisation has long been an effective tool in the staff-room PowerPoint presentation. However, times have changed since the pie charts of yore. The thing about big data, though, is that its conversion into a visual format isn’t just informative — it can be downright beautiful. Take a look at this mesmerising visualisation of pi to see what we mean:

Pi represented graphically (by Martin Krzywinski)

Source: http://mkweb.bcgsc.ca

Scalability

Scalability is the cornerstone of any growing business. While it’s pretty logical that businesses should generally seek to grow, it is important to keep in mind that scaling back to keep things manageable can also be an effective strategy in the right circumstances. Companies getting too ambitious and spreading themselves too thin is a tale as old as time. No SQL databases such as Apache Cassandra help keep your data well managed.

Storage Optimisation

Optimising data has been an issue for many companies since the invention of business computing. There are many ways to streamline your data, but cloud hosting has undoubtedly been the big winner over the past decade or so. With companies routinely parsing terabytes of data to gain an edge, learning how to compress data and get the biggest byte for the buck has practically become a prerequisite for all CFOs. There are hundreds of cloud services to choose from, so taking time to find the one that suits your needs is of paramount importance.

Compression

You can have the best cloud-hosting service in the world, but it will be a graceless fall from cloud nine if you don’t compress the data properly in the first place. If you’re dealing with, say, a million files, each megabyte of data per file adds up to a whopping 1 TB of data in total. Therefore, think about trimming the fat when it comes to processing huge databases. Thankfully, text-based databases are very gentle space-wise, but other file formats can be much less forgiving.

Fraud Detection

Embezzlement has historically been difficult to detect in many companies. A shrewd insider who knows a company’s inner workings can fleece a company for millions if they know what they’re doing. The proper utilisation of big data can allow companies to detect incongruous behaviour in real time.

Retail

The importance of big data for retail cannot be overstated. Walmart, for example, handles more than one million transactions each hour; these are then automatically plugged into databases (totalling more than 2,500 TB), and prices can be optimised to maximise Walmart’s bottom line.

Science

Another use of big data that isn’t just about maximising corporate profit is to do with science. The Large Hadron Collider, for example, was churning out 25,000 TB of data each year at its peak. That’s more than 8.5 continuous years of uncompressed 4K video. Yikes!

Calculating Risk

The most successful companies didn’t get where they are today by shunning risk. Factoring in risk is much easier with the help of big data. Big data can help companies strategically manage the amount of risk in their portfolios.

Analytics

Last, but certainly not least, is the important role of data analytics. What good is big data if you can’t learn from it? The job of a data analyst is to plug the so-called leaks. By deploying predictive models, you can help assure the growth and success of your company.