Data analysis techniques in the era of ‘big data’

Internet and massive data generation have made data analysis techniques an indispensable tool to exploit the full potential of a company.

Regardless of the sector or the size of the company, the information extracted from big data is a great support for decision-making in any area .

For example, to promote small businesses online, such as a hotel with capacity for thirty guests, it is essential to know how many customers visit the website and their country of origin.

As the size of the business is larger and covers a broader spectrum of the market, as may be the case of a company in the insurance sector, you will have to know well the different types of clients it serves and also have metrics available to you. help to set prices, to know what coverage to offer them and at what rates. For a large company in the fashion sector, for example, an error in the prediction of its sales can cost its supply chain a great cost or damage its image due to the possibility of leaving its customers dissatisfied.

Data analysis techniques:  2 criteria to select them

The big data or internet of things exposed to new sets of information that require tailored approaches. There are different analytical techniques that are adapted both to the characteristics of the data collected and to the questions that you want to answer. These techniques respond mainly to two approaches:

  • The purpose for which the data is analyzed.
  • The nature of the data.

Data analysis techniques according to the objective

There are many alternatives to analyze a problem: from simple and everyday metrics to complex techniques that require a large investment of time. The appropriate analysis technique will be decided based on the question that needs to be answered or the type of decision that must be made.

Descriptive techniques

If the objective is to understand the reality facing the business (“where are there more customers?”, “How much is the business growing?”), Descriptive data analysis is a good tool to find the answer:

  1. Accounts, sums and means
  2. Variation rates
  3. Frequency tables
  4. A / B test
  5. Factorial and cluster analysis
  6. Decision trees
  7. Spatial analysis
  8. Application of graph theory

In this way, the small hotel will have to prepare a scorecard using the simplest techniques that measure the traffic on its website , such as the growth of visits. As the market you are facing or the diversity of customers increases, it will be necessary to use more complex techniques, such as A / B testing, to evaluate which ad has generated the most conversion.

Predictive techniques

Do you want to anticipate events and know what is going to happen? (“When will there be more customers?”, “When do I have to start offering the service?”) Use predictive techniques to get the answer :

  1. Temporal series
  2. Regression techniques
  3. Neural networks
  4. Machine learning and deep learning
  5. Algorithms boosting as XGBoost

Thus, using time series and regression techniques, the large fashion company will have sales predictions throughout its network and will be able to make the best decisions regarding supply.

Prescriptive techniques

If what you need is a recommendation , use prescriptive techniques based on the identification of cause / effect rules or optimization algorithms:

  1. Conditional probability methods
  2. Regression techniques
  3. Association rules
  4. Monte Carlo method and stochastic simulation
  5. Genetic algorithms
  6. Spatial optimization techniques

In this way, an insurer can know the sensitivity to the price according to the different types of customers and products with regression techniques, and make better decisions when setting prices through simulations.

Data analysis techniques according to nature

The ability to generate, accumulate and process information has multiplied exponentially and this has made it necessary to adapt analysis techniques. For this reason, technological solutions are also more sophisticated, and this evolution comes from both veteran manufacturers (SAS or IBM, for example) and new collaborative environments (R, Spark, Python or Scala, among others).

Analysis techniques according to the volume of data

The volume of information has increased and the techniques that most take advantage of this factor are those that exploit machine learning in two ways:

  • Continuous recalibration of the model as learning (a neural network, for example).
  • The ability to combine different models, whether it is the same technique applied to different subsamples or the execution of different techniques that compete with each other (bagging, random forest or boosting techniques, among others).

Analysis techniques according to the type of data

The type of information is also more varied . Now the databases are not only numerical, it is also possible to work with text, voice or image, and new techniques emerge that allow us to achieve all the objectives that business management requires today.

The exploitation of texts or images allows the following types of analysis to be carried out:

  • The semantic analysis, starting from the bases of natural language.
  • Sentiment analysis, capable of differentiating moods based on the terms used.
  • Multimedia analysis that makes it possible to identify patterns from images, such as the tracking of people and objects within a video (an example is the developments made by the University of California with which Cognodata collaborates).

We have a wide variety of tools and techniques for data analysis.

The purpose of this analysis and the nature of the data available determine the most appropriate technique. In this context, what is essential for a data-driven company is to have an ally to navigate among the set of solutions on the market and do it in the right direction .