Automation: the future of data science and machine learning?

Machine learning has been one of the biggest advances in the history of computing and is now seen as being able to play an important role in the field of big data and analytics. Big data analytics is a huge challenge from an enterprise perspective. For example, activities such as understanding the large number of different data formats, analysing data preparation and filtering redundant data can be resource intensive. Recruiting data scientist specialists is an expensive proposition and not a means to an end for every company. Experts believe that machine learning can automate many of the tasks associated with analytics – both routine and complex. Automated machine learning can free up significant resources that could be used for more complex and innovative work. Machine learning seems to be moving in this direction all the time.

Automation in the context of information technology

In IT, automation is the connection of different systems and software, enabling them to perform specific tasks without any human intervention. In IT, automated systems can perform both simple and complex jobs. An example of a simple job might be integrating forms with PDFs and sending documents to the correct recipient, while providing off-site backups might be an example of a complex job.

To do your job properly, you need to program or give clear instructions to the automated system. Each time an automated system is needed to modify the scope of its job, the program or instruction set needs to be updated by someone. Although the automated system is effective in its job, errors can occur for a variety of reasons. When errors occur, the root cause needs to be identified and corrected. Clearly, to do its job, an automated system is completely dependent on humans. The more complex the nature of the work, the higher the likelihood of errors and problems.

A common example of automation in the IT industry is the automation of testing of web-based user interfaces. Test cases are fed into the automation script and the user interface is tested accordingly. (For more on the practical application of machine learning, see Machine Learning and Hadoop in Next Generation Fraud Detection.)

The argument in favour of automation is that it performs routine and repeatable tasks and frees up employees to do more complex and creative tasks. However, it is also argued that automation has excluded a large number of tasks or roles previously performed by humans. Now, with machine learning entering various industries, automation can add a new dimension.

The future of automated machine learning?

The essence of machine learning is the ability of a system to continuously learn from data and evolve without human intervention. Machine learning is capable of acting like a human brain. For example, recommendation engines on e-commerce sites can assess a user’s unique preferences and tastes and provide recommendations on the most appropriate products and services to choose from. Given this capability, machine learning is seen as ideal for automating complex tasks associated with big data and analytics. It has overcome the major limitations of traditional automated systems that do not allow for human intervention on a regular basis. There are multiple case studies that demonstrate the ability of machine learning to perform complex data analysis tasks, which will be discussed later in this paper.

As already noted, big data analytics is a challenging proposition for businesses, which can be partially delegated to machine learning systems. From a business perspective, this can bring many benefits such as freeing up data science resources for more creative and mission critical tasks, higher workloads, less time to complete tasks and cost effectiveness.

Case study

In 2015, MIT researchers began working on a data science tool that can create predictive data models from large amounts of raw data using a technique called deep feature synthesis algorithms. The scientists claim the algorithm can combine the best features of machine learning. According to the scientists, they have tested it on three different datasets and are expanding the testing to include more. In a paper to be presented at the International Conference on Data Science and Analytics, researchers James Max Kanter and Kalyan Veeramachaneni said, “Using an automated tuning process, we optimise the entire path without human involvement, allowing it to generalise to different datasets “.

Let’s look at the complexity of the task: the algorithm has what is known as an auto-adjustment capability, with the help of which insights or values can be obtained or extracted from raw data (such as age or gender), after which predictive data models can be created. The algorithm uses complex mathematical functions and a probability theory called Gaussian Copula. It is therefore easy to understand the level of complexity that the algorithm can handle. This technique has also won prizes in competitions.

Machine learning could replace homework

It is being discussed around the world that machine learning could replace many jobs because it performs tasks with the efficiency of the human brain. In fact, there is some concern that machine learning will replace data scientists, and there seems to be a basis for such concern.

For the average user who does not have data analysis skills but has varying degrees of analytical needs in their daily lives, it is not feasible to use computers that can analyse huge volumes of data and provide analysis data. However, Natural Language Processing (NLP) techniques can overcome this limitation by teaching computers to accept and process natural human language. In this way, the average user does not need sophisticated analytical functions or skills.

IBM believes that the need for data scientists can be minimised or eliminated through its product, the Watson Natural Language Analytics Platform. According to Marc Atschuller, vice president of analytics and business intelligence at Watson, “With a cognitive system like Watson, you just ask your question – or if you don’t have a question, you just upload your data and Watson can look at it and infer what you might want to know. ”


Automation is the next logical step in machine learning and we are already experiencing the effects in our everyday lives – e-commerce sites, Facebook friend suggestions, LinkedIn network suggestions and Airbnb search rankings. Considering the examples given, there is no doubt that this can be attributed to the quality of the output produced by automated machine learning systems. For all its qualities and benefits, the idea of machine learning causing huge unemployment seems a bit of an overreaction. Machines have been replacing humans in many parts of our lives for decades, but humans have evolved and adapted to stay relevant in the industry. According to the view, machine learning for all its disruption is just another wave that people will adapt to.

Post time: Aug-03-2021