Why AutoML?

In the previous chapter, we introduced three broad types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. In this book, we will focus on supervised learning because it is widely used for practical prediction problems involving tables, spreadsheets, and CSV files.

In supervised learning, we learn from examples for which the correct answer is already known. We can study what happened in the past, search for patterns in that historical data, and use those patterns to make predictions about new cases.

This sounds straightforward, but creating a useful machine learning system involves a surprising amount of experimentation. There are many possible ways to prepare the data, many learning algorithms to try, and many settings that can affect the results. Repeating all these experiments manually takes time and requires technical knowledge.

AutoML was created to automate much of this repetitive work.

Learning from Known Examples

Let us return to the example of predicting house prices.

Suppose we have information about houses that were sold in the past. For every house, we know properties such as its size, location, number of rooms, age, and condition. We also know the final selling price.

Because the selling price is known for these historical examples, the computer can search for relationships between the properties of a house and its price. It may discover that larger houses usually cost more, but it may also find more subtle patterns. Perhaps the value of an additional room depends on the location. Maybe the age of a house matters less when the building has been renovated. A large garden may be especially valuable in one neighborhood but less important in another.

After learning from previous sales, the computer can estimate the price of a house that it has not seen before. Such a prediction could be useful when searching through property listings. If the predicted value is considerably higher than the listed price, the house may deserve closer attention. If the predicted value is much lower than the asking price, the buyer may want to investigate why the property is so expensive.

The prediction should not be treated as the true value of the house. The data may not include information about a beautiful view, a damaged roof, a noisy road, or an especially attractive interior. Instead, the prediction can be used as a screening tool. It can help a person examine a large number of listings and decide which ones are worth investigating more carefully.

This is a typical supervised learning problem. We have examples from the past, the correct answer is known for those examples, and we want to predict the answer for new cases.

The same approach can be applied to many questions. We can learn from previous customers to predict who may cancel a subscription, from earlier insurance claims to estimate future costs, or from historical patient data to estimate the risk of a medical event.

The idea is simple. The difficulty begins when we try to decide exactly how the computer should learn from these examples.

One Problem, Many Possible Methods

There is no single machine learning method that works best for every dataset.

For the house-price problem, we could try a simple method that searches for approximately straight relationships between the house properties and the price. We could also use a decision tree, which learns a sequence of decisions such as whether the house is large, whether it is in a particular location, or whether it is relatively new.

Other methods build many trees and combine their predictions. Some gradually improve earlier predictions by concentrating on previous mistakes. Neural networks use another approach and may be able to represent complicated relationships between the available values.

Each of these learning methods is called a machine learning algorithm. We do not need to understand their mathematical details yet. For now, it is enough to know that they learn in different ways and may produce different results.

An algorithm that works very well for one problem may perform poorly on another. A method that is effective for a small medical dataset may not be the best choice for millions of financial transactions. A simple method may work surprisingly well when the relationships are clear, while a more complicated method may be needed when many values interact with one another.

This creates an immediate practical question:

Which algorithm should we use?

Unfortunately, we usually cannot answer this question by looking only at the name or subject of the dataset. We need to try several possibilities and compare their results.

Every Algorithm Has Settings

Choosing an algorithm is only the beginning. Most machine learning algorithms also have settings that control how they learn.

A decision tree, for example, can be allowed to grow very deep, creating many detailed decisions, or it can be kept smaller and simpler. A method based on many trees may use a few dozen trees or several hundred. Other settings can control how quickly learning proceeds, how much data is used at each step, or how strongly the method avoids unnecessary complexity.

These settings are often called hyperparameters. A good choice can improve the results, while a poor choice can cause the same algorithm to perform much worse.

The number of possible combinations quickly becomes large. We may have several algorithms, each with many settings, and each setting may have several possible values. The original Auto-WEKA research described this as a combined problem of selecting a learning algorithm and choosing its hyperparameters. The researchers showed that these two decisions could be searched automatically instead of being handled separately by a person.

A person can test these choices manually, but the work becomes repetitive. We select an algorithm, choose some settings, train it, measure its performance, record the result, change the settings, and start again. Then we repeat the same process with another algorithm.

This experimentation is a normal part of machine learning, but it can consume a large amount of time.

The Manual Experimentation Loop

Imagine that we begin the house-price project with a decision tree. We train it and measure the quality of its predictions. Next, we change one of its settings and train it again. We try a smaller tree, then a larger tree, and compare the results.

After that, we test another algorithm. Before we can use it, we may need to prepare the data differently. We train the new method with several combinations of settings and save each result. Then we continue with another algorithm.

During this process, we must make sure that every experiment is evaluated fairly. We cannot judge one method using easy examples and another using difficult examples. We need a consistent validation process so that the results can be compared.

We may also need to handle missing values, convert text categories into a form an algorithm can use, or remove columns that aren't helpful. Sometimes it also helps to create new information from existing columns, or combine several successful methods together.

A complete experiment may therefore contain many steps:

1. Prepare the data.
2. Select an algorithm.
3. Choose its settings.
4. Train it.
5. Evaluate its predictions.
6. Save the result.
7. Change something.
8. Repeat the process.
9. Compare all experiments.
10. Select the most useful result.

None of these steps is mysterious when considered separately. The difficulty comes from the number of times they must be repeated and from the care needed to compare the results correctly.

This is the problem that AutoML tries to solve.

What Is AutoML?

AutoML, short for Automated Machine Learning, is a way of automating repeated parts of the machine learning process.

Instead of manually testing every algorithm and every combination of settings, we describe the prediction problem and give the AutoML system a time budget or a set of constraints. The system then runs a sequence of experiments, compares their results, and helps us identify a strong solution.

Early AutoML research focused strongly on automating algorithm selection and hyperparameter optimization. Auto-WEKA combined these two tasks in one automated search, while auto-sklearn extended the idea by using information from previous datasets and by creating ensembles from models evaluated during the search.

AutoML has since developed into a broader area that covers different parts of building machine learning pipelines. The open-access book AutoML: Methods, Systems, Challenges presents AutoML as a field concerned with automating machine learning tasks that would otherwise require repeated expert decisions and experiments.

The purpose of AutoML is not to remove people from machine learning. Its purpose is to let computers perform more of the repetitive technical search so that people can spend more time understanding the problem, checking the data, and deciding how the results should be used.

What Can AutoML Automate?

The exact behavior depends on the AutoML system, but tools designed for tabular data commonly automate several parts of the experimentation process.

Preparing Data for Algorithms

Real datasets often contain missing values and categorical columns such as city names, product types, or customer groups. Different algorithms may require these values to be prepared in different ways.

An AutoML system can create and test suitable data-preparation steps as part of its experiments. MLJAR AutoML, for example, includes preprocessing for missing values and categorical columns and can apply additional feature-engineering techniques in its more advanced modes.

This does not mean that AutoML can repair every data problem. It can handle many technical transformations, but it cannot know whether a value is incorrect, whether an important column is missing, or whether the dataset was collected fairly.

Trying Different Algorithms

AutoML can train several types of machine learning algorithms rather than requiring us to select one in advance. Each algorithm approaches the problem differently, so testing several of them gives us a better chance of finding one that works well with our data.

For example, an AutoML experiment may compare simple methods, decision trees, random forests, gradient-boosting methods, neural networks, and other approaches. MLJAR AutoML supports a range of algorithms and can also create ensembles and stacked ensembles from the trained solutions.

Searching Algorithm Settings

AutoML can experiment with different hyperparameter values. Instead of asking the user to decide exactly how deep a tree should be or how quickly an algorithm should learn, the system tests different possibilities and measures their results.

This search is not always an attempt to test every possible combination. There may be far too many. AutoML systems use search strategies that try to spend their time on promising experiments.

Evaluating Experiments Consistently

Every experiment should be tested on data that was not used to teach it. Otherwise, a method may appear successful simply because it remembers the examples it has already seen.

AutoML can apply the same validation strategy to many experiments, record their results, and place them in a leaderboard. This makes it easier to compare algorithms and settings under similar conditions.

Validation is so important that we will devote later chapters to understanding it. AutoML can perform the splitting and calculations, but a person still needs to decide whether the validation strategy reflects the real problem.

Combining Several Results

Sometimes the best prediction does not come from one algorithm. It comes from combining several of them.

One method may perform better for some houses, while another performs better for different ones. Their mistakes may not be identical. By combining their predictions, we may obtain a result that is more reliable than either prediction alone.

This approach is called an ensemble. Automated ensemble construction was one of the important components of auto-sklearn, and ensemble methods are also available in MLJAR AutoML.

Organizing the Results

A manual experiment can leave behind a confusing collection of notebooks, filenames, settings, and measurements. AutoML can organize the trained solutions, record their performance, and generate reports that help us understand what happened.

In MLJAR AutoML, the results are stored in an output directory containing information about the trained models and their performance. Later chapters will explain this directory, the leaderboard, evaluation metrics, learning curves, feature importance, and prediction explanations.

Why Is This Automation Useful?

The most obvious benefit of AutoML is time. A computer can repeat the same training and evaluation process many times without becoming bored or forgetting to record a result. This allows us to test more possibilities than we might reasonably examine by hand.

Automation also improves consistency. When experiments are performed through the same system, they are more likely to use the same validation process and reporting format. This makes comparisons easier and reduces the risk of accidentally evaluating different methods in different ways.

AutoML can also provide a strong starting point. A beginner does not need to know in advance which algorithm is likely to work best. An experienced practitioner can use AutoML to establish a baseline before investing time in custom improvements.

Research systems such as auto-sklearn were created partly to reduce the tedious work of designing machine learning pipelines, while still searching across many possible methods and settings. More recent AutoML research continues to study how this search can produce strong results under limited training time.

However, faster experimentation does not mean that every AutoML result is automatically useful.

What AutoML Cannot Decide for Us

AutoML automates a lot of the experimentation, but it doesn't run the whole project. A few decisions will always need you.

The first is the question itself. AutoML can't tell you which prediction would actually be useful — that depends on who will use it and what they'll do with the result. A computer can search for patterns, but only you know why the prediction matters in the first place.

The second is the data. AutoML can fill in missing values and handle messy columns, but it has no way of knowing that a sensor was broken, that a price was entered in the wrong currency, or that a whole group of cases was left out of the dataset. That kind of thing only shows up when someone who understands the data takes a look.

The third is what counts as a costly mistake. Two models can have similar overall accuracy while making very different kinds of errors — missing something important versus flagging something harmless. AutoML can measure both, but deciding which one matters more in your situation is a judgment call, not a calculation.

None of this makes AutoML less useful — it just means it's built to handle the repetitive part well, while leaving the parts that need context and judgment to you.

AutoML Is an Assistant, Not an Autopilot

The word automated can create the wrong impression. It may suggest that we can provide any spreadsheet, press a button, and trust whatever appears at the end. That is not how responsible AutoML should be used.

A better way to think about AutoML is as a skilled assistant. It can perform many repetitive experiments, keep records, compare results, and prepare reports. It can help us explore more possibilities in less time. However, the assistant does not own the problem and does not understand all its consequences.

The human remains responsible for defining the question, understanding the data, reviewing the results, and deciding whether the predictions should be used. This division of work is one of the greatest strengths of AutoML. Computers perform repeated calculations quickly and consistently. People contribute context, experience, values, and judgment.

AutoML is most useful when these strengths work together.

AutoML Does Not Search Forever

There are usually more possible experiments than we could ever run. Even an automated system must decide where to spend its time.

For this reason, AutoML commonly works within a time budget. We may allow it to search for a few minutes while exploring a dataset, several hours when building a stronger solution, or longer when prediction quality is especially important.

A larger time budget allows the system to try more algorithms and settings, but more training time does not guarantee a perfect result. The quality of the data and the strength of the underlying pattern remain more important than the number of experiments.

This leads to a practical lesson that will appear throughout the book: AutoML helps us search efficiently, but it cannot create information that is not present in the data.

A Better Way to Begin Machine Learning

Without AutoML, a beginner may spend a great deal of time deciding which algorithm to try, learning its settings, writing repeated training code, and organizing the results. This technical work can become a barrier before the person has had a chance to investigate whether the original idea is useful.

AutoML changes the starting point. Instead of beginning with the question “Which algorithm should I configure?”, we can begin with more meaningful questions:

• What do I want to predict?
• What examples do I have?
• How would this prediction be used?
• What type of mistake would be most harmful?
• Does the result appear useful on new data?

The algorithms and settings still matter, but AutoML helps us manage them. This allows domain experts to participate more directly in machine learning projects while gradually learning the technical concepts behind the process.

AutoML also benefits experienced practitioners. It can provide a strong baseline, reveal that an unexpectedly simple method works well, or quickly compare approaches before more specialized work begins.

It is not a replacement for machine learning knowledge. It is a practical way to use that knowledge more efficiently.

Chapter Summary

This book focuses mainly on supervised learning, where a computer learns from historical examples with known answers. A house-price system, for example, can study earlier property sales and use the patterns it finds to estimate the value of a new house.

The catch is that many algorithms could solve the same problem, and each one has its own settings to tune. Finding a solution that actually works means running the same loop again and again — prepare the data, train a method, adjust its settings, check the results, and compare. It's not difficult work, just repetitive.

This is exactly what AutoML takes off your plate. It can prepare the data, try out several algorithms, search their settings, evaluate everything consistently, combine the best results, and keep it all organized — so you can explore far more possibilities than you ever could by hand, whether you're just starting out or you've been doing this for years.

What it still can't do is decide the things that matter most. Choosing a meaningful problem, judging whether the data can be trusted, and deciding which kinds of mistakes matter more than others — these stay with you.

Think of AutoML as a skilled assistant, not an autopilot. It handles the repetitive technical work quickly and reliably, while you bring the context, the judgment, and the final call.

In the next chapter, we'll look at the kinds of questions AutoML can answer for you — and the difference between predicting a category, a probability, and a number.