By Hannah Deen
What actually is Data Science and Machine Learning?
Illuminating common misconceptions and a crash course in Machine Learning basics that will give you a better understanding of how you can use AI in your business.
Here at Qlouder, we get a lot of people coming to us with the dream of incorporating Machine Learning and Big Data into their company without really knowing where to start. Everyone wants a piece of the AI pie! More often than not, these people have unrealistic expectations – and why wouldn’t they? The media is full of seemingly surreal-sounding stories of the unending possibilities of Artificial Intelligence.
You may have noticed that I have already used three terms: Data Science, Machine Learning and Artificial Intelligence. These terms are often used interchangeably and incorrectly and though there is some controversy over the correct definitions of each of the terms, I will give you my opinion.
Machine Learning is a term used to describe algorithms that are used by a machine to learn to do certain tasks. I’ll elaborate on these kinds of tasks in a moment, but to give you an idea, through Machine Learning we can train a machine to tell us what is in an image, or tell us if an email is spam or not. We can also train a machine to predict how our stocks will change over Christmas, or where to allocate our delivery trucks. An algorithm provides us with an answer to a single question.
Artificial Intelligence is a broader term which refers to the ability of machines to make decisions, specific or general. Machine Learning is a process, and Artificial Intelligence is the outcome. Artificial General Intelligence is the goal that many researchers dream of. Where a machine is able to make different kinds of decisions. It’s the kind of AI we think of when we talk about Space Odyssey’s HAL 9000, or her’s Samantha, but not the kind that currently exists or can pilot your business while you kick back in Cuba..
Finally, Data Science is the study and practice of all things data. It’s a generic term for expertise in data munging, data analysis, Machine Learning and data visualization. Our Data Science sprint covers everything from building a data lake, a reporting dashboard, a web-scraper, you name it! In contrast, our Machine Learning sprint is much more focused on building, for example, a neural network using TensorFlow, that can be used within the business to automate classification.
The Common Misconceptions
Now that we’ve got our terminology down, let’s address some common misconceptions of what Machine Learning is capable of.
“All we need is data, and lots of it.”
People hear so much about big data and how much data you need to train a neural network that they imagine that it is the only important thing in the Machine Learning world. As long as I have terabytes of data, I’m all set. The problem is that if your data is bad data, it doesn’t really matter how much you have. Let me give you an example. Once upon a time, the US Army wanted to train an algorithm to detect army tanks hiding among the trees. Their dataset was made up of images of the same forest, one set had tanks hiding in the trees, and one didn’t. The training went well, the neural network they trained managed to say correctly if there was a tank in the forest in the test set of images. However, on new images the results of the algorithm seemed totally random. After much confusion, the researchers realized that the images of tanks had been taken on cloudy days and the images without tanks had been taken on sunny days.
It wouldn’t have mattered if the algorithm had 50, 5000 or 5000000 images, it was just really good at detecting whether it was a sunny or a cloudy day. We have to think carefully about the data we’re using and why, to answer a specific question, or train on a specific task, we need specific data.
“Models keep learning, as long as you feed them data.”
This comes up a lot. The idea is that you build a model, serve it somewhere and then it learns after it’s built. Forever. The model gets cleverer and cleverer the longer it is active because it learns on the job, so to speak.
Generally, this is not how models work, The learning in Machine Learning is a process that happens when you initially train a model. What it boils down to, is the stage we call ‘training’. During the model training period, the model -or ‘machine’- learns to perform a specific task. To give an example, Machine Learning is often used for classification tasks. A binary classifier is an algorithm used to determine whether something is one thing or not.
So what can Machine Learning do?
The types of things a single Machine Learning model can be trained to do are quite limited. This makes it easier to come up with realistic challenges that can help in your business.
1. Is it this or that?
This kind of problem is termed a classification problem.. Examples include, will this pipe break soon, yes or no? Is the animal in this picture a cat or a dog? And one of our recent projects, does this picture contain fire, smoke or neither? You can imagine the cases in which automatically detecting whether something is present in images, or classifying whether something will break or not could support health, safety and maintenance in many businesses.
2. Is this right?
This kind of problem uses anomaly detection algorithms. Anomaly detection is often used in detecting fraud or again alerting to breakages in systems.
3. How much or how many?
This is worked out using regression algorithms. These are common in the financial sector and predicting the weather.
4. What are the groups?
This is worked out using clustering algorithms. In this case you give the algorithm a bunch on unlabeled data and ask it to sort it into groups. This can be used to sort profile types. For example, if you want to understand the types of people who buy your product.