My Biggest Career Mistake, In Data Science

Adam Ross Nelson
Jun 03, 2020




An article about the biggest mistake I made in my data science career

Introduction

Itwasn’t long ago in my career that someone asked me a question at work. It was a question that made me realize I had made a huge mistake. This mistake damaged the reputation of the work that I had been doing for five-plus years.

In a broad sense, it was work that I had begun in my PhD program and had continued at two subsequent employers. What follows is a mostly true sketch of this interaction and an explanation of lessons learned.

What was the question?

"When do you think we’ll be able to start using machine learning, artificial intelligence, and predictive modeling?"


Copyright © 2020 Adam Ross Nelson — All Rights Reserved. A data scientist meets with the company CEO, first day on the job. The CEO asks for help in predicting which users will make a purchase. The data scientist gets right to work.

I was months into this role when my colleague asked me “When do you think we’ll be able to start using machine learning, artificial intelligence, and predictive modeling?”

I was stunned. We had been using machine learning, artificial intelligence, and predictive modeling for years. I couldn’t understand the question.

Setting aside my confusion, I gracefully pointed to a well-known project that we had implemented nearly two years prior.

I said, “Do you remember the project we worked on last fall? We had half the company in the conference room late into the night?”

My colleague said, “Of course!” This colleague added, “That was so fun, my team still talks about that as a fond memory. And I know we made a big difference.”

I said, “I was fond of that project too… as it turns out, we used predictive modeling in order to identify which clients we would invite to participate in that project. We’ve been using predictive modeling all along. We use it every day!”

The Problem

The problem was that my colleagues, perhaps most, had no idea that we were (and had been for a long while) successfully implementing machine learning, artificial intelligence, and predictive modeling which are core aspects of doing work in data science. True story, this problem was my fault.


Copyright © 2020 Adam Ross Nelson — All Rights Reserved. A semi-autobiographical tale. In this story, a CEO asks the new data scientist to predict which users will make a purchase. After executing that task with a logistic regression ensemble, the CEO asks: “Did you try machine learning?”

Lesson Learned

The data scientist in this cartoon, and I earlier in my data science career, underreported and mischaracterized the work. Logistic regression is a supervised machine learning technique. Using an ensemble (multiple implementations related models) is also a common technique used in predictive analytics.

I thought I was doing myself and others a favor by avoiding the contemporary, jargony, catch-all, nearly meaningless buzzwords such as “machine learning,” “artificial intelligence,” and “predictive modeling.”

In lieu of catch-all buzzwords I used terms and phrases that I knew to be more specific and precise. The data scientist in this cartoon made a similar mistake.

Unwittingly, by not explicitly describing the ensemble of logistic regressions as a supervised machine learning algorithm few realized the work was an application of data science.

How To Avoid This Mistake

From this experience, I learned, and am still learning, multiple important lessons. Here are a few of them.
  • Be clear and vocal with others about your efforts. Being clear and vocal with others about your efforts means taking time to communicate, in as many modes of communication as possible, exactly what you are doing and why. If you’re lucky a boss or mentor will help with this but don’t wait for permission. Do it! To avoid this mistake find and use as many communication modes as possible (e.g. in-person, one-to-one, small groups, Slack, email, status reports, etc.). Added tip: find the modes of communication you know will be heard. If not sure, ask for help in identifying your company’s most effective modes.
  • Words matter. Meet people where you find them. In my case, I failed to meet colleagues where they were because when they were expecting to hear “predictive analytics” I hit them with “supervised machine learning logistic regression ensemble technique.” I wrongly thought this would be a favor to myself and others by demystifying the work. I was wrong, I convoluted the work. To avoid this mistake, listen for clues about the terms and phrases your audience will recognize before you begin speaking about the work. Use the terms and phrases your audience will recognize. Introduce new terms and phrases with care and compassion.
  • History repeats itself. Looking back at the jobs I had before becoming a data scientist (usually as an educator or university administrator) I see that I had previously underreported or mischaracterized my work. This is the kind of mistake that professionals in any line of work can make — technical or otherwise. To avoid this mistake, learn from your past or you will be forced to relive it.

Conclusion


This article describes one of the biggest mistakes I made in my data science career. I realized I had made this mistake once a colleague asked me “When do you think we’ll be able to start using machine learning, artificial intelligence, and predictive modeling?”

We had been using machine learning, artificial intelligence, and predictive modeling for nearly at that company for a long time, by then.

Letting my colleagues think that data science remained an aspiration was a mistake. I had endeavored to be more transparent by avoiding jargony, catch-all, nearly meaningless buzzwords. But this strategy seems to have backfired.

After explaining the story of how this happened, this article also provides a list of lessons that can be learned from the experience.

“My Biggest Career Mistake, In Data Science”
– Adam Ross Nelson twitter social icon Tweet


Share this article:

0 Comments

Post a comment
Log In to Comment
divider graphic

Related Stories

11

Pandas Essentials For Data Science

Photo by Maarten van den Heuvel on UnsplashPython is a popular language in data science, and of course, the most popular language for machine learn...

Mahbubul Alam
By Mahbubul Alam
29

Predicting survivors of Titanic

F.G.O. Stuart (1843–1923) / Public domainRMS Titanic was a British passenger liner operated by the White Star Line that sank in the North Atlantic ...

Dorian Lazar
By Dorian Lazar
17

All About Missing Data Handling

Missing data is an everyday problem that a data professional need to deal with. Though there are many articles, blogs, videos already available, I ...

Baijayanta Roy
By Baijayanta Roy
arrow-up icon