👨🏿‍🏭 📈 🤴🏾 Should I go to Data Science? 🔪 👩🏻‍🚀 🕤

The advice that I always give when someone asks me where to start to do data science. Better become a software engineer.

This is my personal and optional opinion. Counterarguments in the comments are welcome.

The profession of a specialist is now trending. I don’t argue, data science will be one of the most fashionable areas of activity of the 21st century, but this undeservedly devalues another useful and highly paid profession - software developer.

Often I receive messages from university graduates and just people who change their profession, asking how to enter the data science. Instead, I advise them to simply go into IT.

Having experience in both areas, I will try to convince you to make the right choice.

EDISON , -.

. , , , — .

, software engineering data science solve problems! ;-)

1. -

There are an order of magnitude more jobs in software engineering compared to data science.

Here are some screen shots after a search in Google for " expert on data » ( " data scientist ») and « software development » ( " software engineer Have »).

7,616 scientific vacancies versus 53,8893 in software development. This is the same in the USA, in other countries.

If you believe Glassdoor (a web service with which you can monitor the level of salaries and get reviews of employees about companies), data scientists earn more money. However, my unverified hypothesis is that data vacancies are mainly targeted at very highly qualified employees.

Although, who argues, if you are offered a million in OpenAI - agree without hesitation.

2. There is no consensus that this is generally a “data science”

Company management often has a vague idea of what the phrase “data science” means. It is also easily possible, given the specific limitations in the business, which consist in the fact that a business cannot always afford to strictly follow the structure of roles.

This means that the responsibilities of a “data expert” vary very, very different from company to company.

A conditional sketch that you may need depending on your business tasks. In the spectrum between solving business problems and research there are such professions as Software Engineer ⇒ Data Engineer ⇒ Machine Learning Engineer ⇒ Data Scientist

Although the ideal range of roles between a software engineer and a data specialist can be imagined, it is unlikely that in real life this will be embodied in the reference form. This is especially true for startups undergoing the scaling process when they are just building the infrastructure.

As a result, the hired candidates work on the current problems of the company, and do not fulfill the expert role for which, generally speaking, they were hired.

I have evidence from my colleagues (data scientists) that they write backend code as software developers. I know other “data researchers” who have succeeded in resolving the financial issues of their companies.

This can sharply contrast with your expectations, especially if they formed under the influence of Kaggle contests.

3. The loneliness of the long-distance runner according to the data

Most companies just don't need as many data experts as software engineers. Other companies have so far only hired their first (and for the time being - the only) data specialist.

For this reason, many data researchers work alone, even if they are sitting at the same table with the developers.

This can make it difficult to get feedback and a second opinion. Software engineers either do not understand predictive modeling at all, or are too busy with completely different problems.

On the contrary, one of the advantages of the software development team is the ability to always tell colleagues: “I believe that ABC should be implemented in XYZ. Your opinion? ".

So, be mentally prepared to discuss problems with yourself or ... with a rubber duck.

4. Data Science has a pronounced research character

Be prepared for unpleasant conversations with management on the topic that what you have been working on for 2 weeks will not be used.

Work on both solvable and unsolvable problems is one of the fundamental differences between software development and AI.

Given the errors and limitations inherent in software development - before you begin, you basically have an idea of what is feasible and what is not. You can’t say the same thing about Machine Learning. Whether the model will turn out to be effective is unknown until it is implemented.

5. Business is not yet ready for AI

Even in an era when almost every large company introduces artificial intelligence, most of them do not have the proper infrastructure to support it.

One of my friends, the head of the Data Science department in an intensively developing startup, recently shared his startup wisdom with a glass of coffee:

First you identify the problem, then build the infrastructure, and only then attract data experts. And all this happens very quickly. (Not verbatim)

Another friend of mine, a data science specialist in a well-known company, recently complained in a conversation that she was forced to train an artificial intelligence model on big data on her laptop, and not in the cloud.

If some specific problems have not been resolved at the new place where you came, or even the company is not ready to introduce data science into its processes, then get ready to create something valuable at the limit of its strength.

6. Software Engineering Gives General Skills

Becoming a junior software engineer is like getting an MBA in technology. You will learn a little bit of everything.

You will learn about databases, cloud technology, deployment, security, and writing clean code.

You will learn how to manage the software assembly by observing your scrum leader, senior developer or project manager. You will receive mentoring.

If you find yourself in a company with an established team of engineers, it is almost guaranteed that you will quickly improve your skills and gain masked overall experience.

7. Software development as an activity is much more flexible.

With a more holistic experience with technology, software development provides more options when you decide it's time to change something.

DevOps, security, interface, backend, distributed systems, business intelligence, data engineering, data science ...

I know many developers who have moved from software development to data science. If you are looking at job descriptions for data science, you will immediately notice that they require a lot of skills that are generated by software development.

If you can create E2E projects, you can also do more than just create a model for the Kaggle contest. You can take this model, make it, configure authorization and electronic payments, and then monetize. This is your own startup.

I will never say that data science is useless outside its scope. Making decisions based on big data is simply a killer in our time. But this will also lead to the fact that data science methods will be more and more in demand in other work, as the business is more and more focused on big data.

8. Machine learning will become a common tool for software developers.

As AI becomes more convenient and easier to use, software developers will begin to use it to solve their problems.

I can teach (in the afternoon) a developer to build sklearn classifiers. This does not mean that my padawan will create the next AlphaGo, but it gives him an alternative to hard-coded conditional logic based on user input.

Big data specialists have specialized knowledge, such as statistics, they have a special intuition about how models work. But DevOps and Security engineers also have their own specialized knowledge.

I would say that the point here is not how skills differ, but that some are used less often and others more often. An experienced engineer moves between specializations orders of magnitude faster, while a beginner learns from scratch.

Although I do not think that we will see a complete merger of data science with software development. Rather, data science will become another area of expertise in software development.

9. AI will not replace software engineers in the foreseeable future

No matter how stupid it sounds now, in 2014 I started developing software, out of fear that AI would make any other work obsolete.

And yet, since then, we have not really come close to the tech future. Technology adoption is slow, and we are farther from a strong AI than you can imagine.

Compared to other professions, machine learning is even further away from software development automation. Although we have startups that create cool products, such as code completion with AI support, the coding itself is not really a real job. The present work is to solve problems using technology.

So far, in the era of pre-singularity, the ability to write programs will remain a valuable and highly paid skill for a long time to come.

Conclusion

Firstly, this is all primarily my IMHO. Secondly, I am aware that I mixed data researchers, machine learning engineers and artificial intelligence developers into one heap - and I perfectly understand the difference between them. Nevertheless, it is worth considering these arguments thoughtfully, because this is your life and your career.

Do not take what I have said too seriously. I would prefer you to investigate this issue and make your own decision. This is part of the work of any data expert, in general :)

In the end, they pay us primarily for solving problems.

Translations on the Edison Blog:

Top 50 gaming franchises with over a billion revenue	Will Airbnb survive the coronavirus? [spoiler: yes]

Should I go to Data Science?