The Data Regiment
3 min.
Logistics
March 17, 2024
Data professionals can be thought of as a regiment and are divided into four roles: Intelligence Officers, Data Scientists, Machine Learning Engineers, and Logisticians. Data Analysts, Data Scientists, Machine Learning Engineers, and Logisticians need domain expertise and the ability to communicate findings to non-technical audiences. The various data positions work together to make the most of the data, requiring harmony and collaboration to succeed.
To understand the various positions of data professionals, envision a military operation. Each team member has a distinct role, but they collaborate to achieve their mission - in this case, making the most of the available data. Some positions may overlap, but they all contribute to the team's success.
- Data Analytics: The Intelligence Officers. Data Analysts are the intelligence officers of the data operations team. They decipher valuable insights and patterns from data without using advanced techniques. Their strong domain expertise and ability to communicate findings to non-technical audiences make them the connecting link between the different data positions. Just like intelligence officers, data analysts must master essential technical skills such as SQL, Excel, and BI tools, and a scripting language like Python or R can be advantageous.
- Data Science: The Special Forces. Data Scientists are the special forces of the regiment, using advanced machine learning and statistical techniques to uncover insights and trends from data. They still require domain expertise and the ability to communicate their findings. Sometimes they create non-production models, while other times they analyse historical data. As the team's special forces, they need to have technical skills like SQL, Python/R, and various ML libraries. Their mathematical knowledge depends on the size and focus of the company.
- Machine Learning Engineering: The Combat Engineers. Machine Learning Engineers are the combat engineers, responsible for building and deploying machine learning systems in production. They may not need deep knowledge of the models' mathematics, but they must understand numerical optimisation and computation issues. These engineers may work end-to-end or implement a model designed by a Data Scientist. Their technical skills include SQL, NoSQL, ML libraries, and non-scripting programming languages like Python, Scala, or Java.
- Data Engineering: The Logisticians. Data Engineers are the logisticians of the team, ensuring data flows smoothly and building the infrastructure used by other data professionals. They don't need deep machine learning knowledge but must understand the complexities of data flows. Their technical skills vary from SQL, NoSQL, Big Data technologies, ETL knowledge, to software engineering, including Hadoop, Spark, Kafka, and RabbitMQ.
- Backend Engineering: The Communications Specialists. Backend Engineers, like communications specialists in the military, manage the systems responding to user requests. They write API layers, authentication, microservices, and more. The line between them and Data Engineers can blur, as they also manage data flow in microservice architectures. Backend Engineers work closely with Machine Learning Engineers, who need to expose deployed models to other services. Technical skills for Backend Engineers include SQL, NoSQL, system engineering, and software engineering.
- DevOps: The Base Commanders. DevOps professionals are the base commanders of the team, responsible for the base infrastructure like servers, databases, load balancing, and CI/CD. Their role might lean more towards engineering or sys admin, depending on the company. In smaller companies, this role may fall onto the rest of the engineering team. Technical skills for DevOps include system engineering and scripting languages like Python, Bash, and Terraform.
Conclusion: United in the Data Mission. Just like a military operation, the various data positions work together to make the most of the data. Each role has its unique contribution, and even with some overlap, they all have their part to play. Understanding the nuances of each position helps us appreciate the harmony and collaboration required to succeed in the data mission.
