Kakao Hadoop Engineering Team Hosts Small Talk on Data Roles at 42 Seoul

Data platform engineers from Kakao's Hadoop Engineering Team conducted a small talk on data roles for aspiring developers (cadets) at 42 Seoul. The event was organized to address low mentoring application rates, difficulties in the hiring process, and high psychological barriers to entry.

Roles of Data Engineers and Platform Engineers

The data flow within enterprises is structured such that data platform engineers build the underlying environment, data engineers extract, transform, and load data, and finally deliver it to analysts or planners.

• Data Engineer: Provides data tailored to the needs of analysts and planners, requiring proficiency in storage, messaging queues, and processing technologies.
• Data Platform Engineer: Builds and operates platforms that facilitate pipeline construction, requiring system engineering knowledge such as OS and networking.

Competencies for Junior Developers

While immediate mastery of all competencies is not expected from junior developers, the following three areas were recommended for rapid growth.

1. Understanding Java: Since most data engineering tools are Java-based, the ability to read and write code is necessary.
2. Understanding Hadoop: As a core component of data pipelines, experience in building distributed clusters and understanding internal operations is recommended.
3. Linux Project Experience: It is important to build a distributed processing environment from data collection to loading to gain problem-solving experience.

Degrees and Job Fulfillment

Kakao prioritizes applicants' knowledge and experience over academic credentials, with master's and doctoral degrees being optional. The speaker cited pride in contributing to the impact of Kakao's services on daily life as a key motivator. This small talk served as an opportunity to increase interest in and accessibility to information about data roles.