AI Briefing#hadoop
The latest AI and developer news about #hadoop, with the original source and a short summary.
The latest AI and developer news about #hadoop, with the original source and a short summary.
This covers the operational and design challenges LY Corporation faced while integrating 1EB-scale HDFS clusters with different architectures.
Automated analysis procedures by integrating skills and context documents into the existing Hadoop environment without a new platform.
Hyundai AutoEver built an incident response automation agent using Amazon Bedrock and LangGraph.
By tuning YARN's RESERVED Resource and Trino settings, available memory was increased by about 40%.
SK Planet developed 'Router', a real-time data pipeline service combining Kafka and Spark Streaming, to solve the latency issues of existing batch processing.
SK Planet adopted the open-source Trino as its company-wide data analytics engine, resolving data ingestion delays and building a real-time analytics environment.
Developed by 3 people over 6 months, the system successfully migrated 4,000 Flows and 40PB of data.
Detailed introduction of self-developed Ruby-based monitoring code and alert management features to overcome the limitations of commercial solutions.
It is being developed and operated in-house with five components to resolve issues of external solution costs and functional limitations.
Kakao uses its self-developed khp-package instead of Apache Bigtop to automate compatibility verification and deployment of Hadoop components.
Based on the Kakao Winter Internship briefing, this summarizes the scope of work and hiring criteria for DB and Hadoop engineering roles.
Engineers from Kakao's Hadoop Engineering Team held a small talk for 42 Seoul cadets, sharing insights on data roles and required competencies.
Six technical sessions covering real-time classification, Airflow 2.0, PostgreSQL, and more will be presented on Day2.
Kakao will present seven technical sessions at the if(kakao)2021 conference, covering cloud-native transformation, 9rum portal integration, Hadoop migration, and the Kubernetes Live Upgrade API.