AI Briefing
๊ณต์ง€์‚ฌํ•ญ ๋ชฉ๋ก

๐Ÿ”œ ์ถ”๊ฐ€ ์˜ˆ์ • ์†Œ์Šค (Upcoming) โ€” Bucket C

ํ˜„์žฌ ํ™œ์„ฑํ™”๊ฐ€ blocked๋œ ์†Œ์Šค๋“ค โ€” ํ‘œ์ค€ HTML ์Šคํฌ๋ž˜ํผ(firecrawl + sitemap)๋กœ ์ˆ˜์ง‘์ด ์–ด๋ ค์›Œ ๋‹ค์Œ ๋‹จ๊ณ„ ๊ธฐ๋ฒ•์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.


โ›” Bucket C โ€” blocked (3)

๋ชจ๋‘ SPA / JS pagination / ๋น„ํ‘œ์ค€ ์‚ฌ์ดํŠธ ๊ตฌ์กฐ. sitemap.xml๋„ ์‹ค์ œ ๊ธฐ์‚ฌ URL ์—†์Œ.

| ์†Œ์Šค | URL | ์›์ธ | |------|-----|------| | LG AI Research | lgresearch.ai/blog | JS ํŽ˜์ด์ง€๋„ค์ด์…˜, sitemap์— ๊ธฐ์‚ฌ 1๊ฐœ๋งŒ, RSS ์—†์Œ | | NAVER LABS | naverlabs.com/storyList | storyList SPA, sitemap 404, RSS ์—†์Œ | | SK DEVOCEAN | devocean.sk.com | ์นดํ…Œ๊ณ ๋ฆฌ landing๋งŒ, sitemap 6 URL ๋ชจ๋‘ nav, RSS ์—†์Œ |

โœ… ์ด๋ฒˆ ๋ผ์šด๋“œ ํ•ด๊ฒฐ

  • kakao-brain โ†’ kakao-tech์˜ ์ค‘๋ณต ๋„๋ฉ”์ธ์ž„์ด ํ™•์ธ๋˜์–ด deprecate. tech.kakao.com/posts/feed(472KB ์ „์ฒด ์•„์นด์ด๋ธŒ)๋กœ kakao-tech ํ”ผ๋“œ ํ™•์žฅ.
  • upstage โ†’ sitemap.xml ๋ชจ๋“œ๋กœ 110๊ฐœ ๊ธฐ์‚ฌ ํ™œ์„ฑํ™”
  • perplexity, naver-clova, kakaostyle โ†’ firecrawl + JS ๋ Œ๋”๋ง์œผ๋กœ ํ™œ์„ฑํ™”
  • anthropicร—3, mistral, samsung, xai โ†’ firecrawl + ์ •์  HTML๋กœ ํ™œ์„ฑํ™”

๋‹ค์Œ ๋‹จ๊ณ„ ์˜ต์…˜ (๋‚จ์€ 3๊ฐœ)

  1. API ์—”๋“œํฌ์ธํŠธ ํƒ์ƒ‰ โ€” DevTools Network ๋ถ„์„์œผ๋กœ XHR JSON endpoint ๋ฐœ๊ฒฌ
  2. playwright-python ๋ณ„๋„ ์ปฌ๋ ‰ํ„ฐ โ€” headless Chromium + BeautifulSoup
  3. ์™ธ๋ถ€ ํ”ผ๋“œ ์„œ๋น„์Šค โ€” RSSHub, Feed Creator ๋“ฑ ํ”„๋ก์‹œ

์ตœ๊ทผ ๊ณต์ง€