Post-Deepseek World

Deepseek has reset priors of the tech community at large, and opened a much larger application game. Here is a mix of business and technical articles that are pointing in the same overall direction - models (and implicitly agents) will go 'deep' in order to drive up accuracy and value. R1 (and 100 others soon) will make it much easier to do this. The machine is cranking, and we should have seriously powerful agents/models soon.

-Venkata

Subscribe to this Domain-specific Agents and Knowledge Newsletter

Previous newsletter on Knowledge Agents and Economics

1. DeepSeek R1 and the Resurgence of Domain-Specific Models

Author: Jon Turow

URL: https://substack.com/home/post/p-155800514

Explores the implications of the new AI model DeepSeek R1, highlighting how it could shift the landscape from reliance on massive compute power to domain-specific expertise as a key driver of AI innovation.

Well argued piece. Worth your time.

TL;DR

1. Domain-specific models will win over general-purpose ones
2. Innovative training will focus on on high-value examples
3. Wide applications in areas will deep domain expertise such as finance, healthcare, legal, and industrial sectors
4. Future AI development fueled by expert knowledge, not just compute

2. Clouded Judgement 1.3.25 - Domain Specific Models

Author: Jamin Ball

URL: https://cloudedjudgement.substack.com/p/clouded-judgement-1325-domain-specific

2025 is poised to spotlight domain-specific AI models, moving away from generalized systems. These specialized models, trained on unique datasets, promise to transform industries requiring deep expertise and contextual understanding.

I generally read all of Jamin Ball 's substack posts. It is worth subscribing to.

TL;DR

1. Precision in specialized fields like protein engineering and aerospace
2. Continuous improvement through real-time feedback mechanisms
3. Custom models for unique enterprise needs, enhancing operational efficiency
4. A future where every enterprise tailors AI solutions, driving market growth

3. HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs

Authors: Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, Benyou Wang

URL: https://huggingface.co/papers/2412.18925

The article discusses the introduction of HuatuoGPT-o1, a medical language model designed for complex reasoning in healthcare. By addressing the limitations of current methods, this model aims to enhance medical decision-making through a novel framework for verifiable medical reasoning.

Deepseek is not the only innovation. Ceative and good solutions are coming out of China in volume. This requires only a training set of 40K (read: small but good datasets go a long way)

TL;DR

1. Improves medical problem-solving accuracy
2. Using a verifier for model output correctness
3. Reinforcement learning boosts complex reasoning performance
4. Potential to influence AI applications in various specialized fields

4. SciCode: A Research Coding Benchmark Curated by Scientists

Author: Minyang Tian et al.

URL: https://arxiv.org/abs/2407.13168

This article discusses SciCode, a coding benchmark specifically designed for evaluating language models (LMs) in generating code to solve real scientific research problems. Developed with insights from scientific experts across various fields, the benchmark consists of 338 subproblems derived from 80 main challenges, aimed at assessing the capabilities of AI in practical scientific applications.

LLMs struggle to understand specialized codebases. This is saying, wait for 1 year. LLMs will write fluid dynamics and other codes.

TL;DR

1. Benchmarking LMs with real scientific problems
2. Insights from 16 diverse scientific disciplines
3. Potential use for testing AI in research scenarios
4. Future of AI as an assistant in scientific discovery

5. The Knowledge Revival

Author: Robert Pondiscio

URL: https://thenext30years.substack.com/p/curriculum-for-deep-thinking

This article argues for a knowledge-rich curriculum as essential for cognitive development in education, countering the notion that skills can be learned in isolation. It emphasizes that deep thinking is fundamentally linked to domain-specific knowledge rather than generic skills.

This is really about surviving and building careers in the AI era by developing deep thinking. What struck me is how thinking and domain-knowledge are interlinked.

1. Deep understanding over rote memorization
2. Instructions for curriculum design
3. Future demands for coherent, knowledge-focused education

6. KAG: A Better Alternative to RAG for Domain-Specific Knowledge Applications

Author: Sarayavalasaravikiran

URL: https://ai.plainenglish.io/kag-a-better-alternative-to-rag-for-domain-specific-knowledge-applications-046054bedde1

The article explores the limitations of Retrieval-Augmented Generation (RAG) in specialized fields and introduces Knowledge Augmented Generation (KAG) as a superior alternative that integrates structured knowledge systems.

No surprise but found the technical discussion interesting and useful guide if you are going to implement something of your own

TL;DR

1. Combines retrieval mechanisms with knowledge graphs
2. Enhanced logic and reasoning capabilities for complex queries
3. Applicable in domains like law and healthcare requiring precision and coherence
4. Potential to redefine knowledge-based AI systems and their applications