Ongoing2026

Preference-tuned small medical LLMs

DPO and KTO pipelines for smaller models on curated clinical documents, traced end to end with Langfuse.

Authors

Why smaller, preference-tuned models

Domain-specific document and clinical workflows often benefit from smaller models that can be self-hosted (such as with vLLM) with predictable latency and controlled operating cost, while still matching strict domain behavior.

This ongoing line of work focuses on preference-tuning smaller LLMs over curated clinical documents rather than relying solely on prompt engineering over large general-purpose APIs.

DPO and KTO pipelines

We are building and evaluating both Direct Preference Optimization (DPO) and Kahneman-Tversky Optimization (KTO) training pipelines on curated clinical documents.

Running both formulations inside one reproducible pipeline lets us compare pairwise preference datasets against binary feedback signals on the same frozen evaluation sets.

End-to-end observability with Langfuse

Building on prior production experience shipping a medical RAG platform with LLMOps tracing, every stage of the pipeline is traced end to end in Langfuse.

That matches how we run all model work at Untern: start from real data, freeze a test set before training, and gate every iteration against measurable outputs.

  • Curated clinical document datasets paired with frozen evaluation gates.
  • DPO and KTO alignment pipelines targeting smaller, deployable model sizes.
  • Full end-to-end run and evaluation tracing in Langfuse.

Have a problem AI should solve?

Tell us what the data looks like and what “working” would mean for you. One of the founders will reply.