Step 2.7’s MTP seems broken (at least for ik_llama.cpp) where the draft model st... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

alfiedotwtf 35 days ago | parent | context | favorite | on: DSpark: Speculative decoding accelerates LLM infer...

Step 2.7’s MTP seems broken (at least for ik_llama.cpp) where the draft model starts and ends in block 3 but ik_llama bails out looking for block 0 :(

girvo 35 days ago [–]

Aw that’s a shame; I’m running the official llama.cpp on my Spark-alike, and it works great now. Proper triple head too which is what it is trained on, gets me up to 35-40tk/s decode

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact