Microsoft Research Team Proposes LLM Accelerator LLMAA group of researchers at Microsoft proposes the LLM Accelerator LLMA. It is reported that. This inference decoding technique with references can speed up LLM inference in many real-world settings by exploiting the overlap between the output of the LLM and the references. LLMA works by selecting a span of text from the reference, copying its tokens into the LLM decoder, and then doing efficient parallel inspection based on the output token probabilities.
