Breakthrough reduces AI's cost and carbon footprint

While the environmental impact of AI has been an ongoing concern, organisations like Trellis Data have been looking for solutions – recently finding one in large language model (LLM) decoders that are over three times faster. Trellis Data has created the world’s fastest decoder, named the Dynamic Depth Decoding (D3) technique. Using speculative decoding, it achieves an average speed increase of 44 per cent compared with the previous fastest decoder. Used on the architecture behind AI systems – LLMs – the D3 technique allows it to generate text approximately 3.16 times faster than standard decoding without limiting accuracy.