Stanford and UC Berkeley introduce TokenGate for faster long-context prefill
Stanford and UC Berkeley researchers released TokenGate, a drop-in learned token pruning module for LLMs that cuts long-context prefill time-to-first-token by up to 11.3x.
Stanford and UC Berkeley researchers released TokenGate, a drop-in learned token pruning module for LLMs that cuts long-context prefill time-to-first-token by up to 11.3x.