Google adds TPU 8tx and TPU 8i Max to its chip lineup

Google says it is adding two chips to its eighth-generation TPU line: TPU 8tx for secure, confidential inference, and TPU 8i Max for long context windows. Amin Vahdat, the company's chief technologist for AI and infrastructure, wrote the post. Google says the additions mark four new TPU systems in a single year.

The first two arrived in April. At Google Cloud Next, Google introduced TPU 8t for large-scale training and TPU 8i for real-time reasoning. It says those chips offer up to 2.7 times better training performance and 80 percent better inference price performance than the previous generation. Those figures are for TPU 8t and TPU 8i, not for the chips named today.

TPU 8i Max is two TPU 8i chips in one package. Google says it doubles the compute, the on-chip SRAM and the high-bandwidth memory bandwidth of TPU 8i. It says the package fits twice the silicon in the same rack space, so a data center can put more compute in the same floor area and the same power draw. It does not give a token limit, a watt figure or a price.

TPU 8tx is the security configuration. Google says it is for secure, confidential inference, aimed at customers that want hardware-level protection for sensitive data. The post does not list a core count, a memory size or an availability date for TPU 8tx.

Google also points developers at software it has already opened. It says PyTorch code can run on TPUs through TorchTPU, which it open-sourced last month. It says a new TPU Sync layer cuts some cache-transfer delays and time to first token by more than 90 percent. It does not say when either new chip will ship.