AI Infra2023
Traces how Transformer LLMs route information from input to output, attributing each block's effect to individual attention heads and feed-forward neurons. Click any edge to see what a head promotes or suppresses in vocabulary space.
AI Infra2024
Open-weights 314B-parameter Mixture-of-Experts language model (8 experts, 2 active per token, 8,192-token context) released under Apache 2.0. Ships a raw JAX checkpoint plus reference inference code; needs heavy multi-GPU memory to load.
Generates outcome-specific, dialectical rationales with an LLM and derives continuous, calibrated risk scores for irregularly sampled medical time series—mitigating risk polarization. Reports +3.3% average AUPRC and 81% reduction in calibration error across three benchmarks; code released.
Performs native structural reasoning for proteins, small molecules and inorganic crystals by tokenizing coordinates, topologies and periodic connectivities into a unified structure-aware vocabulary. Treats structural tokens as addressable evidence to produce interpretable prediction traces and improves accuracy across biology, chemistry and materials benchmarks.