Running a trained AI model to produce a response from new input. Unlike training (teaching the model), inference happens every time a model is used and drives ongoing costs, speed, and usage limits.
Running inference on the larger model gave better answers but doubled our API costs.
