What it means
Latency is waiting time. On a website, it is the time between clicking and seeing something happen. In an AI feature, it is the time between asking and receiving the first words of the answer, or the full answer.
Delays add up across a chain: network, server, database, AI model and any tools the model calls.
Why it matters for a business
People notice delays quickly. Slow pages lose visitors, and slow assistants feel broken, especially in voice and live chat. Measuring and reducing latency is often one of the cheapest ways to improve conversion and satisfaction.
A business example
Things to watch
-
Measure real latency from users' locations, not only your office.
-
Stream AI answers so people see progress.
-
Use faster models where people are waiting.
-
Cache results that do not change often.