Was it us or them? Answering “the live froze for me” with evidence

A viewer writes in after your live event: the video froze, and for a few minutes they couldn’t hear the host. Support forwards the ticket to engineering with one question: was it us, or was it them? And right behind it come two more: what exactly broke, and when?

I work on a live-events app built on LiveKit, and I’ve handled tickets like this for a long time. Most of the time I didn’t know where to start.

Why the question is hard

By the time the ticket arrives, the live is over. Nobody can go back and watch what that viewer saw.

Server logs tell you a lot about the media infrastructure. They usually don’t tell you what one viewer actually received, or how their phone or laptop coped with it.

Room-level dashboards have the same gap. They show that quality was fine overall, while the complaint is about one person.

So the investigation becomes a process of elimination. The video team checks the stream, the app team checks the client, someone looks at the host’s setup, and the answer often ends up as “probably your Wi-Fi”. Sometimes that’s right, and sometimes nobody actually knows.

What that uncertainty costs

One ticket can pull in several people, because every team first has to rule itself out.

Support then has two bad options: apologise for something that may not have been your fault, or blame the customer’s connection without evidence.

And some people never write in. They have a bad session and leave. One person’s terrible session can almost vanish inside a healthy room average; there is a real example below.

Compare the viewer with the room

What helped me was to stop looking at the viewer who complained on their own, and compare them with everyone else who was receiving the same media at the same moment.

If one viewer says the host’s voice broke up at 8:14, dozens of others were listening to the same host at 8:14. What happened to them narrows the problem down to roughly three cases:

The source or stream had a problem. Many viewers receiving the same host got worse at the same time.
The viewer’s connection had a problem. One viewer got worse while other people receiving the same streams stayed healthy.
The viewer’s device had a problem. The network delivered fine, but device-side readings showed the phone could no longer keep up.

Knowing whose side it was is only half the answer. The other half is the specific cause and the exact moment: the host’s stream breaking up at 8:14, a viewer’s connection losing a quarter of its video packets for a few minutes, a phone that overheated within two minutes of joining. Each of those has a different fix and needs a different reply to the customer, and “around 8pm, it froze” is not enough to act on.

Two real cases

Both come from production rooms and are anonymised. Nobody filed a ticket in either case. I found them by looking for the participants with the worst experience, and those are not always the people who complain.

The room that looked mildly unhealthy

Over 49 minutes, nineteen viewers watched three people on camera. Average video packet loss across the room was about 1%. That isn’t alarming, and from the aggregate alone you would assume a small problem shared by everyone.

It wasn’t shared. One viewer, I’ll call her Maya, had about 9% video packet loss and kept getting blocky or stalled video for roughly 14 of those 49 minutes. At the same moments, the other viewers watching the same cameras were mostly fine. Twelve of the other eighteen had no bad stretches at all.

Without Maya, the room average drops from about 1% to 0.17%. The average had spread one person’s bad session across the whole room.

The comparison also showed what exactly was wrong. She lost video from all three people on camera, not just one, so the trouble was on her way in. Her browser measured round trips of about 500 ms, against 50–200 ms for the others. And the bad stretches came and went through the whole session instead of clustering around one event. That points at her network: her Wi-Fi, her access network or her provider, not a shared stream or platform problem.

The host whose voice broke up

In another room, two people were on camera and 37 viewers were watching. About twenty minutes in, one host’s audio got worse for 36 of the 37 viewers at almost the same moment. The co-host stayed clear.

When almost everyone loses the same person at once, the viewers’ own connections can’t explain it.

The host’s own telemetry showed what happened and when. At 19 minutes 56 seconds, a few seconds before viewers noticed, the host’s phone reported that its upload had hit the connection’s limit. It stopped sending its HD video and kept only a small, low-quality layer. The HD video came back about a minute and a half later. The phone itself wasn’t under heavy load.

So the problem most likely started on the host’s upstream network. A shared server failure was unlikely, because the co-host stayed healthy. Both rooms would have produced the same ticket, “the live froze for me”, with completely different causes behind it.

The technical versions, with the signals and the edge cases I had to rule out, are in One viewer was losing 9% of her video while the room average looked fine and Stream, network or phone? Three real live-stream incidents, taken apart. The second one adds a third case: a phone that overheated and started throwing away a large part of the video it received.

Questions to ask your team this week

You don’t need Rewitness for any of these.

When somebody reports a bad session, can you tell reasonably quickly whether the problem was shared or isolated to that participant?
Do you collect enough telemetry from the viewers who did not complain to compare them with the person who did?
Can you line up what different viewers experienced from the same host at the same point in time?
When support tells somebody “it was your connection”, what evidence is that conclusion based on?

If some answers are “no” or “we’d have to check”, that’s normal. Most monitoring grows around servers and room-level metrics. Those matter, but they answer a different question.

What I’m building

That is why I started building Rewitness. A small SDK in your web or React Native app records the media, network and device signals needed to reconstruct what each participant experienced during a live session. Besides WebRTC stats, it records LiveKit room events and device readings, and optionally whether each video tile actually appeared on screen and any errors your app sent to Sentry or Datadog.

Rewitness lines those recordings up on one timeline and compares them using fixed, explicit rules, so every conclusion can be traced back to the readings that produced it.

When a complaint comes in, you enter who it was and roughly when. The answer tells you whose side the problem was on, what the specific cause was, and exactly when it started and ended. It also shows what that person lost and from whom, how many others lost the same thing, and the readings behind each conclusion. For Maya, that is a set of stretches where her video from all three cameras broke up while everyone else kept receiving it, with her packet loss and round-trip time against the room’s. For the host, it is the moment nearly every viewer lost them at once while the co-host stayed clear, with the host’s own readings from that minute beside it.

To dig deeper, you can step through the reconstructed session: who was on stage, what different viewers were receiving, who was struggling, and which signals changed at the same time.

It has limits. It doesn’t read your media server’s logs, so some server-side failures can only be inferred from how they affected participants. And it only sees apps that include the SDK.

I would rather state those limits up front. The goal is to give your team enough evidence to check the answer themselves.