Sign inNOYU
Experiments

Reading your result

What arrives when an experiment closes, how carefully to read it, and what to do next.

Igor VeselukhaBy Igor3 min read

When your window closes, NOYU works out what happened and sends you a Result. It arrives in your chat and keeps its own page, so you can come back to it.

What is in it

  • Before and during. Your chosen metric averaged over the period before you started, next to the same metric during the experiment.
  • A chart showing both stretches.
  • A plain-language read of what the numbers did.
  • The confounders that applied, not a generic list. If you were ill for four days, or travelled, or your training block changed, the Result says so.
  • How many days you logged, out of the window.

Then it asks one question: are you adopting this, dropping it, or not sure? Your answer is kept and NOYU takes it into account later, so it will not keep suggesting something you have already tried and rejected.

Where "before" comes from

The baseline is your own history for that metric in the weeks before you started. NOYU does not ask you to reconstruct what you were doing beforehand.

The reason is precise. Backfilling the outcome is unnecessary, because your wearable already recorded it. Backfilling the behaviour would mean asking you to remember what time you ate dinner three weeks ago, and that data would be worthless. So NOYU uses the measured thing you have and does not invent the remembered thing you do not.

How much to believe it

Less than you would like, and NOYU says so plainly.

There was no control group. Nothing was blinded. You knew what you were testing, which changes behaviour on its own. And in a two-week window, a work crisis or a cold moves your numbers more than most interventions will.

This is why the Result names the confounders that applied instead of hiding them in a caveat. It is signal, not proof, and it will not tell you the change is proven.

What you get is a reasonable read on whether it is worth continuing. For a personal decision about your own routine, that is usually enough.

If you did not keep it up

Below 60% adherence, the Result does not offer a verdict.

It does not fail you either. It gives an honest read of what you did log and treats that as worth having, because it is. A fortnight of partial data still tells you something, and the most common reason for low adherence is that the change did not fit your life, which is itself the answer to a useful question.

Reading the numbers sensibly

A few things to hold in mind:

  • Small movements are usually noise. If your sleep score went from 74 to 76, that is a normal fortnight, not an effect.
  • Look at the direction and size together. A large, consistent shift is worth more than a small one, even over the same window.
  • Check the confounders first. If you were ill for a third of the window, the comparison is not clean and no amount of arithmetic fixes that.
  • Consider re-running it. The same experiment in a quieter month is a reasonable thing to do, and a repeated result is worth much more than a single one.

After the Result

Whatever you choose, the journal item you were ticking stays available. If you adopt the change, NOYU puts it back on your short list so you keep tracking it.

You can then start another experiment. You can only run one at a time, so the Result is the point at which the next one becomes available.

Related

Was this article useful?

Still need help?

Talk to us.

NOYU is a small team and we answer every email ourselves. Ask us anything, whether you're a member already or just finding out how NOYU works.

IgorJames

Igor and James

Email us