Methodology · how the outcome was measured

An effect of 12.65 standard deviations is not biology — it is the measurement scale

The test that popularized the foam sequence + Nd:YAG measured elimination by a 0 to 5 note given in photography by three blind evaluators. The scale does not separate 80% from 100% — and that is where an effect size of 12.65 standard deviations comes out.

View the video lesson

The result of this trial is cited by a number: up to 95% elimination at three years in the reticular veins. What almost never accompanies the citation is the definition of elimination used by the authors — and it changes quite a lot what the number authorizes to say in the query.

Who has been studied

There were 320 patients, all women, aged 19 to 72 years, with a mean of 46.3 years and Fitzpatrick II to IV phototypes, recruited from a single center in Madrid. Twenty-two of the 320 patients did not complete the study. The allocation was uneven by design: 550 legs for foam with laser and 90 for foam only, remaining 517 and 79 legs for analysis.

The ruler: a note given in photography

The treated areas were recorded by macrophotography before treatment and at three months, two years and three years, with frontal, dorsal and oblique photos of both legs; the most affected areas were photographed at a constant distance of 18 cm, with the position of vessels marked against fixed points of the skin. A computerized system produced objective elimination percentages, but only for the photographed area, serving as a guide. The final score of the entire leg was given by three blind investigators, and the average of the three went into the analysis.

The scale is ordinal: 0 point for no change or worsening, 1 point for 20% elimination, 2 for 40%, 3 for 60%, 4 for 80% and 5 for total elimination. The agreement between evaluators was high, with Cohen's Kappa of 0.86 and 0.81 in the comparison of the second and third evaluator with the first.

The six steps in which the outcome was recorded, from no change until total elimination. It's the appraiser's eye on the photograph, not the software.
The six steps in which the outcome was recorded, from no change until total elimination. It's the appraiser's eye on the photograph, not the software.

The roof of the scale

In the discussion, the authors declare the limitation with all the letters: the classifications do not offer clear discrimination between 4, which is equivalent to 80% elimination, and 5, which is equal to 100%. A note 5 only serves to indicate that the result is closer to 100% than 80%. And they explain why the averages are so close to five: the three evaluators attributed 5 to almost all the legs treated with foam and laser.

The columns show the distance between the two arms in the note given by photo at the age of three. It is the difference that sustains the whole essay — and it is born from a scale that stops at 5.
The columns show the distance between the two arms in the note given by photo at the age of three. It is the difference that sustains the whole essay — and it is born from a scale that stops at 5.

Effect size denounces ruler

Cohen's d values were 7.56, 4.72 and 4.87 at three months and 8.22, 11.03 and 12.65 at three years, for classes I, II and III. The article itself defines a large effect from 0.8. A d of 12.65 does not describe a plausible biological difference: it describes a large numerator over a tiny denominator. And the denominator is in the table — the standard deviations at three years were 0.15, 0.19 and 0.23 in the foam group and 0.62, 0.45 and 0.37 in the combined group.

The threshold bar almost disappears next to the measured effect. When a d reaches this size, what it denounces is the tiny standard deviation of the scale, not biology.
The threshold bar almost disappears next to the measured effect. When a d reaches this size, what it denounces is the tiny standard deviation of the scale, not biology.

The patient does not always agree with the photo

At the age of three, 86% of the patients in the combined group declared themselves satisfied or very satisfied — 39.7% satisfied and 46.4% very satisfied. At the same time, 8.1% were only reasonably satisfied, 4.0% dissatisfied and 1.7% very dissatisfied, with almost all of them in 5. The authors explicitly record the discrepancy between the assessment of evaluators and patient satisfaction.

The ring is the same cohort that received a score of 5 in almost all legs: 86% declared themselves satisfied or very satisfied. What's left of the circle is what the photograph didn't capture.
The ring is the same cohort that received a score of 5 in almost all legs: 86% declared themselves satisfied or very satisfied. What's left of the circle is what the photograph didn't capture.

What the study does not authorize to state

The primary outcome is a subjective note in photography, with a ceiling of scale declared by the authors; it is neither vessel count nor instrumental measurement of the entire leg. The population is exclusively female, of phototypes II to IV, in a single center, with allocation of 550 against 90 legs. Unsatisfied patients in the foam arm received a free treatment session combined at the end of the study, which may influence the responses to the satisfaction questionnaire. And the control arm used 0.3% polydocanol, up to 20 mL in both legs, in two sessions — a dose chosen to allow the entire leg to be treated safely: the authors themselves write that, in these dosages, two sessions are safe but not very effective, and that the isolated foam could achieve similar results with a much larger number of sessions. What the test supports is a commitment to a published parameter technique, evaluated by photo at age three in that population — never a guarantee of results.

Content addressed exclusively to health professionals. No individual result is guaranteed: the indication depends on clinical evaluation and imaging examination of each patient (CFM 2,336/2023).

Practical training

This technique is taught with a real patient.

Questions? Talk to Júlia