ObservationValidator scope LIFO validation produces false positives for parallel observations (original) (raw)

Description

After merging #7253, @jonatan-ivanov identified that the LIFO scope ordering validation produces false positives when observations run in parallel on different threads.

The current implementation tracks scopes in a single global Deque<Context>, so sibling scopes on separate threads are incorrectly flagged as out-of-order.

Example

// Thread-1: scope1.open → (wait) → scope1.close // Thread-2: (wait) → scope2.open → (wait) → scope2.close

Global order: scope1.open → scope2.open → scope1.close → scope2.close

This is valid because scope1 and scope2 are independent siblings on different threads (e.g., two separate HTTP requests), but the validator reports an invalid closing order.

See: #7253 (comment)

Proposed Fix

Replace the global Deque<Context> with a per-thread stack using ThreadLocal<Deque<Context>> so that each thread tracks its own scope ordering independently.

Reproducer

Click to reveal test

@Test void siblings() throws InterruptedException { TestObservationRegistry registry = TestObservationRegistry.builder() .validateScopesClosedInReverseOrderOfOpening(true) .build(); registry.observationConfig().observationHandler(new ObservationTextPublisher());

CountDownLatch scopeOneOpenLatch = new CountDownLatch(1);
CountDownLatch scopeTwoOpenLatch = new CountDownLatch(1);
CountDownLatch scopeOneCloseLatch = new CountDownLatch(1);

AtomicReference<Exception> error = new AtomicReference<>();
Thread thread1 = new Thread(() -> {
    Observation observation1 = Observation.start("one", registry);
    try {
        Scope scope1 = observation1.openScope();
        scopeOneOpenLatch.countDown();
        scopeTwoOpenLatch.await();
        scope1.close();
        scopeOneCloseLatch.countDown();
    }
    catch (Exception e) {
        error.set(e);
    }
    finally {
        scopeOneCloseLatch.countDown();
    }
});
Thread thread2 = new Thread(() -> {
    try {
        scopeOneOpenLatch.await();
        Observation observation2 = Observation.start("two", registry);
        Scope scope2 = observation2.openScope();
        scopeTwoOpenLatch.countDown();
        scopeOneCloseLatch.await();
        scope2.close();
        observation2.stop();
    }
    catch (Exception e) {
        // noop
    }
});

thread1.start();
thread2.start();
thread1.join();
thread2.join();

assertThat(error.get()).isNull();

}