Help the compiler vectorize adjacent_difference by AlexGuteniev · Pull Request #4958 · microsoft/STL (original) (raw)

📜 The approach

The following things prevented the original algorithm from vectorization:

• Loop-carried dependency, the previous input is used as one of operands.
  • This seems expected that the compiler doesn't transform such code to eliminate this automagically, too much of a transformation.
  • This was addressed by transforming the code to read the input array twice per iteration instead of carrying the values through the loop.
• Odd iterator pattern where the compiler cannot understand the iteration.
  • This seemed to me a strange limitation, so it was reported as DevCom-10742868.
  • This was addressed by using integer index.

🛑 Correctness concern

The standard defines exact steps for this algorithm. The optimization alters the steps.
In particular the standard wants the subtracted value to be saved from the previous iteration, rather than being read again.
The two below sections explain what precautions are made to make the change unobservable, so I hope the change is correct.

✅ Checks for eligibility

The following checks were added:

• No Aliasing (see below)
• Iterators can be pointers
• Source iterator is not volatile (read order is altered)
• Trivially copyable (we skip copying where the standard asks for it)

There's no need in check for integral types or so, since the compiler makes the final decision anyway, and it may be able optimize even something that wouldn't pass a strict check.

⚠️ No Aliasing

Apparently there's no rule that the source and the destination ranges may not overlap.
We should handle aliasing.

Unlike the #4431 precedent, we can't yield to the compiler here. The compiler is able to insert overlaps check that prevents vectorization and go to the scalar fallback in case of checks failure, but:

• We apply transformation that would change the meaning of the program in case of overlapping range, and the meaning would be changed no matter if vectorization happens
• The checks that compiler inserts may be too loose, it may allow like equal source and destination pointer, as these are thc checks if the transformed algorithm would not change the meaning

So we do our own checks.

Then we tell the compiler with __restrict that we already checked, and it should not bother. This is done in a separate function, because the __restrict is not aliased within scope, so saying __restrict within the original algorithm would apparently be a lie.

The extra check by the compiler, if not prevented would slightly add run time and dead code size.

😾 Compiler warnings

We have a great feature called integral promotion. Smaller types are converted to integers, and there is a warning about converting them back. Local pragma suppresses them in benchmark, but not in the test.

@StephanTLavavej used a function object with static_cast to avoid warnings in the test.

⏱️ Benchmark results

🥇 Results interpretation

• Overall, we're good 😸
• 8-bit case failed to vectorize for no reason, reported DevCom-10745948
• Still 8-bit case is noticeably better. I didn't analyze that, but looks like this a consistent thing, not codegen gremlins. I think it is a side effect of eliminating loop-carried dependency, so the processor can parallelize and overlap iterations
• AVX2 is only slightly faster. I did not analyze, but think that memory wall is being hit here 🧱