Vector (original) (raw)

Feedback in V8

Feedback in V8

Michael Stanton

• Google V8: Compiler Team/Manager
• V8: ICs and Feedback Vectors
• Cats, climbing and old typewriters :p

Feedback

The transmission of evaluative or corrective information about an action, event, or process to the original or controlling source; also : the information so transmitted

What people think we do

What people think we do

What we actually do

How does v8 achieve Performance?

• Compilation pipeline with learning
• HiddenClass-based object layout
• Inline Caches to maintain and observe layout

Compilation pipeline with learning

Source

Code

Ignition

Byte

Code

Run for a while

Gather feedback

Compilation pipeline with learning

Source

Code

Ignition

Turbofan

Byte

Code

Optimized

Code

Compilation pipeline with learning

Source

Code

Ignition

Turbofan

Byte

Code

Optimized

Code

Deoptimization

How does v8 achieve Performance?

• Compilation pipeline with learning

• Hidden-Class-based object layout

• Inline Caches to maintain and observe layout

• Objects have a "hidden class" (called a Map in V8)

• The Map is the first pointer in every object

• The Map describes the layout in memory

• Adding/removing properties changes the Map

Hidden-Class-based Object layout

let o = { x: 1 };
o.y = 2;

Evolution of o.map

Hidden-Class-based Object layout

let o = { x: 1 };
o.y = 2;

Evolution of o.map

Hidden-Class-based Object layout

let o = { x: 1 };
o.y = 2;

Evolution of o.map

Umm...Okay...

How does v8 achieve Performance?

• Compilation pipeline with learning
• Hidden-Class-based object layout
• Inline Caches to maintain and observe layout

Inline Caches to maintain and observe layout

• An Inline Cache (IC) is a listening site placed in your code.
• We have them at LOAD, STORE and CALL locations.
• It caches the Map of objects that pass by in the Feedback Vector for the function.

Every function has a Feedback Vector. It's just an array that holds state for each IC.

function processLogFile(fileName) {
  this.collectEntries = true;
  this.lastLogFileName_ = fileName;
  var line;
  while (line = readline()) {
    this.processLogLine(line);
  }
  print();
  print("Load: " + this.LoadIC);
  print("Store: " + this.StoreIC);
  ...
}

Inline Caches to maintain and observe layout

An Inline Cache is also a state machine.

Inline Caches to maintain and observe layout

mrale.ph/blog

function load(a) { return a.key; }

// pseudo-code for the LOAD IC:
if (vector[slot].state == MONO) {
  if (a.map == vector[slot].map) {
    return valueAtOffset(a, vector[slot].offset);
  } else {
    // Call into the Runtime for instructions.
    // Update IC state.
  }
} else {
  ..
}

Some facts

• We can say an IC is monomorphic when it only saw objects of one Map.
• It never makes sense to talk about an object as monomorphic.
• The term only applies to load of a particular property at a particular site.

How v8 achieves Performance

• Compilation pipeline with learning
• Hidden-Class-based object layout
• ICs to maintain and observe layout

Feedback Workflow

Let's look at a simple property load.

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Parsing

Here is the AST from parsing:

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

// Run with --print-ast
FUNC
. NAME "load"
. PARAMS
. . VAR "a"
. RETURN
. . PROPERTY Slot(0) at 29
. . . VAR PROXY parameter[0] "a"
. . . NAME key

Parsing

Here is the AST from parsing:

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

// Run with --print-ast
FUNC
. NAME "load"
. PARAMS
. . VAR "a"
. RETURN
. . PROPERTY Slot(0) at 29
. . . VAR PROXY parameter[0] "a"
. . . NAME key

Here is the feedback vector specification:

Compiling bytecode

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Here is the feedback vector specification:

// Run with --print-bytecode
"load" -- Parameter count 2

   StackCheck
   Nop
   LdaNamedProperty a0, [0], [3]  // "key"
   Return

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Here is the feedback vector:

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

We execute the LoadIC for "a.key"...

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Return the answer & remember the Map.

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Now with a different object...

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Profiling

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

The Runtime Profiler asks:

• Is the function "hot?"
• Is there enough feedback?

// --trace-opt output
[marking <JSFunction load> for 
    optimized recompilation, 
  reason: small function, 
  ICs with typeinfo: 1/1 (100%), 
  generic ICs: 0/1 (0%)]

(Run with --nouse-osr to ensure we optimize load(), and not the whole script)

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

Monomorphism: The Golden Idol

• It's tempting to see every problem as "insufficient monomorphism."
• This becomes counter-productive
  • Advice to duplicate code :(
  • Language expressiveness suffers

Monomorphism: The Golden Idol

• It's a good thing, but there are many good things.
• Spend more time improving baseline performance
• A sports car isn't appropriate for everything

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it

The more references there are to "a" downwind of the feedback, the greater the benefit of knowing the object's class.

Downwind includes inlined functions.

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: use it...but what if some of the maps are stale and never used recently?

// TurboFan emits something like this:

if (a.map == 0x43501231) {         // Maps embedded in code.
  return loadAtOffset(a, 0xc);     // No Feedback Vector.
} else if (a.map == 0x99503211) {
  return loadAtOffset(a, 0x10);
} else {
  DEOPTIMIZE();                    // Oh noes!
}

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites?

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites? deoptimize
  • Be careful about "hoisting" map checks out of loops!

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites? deoptimize
  • Be careful about "hoisting" map checks out of loops!

You lose touch with the Feedback Vector, and have nowhere to put the new map after deoptimization.

Eventually V8 stops reoptimizing.

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites? deoptimize
  • Be careful about "hoisting" map checks out of loops!
• How much to inline?
  • Increases compilation time
  • Which calls to inline, and to what depth?

Optimized execution

// load(a) - Turbofanned
...
mov  ecx,[eax-0x1]  // ecx = a.map
mov  edx,0x37e0b4ad // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1        // if not same, deopt
mov  eax,[eax+0xb]  // return a.key
...

Feedback Vector not being used

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

load({ key: "usb", name: "francis" });

Deopt path is taken

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

load({ key: "usb", name: "francis" });

// load(a) - Turbofanned
...
mov  ecx,[eax-0x1]  // ecx = a.map
mov  edx,0x37e0b4ad // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1        // if not same, deopt
mov  eax,[eax+0xb]  // return a.key
...

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

load({ key: "usb", name: "francis" });

Conclusion

We are watching...

Conclusion

We are watching...

with love!

Questions?

Shameless plug