Inception Pattern – Dream within a dream …

QuantumLab is meant less as an high performance machinery, as it is a place for experimentation. As such we want to be able to swap in and out terms and implementations easily and effortlessly while still keeping the code readable. This is where the inception pattern, as I call it, comes in. Especial thanks to @Betawolf for the ideas and guidance in figuring this one out.

A problem scenario:

Suppose we want to investigate the differences between Hartree-Fock results obtained within the RI approximation for the integrals and the approximation-free case. Alternatively, consider having two implementations for the integrals – one that is pure julia and one that calls on an external library. This means that we need to have two top-level functions: `evaluateHartreeFockwithRI` and `evaluateHartreeFockwithoutRI` (resp. `evaluateHartreeFockJulia` and `evaluateHartreeFockExtLib`). But now we can already see what this leads into. In the future, we might want to add convergence acceleration: `evaluateHartreeFockDIIS` and the like. But then a million combinations (`evaluateHartreeFockExtLibDIISSomeOtherStuffAndEvenMore`, etc.) quickly creep up.

Even if we don’t make a new function out of each and every combination, we still need to have them as options to the `evaluateHartreeFock` function. This still quickly destroys decoupling. The `evaluateHartreeFock` function has to be aware, that the `computeMatrixCoulomb` function which it calls upon, internally calls on another function to compute integrals. The `evaluateHartreeFock` function then has one boolean parameter, like `employRIinCoulomb`, for each option in any of its children which it then has to pass down to the only function for which it makes sense, the very bottom function in the descendence tree.

To summarize: The problem lies within the fact that the top level function only knows about its direct children, which in turn only know about their children. However we want to be able to choose the bottom implementation at the very top.

Pattern desciption:

  1. The complete call stack for the default implementation is written.
    function A(x)
      ...
      b = B(y)
      ...
    end
    
    function B(y)
      ...
      c = C(z)
      ...
    end
    
    function C(z)
      ...[compute return value by default implementation]...
      return c
    end
  2. Additional implementations are added.
    function C2(z)
      ...[compute return value by second implementation]...
      return c
    end
  3. Following the call stack upwards, all functions are turned into parameters in their parents declaration.
    function B(y,C=C)
      ...
      c = C(z)
      ...
    end
    
    function A(x,B=B)
      ...
      b = B(y)
      ...
    end
  4. To define a new “inception” at the top level, the specific call tree to be substituted is iteratively inserted.
    # To call A such, that C2 is used instead of C on the bottom level:
    
    local Btmp(y) = B(y,C2)
    local Atmp(x) = A(x,Btmp)
    
    Atmp(x)
    
    # of course this can also be inlined:
    
    A(x,y->B(y,C2))

Please note, that this pattern is extensible to arbitrary deep call stacks, can contain functions with and without arguments and can be added ex post to a given function call stack. It does however come at the price of disallowing inlining optimizations. Beware of this fact and avoid using the pattern on very small functions on the bottom of the call stack, where these optimizations can make a performance difference.