Speculative execution

Speculative executionis anoptimizationtechnique where acomputer systemperforms some task that may not be needed. Work is done before it is known whether it is actually needed, so as to prevent a delay that would have to be incurred by doing the work after it is known that it is needed. If it turns out the work was not needed after all, most changes made by the work are reverted and the results are ignored.

The objective is to provide moreconcurrencyif extraresourcesare available. This approach is employed in a variety of areas, includingbranch predictioninpipelined processors,value prediction for exploiting value locality, prefetchingmemoryandfiles,andoptimistic concurrency controlindatabase systems.^[1]^[2]^[3]

Speculative multithreadingis a special case of speculative execution.

Overview

Modernpipelined microprocessorsuse speculative execution to reduce the cost ofconditional branchinstructions using schemes that predict the execution path of a program based on the history of branch executions.^[2]In order to improve performance and utilization of computer resources, instructions can be scheduled at a time when it has not yet been determined that the instructions will need to be executed, ahead of abranch.^[4]

Variants

Speculative computationwas a related earlier concept.^[5]

Eager execution

Eager execution is a form of speculative execution where both sides of the conditional branch are executed; however, the results are committed only if the predicate is true. With unlimited resources, eager execution (also known asoracle execution) would in theory provide the same performance as perfectbranch prediction.With limited resources, eager execution should be employed carefully, since the number of resources needed growsexponentiallywith each level of branch executed eagerly.^[6]

Predictive execution

Predictive execution is a form of speculative execution where some outcome is predicted and execution proceeds along the predicted path until the actual result is known. If the prediction is true, the predicted execution is allowed to commit; however, if there is a misprediction, execution has to be unrolled and re-executed. Common forms of this includebranch predictorsandmemory dependence prediction.A generalized form is sometimes referred to as value prediction.^[7]

Runahead

Runaheadis a technique that allows acomputer processorto speculatively pre-processinstructionsduringcachemiss cycles. The pre-processed instructions are used to generate instruction anddata stream prefetchesby executing instructions leading tocache misses(typically calledlong latency loads) before they would normally occur, effectively hidingmemory latency.In runahead, the processor uses the idle execution resources to calculate instruction and data stream addresses using the available information that is independent of a cache miss. Once the processor has resolved the initial cache miss, all runahead results are discarded, and the processor resumes execution as normal. The primary use case of the technique is to mitigate the effects of thememory wall.The technique may also be used for other purposes, such as pre-computing branch outcomes to achieve highly accuratebranch prediction.^[8]

Related concepts

Lazy execution

Lazy execution is the opposite of eager execution, and does not involve speculation. The incorporation of speculative execution into implementations of theHaskell programming language,a lazy language, is a current research topic.Eager Haskell,a variant of the language, is designed around the idea of speculative execution. A 2003 PhD thesis madeGHCsupport a kind of speculative execution with an abortion mechanism to back out in case of a bad choice calledoptimistic execution.^[9]It was deemed too complicated.^[10]

Security vulnerabilities

Starting in 2017, a series ofsecurity vulnerabilitieswere found in the implementations of speculative execution on common processor architectures which effectively enabled an elevation ofprivileges.

These include:

References

^Lampson, Butler(2006)."Lazy and Speculative Execution in Computer Systems".In Momenzadeh, Mariam; Shvartsman, Alexander A. (eds.).Principles of Distributed Systems.10th International Conference on Principles of Distributed Systems. Lecture Notes in Computer Science. Vol. 4305. Bordeaux, France: Springer. pp. 1–2.doi:10.1007/11945529_1.ISBN 978-3-540-49991-6.
^^a ^bRaghavan, Prabhakar; Shachnai, Hadas; Yaniv, Mira (1998)."Dynamic schemes for speculative execution of code".Proceedings of the Sixth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.IEEE. pp. 309–314.doi:10.1109/MASCOT.1998.693711.Retrieved18 January2011.
^ Kung, H. T.;John T. Robinson (June 1981)."On optimistic methods for concurrency control"(PDF).ACM Trans. Database Syst.Vol. 6.Archived(PDF)from the original on August 31, 2019.
^Bernd Krieg-Brückner (1992).ESOP '92: 4th European Symposium on Programming, Rennes, France, February 26-28, 1992: proceedings.Springer. pp. 56–57.ISBN 978-3-540-55253-6.Retrieved18 January2011.
^Randy B. Osborne (1990-03-21)."Speculative Computation in Multilisp".Parallel Lisp: Languages and Systems(PS). Lecture Notes in Computer Science. Vol. 441.Digital Equipment Corporation Research Lab.pp. 103–137.doi:10.1007/BFb0024152.ISBN 3-540-52782-6.Archived fromthe originalon 2017-02-07.Retrieved2018-01-26.
^Jurij Šilc; Borut Robič; Theo Ungerer (1999).Processor architecture: from dataflow to superscalar and beyond.Springer. pp.148–150.ISBN 978-3-540-64798-0.Retrieved21 January2011.
^Mark D., Hill;Norman P., Jouppi;Gourindar S., Sohi (2000).Readings in Computer Architecture.Morgan Kaufman.ISBN 9781558605398.Retrieved5 January2018.
^Pruett, Stephen; Patt, Yale (October 2021)."Branch Runahead: An Alternative to Branch Prediction for Impossible to Predict Branches".MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture.MICRO '21. New York, NY, USA: Association for Computing Machinery. pp. 804–815.doi:10.1145/3466752.3480053.ISBN 978-1-4503-8557-2.S2CID 239011545.
^Jones, Simon Peyton; Ennals, Robert (1 August 2003)."Optimistic Evaluation: a fast evaluation strategy for non-strict programs".Retrieved15 May2019– via www.microsoft.com.{{cite journal}}:Cite journal requires|journal=(help)
^"[Haskell] Optimistic Evaluation?".31 August 2006.

[1] Lampson, Butler(2006)."Lazy and Speculative Execution in Computer Systems".In Momenzadeh, Mariam; Shvartsman, Alexander A. (eds.).Principles of Distributed Systems.10th International Conference on Principles of Distributed Systems. Lecture Notes in Computer Science. Vol. 4305. Bordeaux, France: Springer. pp. 1–2.doi:10.1007/11945529_1.ISBN 978-3-540-49991-6.

[DivisionRaghavan1998-2] Raghavan, Prabhakar; Shachnai, Hadas; Yaniv, Mira (1998)."Dynamic schemes for speculative execution of code".Proceedings of the Sixth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.IEEE. pp. 309–314.doi:10.1109/MASCOT.1998.693711.Retrieved18 January2011.

[3] Kung, H. T.;John T. Robinson (June 1981)."On optimistic methods for concurrency control"(PDF).ACM Trans. Database Syst.Vol. 6.Archived(PDF)from the original on August 31, 2019.

[Krieg-Brückner1992-4] Bernd Krieg-Brückner (1992).ESOP '92: 4th European Symposium on Programming, Rennes, France, February 26-28, 1992: proceedings.Springer. pp. 56–57.ISBN 978-3-540-55253-6.Retrieved18 January2011.

[5] Randy B. Osborne (1990-03-21)."Speculative Computation in Multilisp".Parallel Lisp: Languages and Systems(PS). Lecture Notes in Computer Science. Vol. 441.Digital Equipment Corporation Research Lab.pp. 103–137.doi:10.1007/BFb0024152.ISBN 3-540-52782-6.Archived fromthe originalon 2017-02-07.Retrieved2018-01-26.

[ŠilcRobič1999-6] Jurij Šilc; Borut Robič; Theo Ungerer (1999).Processor architecture: from dataflow to superscalar and beyond.Springer. pp.148–150.ISBN 978-3-540-64798-0.Retrieved21 January2011.

[7] Mark D., Hill;Norman P., Jouppi;Gourindar S., Sohi (2000).Readings in Computer Architecture.Morgan Kaufman.ISBN 9781558605398.Retrieved5 January2018.

[8] Pruett, Stephen; Patt, Yale (October 2021)."Branch Runahead: An Alternative to Branch Prediction for Impossible to Predict Branches".MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture.MICRO '21. New York, NY, USA: Association for Computing Machinery. pp. 804–815.doi:10.1145/3466752.3480053.ISBN 978-1-4503-8557-2.S2CID 239011545.

[Robert_Ennals_and_Simon_Peyton_Jones-9] Jones, Simon Peyton; Ennals, Robert (1 August 2003)."Optimistic Evaluation: a fast evaluation strategy for non-strict programs".Retrieved15 May2019– via www.microsoft.com.{{cite journal}}:Cite journal requires|journal=(help)

[10] "[Haskell] Optimistic Evaluation?".31 August 2006.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]