Wikipedia

AI accelerator

(Redirected fromNeural Processing Unit)

AnAI accelerator,deep learning processororneural processing unit(NPU) is a class of specializedhardware accelerator[1]or computer system[2][3]designed to accelerateartificial intelligenceandmachine learningapplications, includingartificial neural networksandcomputer vision.Typical applications include algorithms forrobotics,Internet of Things,and otherdata-intensive or sensor-driven tasks.[4]They are oftenmanycoredesigns and generally focus onlow-precisionarithmetic, noveldataflow architecturesorin-memory computingcapability. As of 2024,a typical AIintegrated circuitchipcontains tens of billionsofMOSFETs.[5]

AI accelerators are used in mobile devices such as AppleiPhonesandHuaweicellphones,[6]and personal computers such asIntellaptops,[7]AMDlaptops[8]andApple siliconMacs.[9]Accelerators are used incloud computingservers, includingtensor processing units(TPU) inGoogle Cloud Platform[10]and Trainium and Inferentia chips inAmazon Web Services.[11]A number of vendor-specific terms exist for devices in this category, and it is anemerging technologywithout adominant design.

Graphics processing unitsdesigned by companies such asNvidiaandAMDoften include AI-specific hardware, and are commonly used as AI accelerators, both fortrainingandinference.[12]

History

edit

Computer systems have frequently complemented theCPUwith special-purpose accelerators for specialized tasks, known ascoprocessors.Notableapplication-specifichardware unitsincludevideo cardsforgraphics,sound cards,graphics processing unitsanddigital signal processors.Asdeep learningandartificial intelligenceworkloads rose in prominence in the 2010s, specialized hardware units were developed or adapted from existing products toacceleratethese tasks.

Early attempts

edit

First attempts likeIntel's ETANN 80170NX incorporated analog circuits to compute neural functions.[13]

Later all-digital chips like the Nestor/IntelNi1000followed. As early as 1993,digital signal processorswere used as neural network accelerators to accelerateoptical character recognitionsoftware.[14]

By 1988, Wei Zhang et al. had discussed fast optical implementations of convolutional neural networks for Alpha bet recognition.[15][16]

In the 1990s, there were also attempts to create parallel high-throughput systems for workstations aimed at various applications, including neural network simulations.[17][18]

FPGA-based accelerators were also first explored in the 1990s for both inference and training.[19][20]

In 2014, Chen et al. proposed DianNao (Chinese for "electric brain" ),[21]to accelerate deep neural networks especially. DianNao provides 452 Gop/s peak performance (of key operations in deep neural networks) in a footprint of 3.02 mm2and 485 mW. Later, the successors (DaDianNao,[22]ShiDianNao,[23]PuDianNao[24]) were proposed by the same group, forming the DianNao Family[25]

Smartphonesbegan incorporating AI accelerators starting with theQualcomm Snapdragon 820in 2015.[26][27]

Heterogeneous computing

edit
Main article:Heterogeneous computing

Heterogeneous computing incorporates many specialized processors in a single system, or a single chip, each optimized for a specific type of task. Architectures such as theCell microprocessor[28]have features significantly overlapping with AI accelerators including: support for packed low precision arithmetic,dataflow architecture,and prioritizing throughput over latency. The Cell microprocessor has been applied to a number of tasks[29][30][31]including AI.[32][33][34]

In the 2000s,CPUsalso gained increasingly wideSIMDunits, driven by video and gaming workloads; as well as support for packed low-precisiondata types.[35]Due to the increasing performance of CPUs, they are also used for running AI workloads. CPUs are superior forDNNswith small or medium-scale parallelism, for sparse DNNs and in low-batch-size scenarios.

Use of GPUs

edit

Graphics processing unitsor GPUs are specialized hardware for the manipulation of images and calculation of local image properties. The mathematical basis of neural networks andimage manipulationare similar,embarrassingly paralleltasks involving matrices, leading GPUs to become increasingly used for machine learning tasks.[36][37]

In 2012, Alex Krizhevsky adopted two GPUs to train a deep learning network, i.e., AlexNet,[38]which won the champion of the ISLVRC-2012 competition. During the 2010s, GPU manufacturers such asNvidiaadded deep learning related features in both hardware (e.g., INT8 operators) and software (e.g., cuDNN Library).

Over the 2010s GPUs continued to evolve in a direction to facilitate deep learning, both for training and inference in devices such asself-driving cars.[39][40]GPU developers such as NvidiaNVLinkare developing additional connective capability for the kind of dataflow workloads AI benefits from. As GPUs have been increasingly applied to AI acceleration, GPU manufacturers have incorporatedneural network-specifichardware to further accelerate these tasks.[41][42]Tensor coresare intended to speed up the training of neural networks.[42]

GPUs continue to be used in large-scale AI applications. For example,Summit,a supercomputer from IBM forOak Ridge National Laboratory,[43]contains 27,648Nvidia TeslaV100 cards, which can be used to accelerate deep learning algorithms.

Use of FPGAs

edit

Deep learning frameworks are still evolving, making it hard to design custom hardware.Reconfigurabledevices such asfield-programmable gate arrays(FPGA) make it easier to evolve hardware, frameworks, and softwarealongside each other.[44][19][20][45]

Microsoft has used FPGA chips to accelerate inference for real-time deep learning services.[46]

Use of NPUs

edit

Neural Processing Units (NPU) are another more native approach. Since 2017, several CPUs and SoCs have on-die NPUs: for example,Intel Meteor Lake,Apple A11.

Emergence of dedicated AI accelerator ASICs

edit

While GPUs and FPGAs perform far better than CPUs for AI-related tasks, a factor of up to 10 in efficiency[47][48]may be gained with a more specific design, via anapplication-specific integrated circuit(ASIC).[49]These accelerators employ strategies such as optimizedmemory use[citation needed]and the use oflower precision arithmeticto accelerate calculation and increasethroughputof computation.[50][51]Some low-precisionfloating-point formatsused for AI acceleration arehalf-precisionand thebfloat16 floating-point format.[52][53]Cerebras Systemshas built a dedicated AI accelerator based on the largest processor in the industry, the second-generation Wafer Scale Engine (WSE-2), to support deep learning workloads.[54][55]

Ongoing research

edit

In-memory computing architectures

edit

In June 2017,IBMresearchers announced an architecture in contrast to theVon Neumann architecturebased onin-memory computingandphase-change memoryarrays applied to temporalcorrelationdetection, intending to generalize the approach toheterogeneous computingandmassively parallelsystems.[56]In October 2018, IBM researchers announced an architecture based on in-memory processing andmodeled on the human brain's synaptic networkto acceleratedeep neural networks.[57]The system is based on phase-change memory arrays.[58]

In-memory computing with analog resistive memories

edit

In 2019, researchers from Politecnico di Milano found a way to solve systems of linear equations in a few tens of nanoseconds via a single operation. Their algorithm is based onin-memory computingwith analog resistive memories which performs with high efficiencies of time and energy, via conductingmatrix–vector multiplicationin one step using Ohm's law and Kirchhoff's law. The researchers showed that a feedback circuit with cross-point resistive memories can solve algebraic problems such as systems of linear equations, matrix eigenvectors, and differential equations in just one step. Such an approach improves computational times drastically in comparison with digital algorithms.[59]

Atomically thin semiconductors

edit

In 2020, Marega et al. published experiments with a large-area active channel material for developing logic-in-memory devices and circuits based onfloating-gatefield-effect transistors(FGFETs).[60]Such atomically thinsemiconductorsare considered promising for energy-efficientmachine learningapplications, where the same basic device structure is used for both logic operations and data storage. The authors used two-dimensional materials such as semiconductingmolybdenum disulphideto precisely tune FGFETs as building blocks in which logic operations can be performed with the memory elements.[60]

Integrated photonic tensor core

edit

In 1988, Wei Zhang et al. discussed fast optical implementations ofconvolutional neural networksfor Alpha bet recognition.[15][16] In 2021, J. Feldmann et al. proposed an integratedphotonichardware acceleratorfor parallel convolutional processing.[61]The authors identify two key advantages of integrated photonics over its electronic counterparts: (1) massively parallel data transfer throughwavelengthdivisionmultiple xingin conjunction withfrequency combs,and (2) extremely high data modulation speeds.[61]Their system can execute trillions of multiply-accumulate operations per second, indicating the potential ofintegratedphotonicsin data-heavy AI applications.[61]Optical processors that can also perform backpropagation for artificial neural networks have been experimentally developed.[62]

Nomenclature

edit

As of 2016, the field is still in flux and vendors are pushing their own marketing term for what amounts to an "AI accelerator", in the hope that their designs andAPIswill become thedominant design.There is no consensus on the boundary between these devices, nor the exact form they will take; however several examples clearly aim to fill this new space, with a fair amount of overlap in capabilities.

In the past when consumergraphics acceleratorsemerged, the industry eventually adoptedNvidia's self-assigned term, "the GPU",[63] as the collective noun for "graphics accelerators", which had taken many forms beforesettling on an overallpipelineimplementing a model presented byDirect3D[clarification needed].

All models of IntelMeteor Lakeprocessors have aVersatile Processor Unit(VPU) built-in for acceleratinginferencefor computer vision and deep learning.[64]

Deep learning processors (DLPs)

edit

Inspired from the pioneer work of DianNao Family, many DLPs are proposed in both academia and industry with design optimized to leverage the features of deep neural networks for high efficiency. At ISCA 2016, three sessions (15%) of the accepted papers, focused on architecture designs about deep learning. Such efforts include Eyeriss (MIT),[65]EIE (Stanford),[66]Minerva (Harvard),[67]Stripes (University of Toronto) in academia,[68]TPU (Google),[69]and MLU (Cambricon) in industry.[70]We listed several representative works in Table 1.

Table 1. Typical DLPs
Year DLPs Institution Type Computation Memory Hierarchy Control Peak Performance
2014 DianNao[21] ICT, CAS digital vectorMACs scratchpad VLIW 452 Gops (16-bit)
DaDianNao[22] ICT, CAS digital vector MACs scratchpad VLIW 5.58 Tops (16-bit)
2015 ShiDianNao[23] ICT, CAS digital scalar MACs scratchpad VLIW 194 Gops (16-bit)
PuDianNao[24] ICT, CAS digital vector MACs scratchpad VLIW 1,056 Gops (16-bit)
2016 DnnWeaver Georgia Tech digital Vector MACs scratchpad - -
EIE[66] Stanford digital scalar MACs scratchpad - 102 Gops (16-bit)
Eyeriss[65] MIT digital scalar MACs scratchpad - 67.2 Gops (16-bit)
Prime[71] UCSB hybrid Process-in-Memory ReRAM - -
2017 TPU[69] Google digital scalar MACs scratchpad CISC 92 Tops (8-bit)
PipeLayer[72] U of Pittsburgh hybrid Process-in-Memory ReRAM -
FlexFlow ICT, CAS digital scalar MACs scratchpad - 420 Gops ()
DNPU[73] KAIST digital scalar MACS scratchpad - 300 Gops(16bit)

1200 Gops(4bit)

2018 MAERI Georgia Tech digital scalar MACs scratchpad -
PermDNN City University of New York digital vector MACs scratchpad - 614.4 Gops (16-bit)
UNPU[74] KAIST digital scalar MACs scratchpad - 345.6 Gops(16bit)

691.2 Gops(8b) 1382 Gops(4bit) 7372 Gops(1bit)

2019 FPSA Tsinghua hybrid Process-in-Memory ReRAM -
Cambricon-F ICT, CAS digital vector MACs scratchpad FISA 14.9 Tops (F1, 16-bit)

956 Tops (F100, 16-bit)

Digital DLPs

edit

The major components of DLPs architecture usually include a computation component, the on-chip memory hierarchy, and the control logic that manages the data communication and computing flows.

Regarding the computation component, as most operations in deep learning can be aggregated into vector operations, the most common ways for building computation components in digital DLPs are theMAC-based (multiplier-accumulation) organization, either with vector MACs[21][22][24]or scalar MACs.[69][23][65]Rather thanSIMDorSIMTin general processing devices, deep learning domain-specific parallelism is better explored on these MAC-based organizations. Regarding the memory hierarchy, as deep learning algorithms require high bandwidth to provide the computation component with sufficient data, DLPs usually employ a relatively larger size (tens of kilobytes or several megabytes) on-chip buffer but with dedicated on-chip data reuse strategy and data exchange strategy to alleviate the burden for memory bandwidth. For example, DianNao, 16 16-in vector MAC, requires 16 × 16 × 2 = 512 16-bit data, i.e., almost 1024 GB/s bandwidth requirements between computation components and buffers. With on-chip reuse, such bandwidth requirements are reduced drastically.[21]Instead of the widely used cache in general processing devices, DLPs always use scratchpad memory as it could provide higher data reuse opportunities by leveraging the relatively regular data access pattern in deep learning algorithms. Regarding the control logic, as the deep learning algorithms keep evolving at a dramatic speed, DLPs start to leverage dedicated ISA (instruction set architecture) to support the deep learning domain flexibly. At first, DianNao used a VLIW-style instruction set where each instruction could finish a layer in a DNN. Cambricon[75]introduces the first deep learning domain-specific ISA, which could support more than ten different deep learning algorithms. TPU also reveals five key instructions from the CISC-style ISA.

Hybrid DLPs

edit

Hybrid DLPs emerge for DNN inference and training acceleration because of their high efficiency. Processing-in-memory (PIM) architectures are one most important type of hybrid DLP. The key design concept of PIM is to bridge the gap between computing and memory, with the following manners: 1) Moving computation components into memory cells, controllers, or memory chips to alleviate the memory wall issue.[72][76][77]Such architectures significantly shorten data paths and leverage much higher internal bandwidth, hence resulting in attractive performance improvement. 2) Build high efficient DNN engines by adopting computational devices. In 2013, HP Lab demonstrated the astonishing capability of adopting ReRAM crossbar structure for computing.[78]Inspiring by this work, tremendous work are proposed to explore the new architecture and system design based on ReRAM,[71][79][80][72]phase change memory,[76][81][82]etc.

Benchmarks

edit

Benchmarks such as MLPerf and others may be used to evaluate the performance of AI accelerators.[83]Table 2 lists several typical benchmarks for AI accelerators.

Table 2. Benchmarks.
Year NN Benchmark Affiliations # of microbenchmarks # of component benchmarks # of application benchmarks
2012 BenchNN ICT, CAS N/A 12 N/A
2016 Fathom Harvard N/A 8 N/A
2017 BenchIP ICT, CAS 12 11 N/A
2017 DAWNBench Stanford 8 N/A N/A
2017 DeepBench Baidu 4 N/A N/A
2018 AI Benchmark ETH Zurich N/A 26 N/A
2018 MLPerf Harvard, Intel, and Google, etc. N/A 7 N/A
2019 AIBench ICT, CAS and Alibaba, etc. 12 16 2
2019 NNBench-X UCSB N/A 10 N/A

Potential applications

edit

See also

edit

References

edit
  1. ^"Intel unveils Movidius Compute Stick USB AI Accelerator".July 21, 2017. Archived fromthe originalon August 11, 2017.RetrievedAugust 11,2017.
  2. ^"Inspurs unveils GX4 AI Accelerator".June 21, 2017.
  3. ^Wiggers, Kyle (November 6, 2019) [2019],Neural Magic raises $15 million to boost AI inferencing speed on off-the-shelf processors,archived fromthe originalon March 6, 2020,retrievedMarch 14,2020
  4. ^"Google Designing AI Processors".May 18, 2016.Google using its own AI accelerators.
  5. ^Moss, Sebastian (March 23, 2022)."Nvidia reveals new Hopper H100 GPU, with 80 billion transistors".Data Center Dynamics.RetrievedJanuary 30,2024.
  6. ^"HUAWEI Reveals the Future of Mobile AI at IFA".
  7. ^"Intel's Lunar Lake Processors Arriving Q3 2024".Intel.
  8. ^"AMD XDNA Architecture".
  9. ^"Deploying Transformers on the Apple Neural Engine".Apple Machine Learning Research.RetrievedAugust 24,2023.
  10. ^Jouppi, Norman P.; et al. (June 24, 2017)."In-Datacenter Performance Analysis of a Tensor Processing Unit".ACM SIGARCH Computer Architecture News.45(2): 1–12.arXiv:1704.04760.doi:10.1145/3140659.3080246.
  11. ^"How silicon innovation became the 'secret sauce' behind AWS's success".Amazon Science.July 27, 2022.RetrievedJuly 19,2024.
  12. ^Patel, Dylan; Nishball, Daniel; Xie, Myron (November 9, 2023)."Nvidia's New China AI Chips Circumvent US Restrictions".SemiAnalysis.RetrievedFebruary 7,2024.
  13. ^Dvorak, J.C. (May 29, 1990)."Inside Track".PC Magazine.RetrievedDecember 26,2023.
  14. ^"convolutional neural network demo from 1993 featuring DSP32 accelerator".YouTube.June 2, 2014.
  15. ^abZhang, Wei (1988). "Shift-invariant pattern recognition neural network and its optical architecture".Proceedings of Annual Conference of the Japan Society of Applied Physics.
  16. ^abZhang, Wei (1990). "Parallel distributed processing model with local space-invariant interconnections and its optical architecture".Applied Optics.29(32): 4790–7.Bibcode:1990ApOpt..29.4790Z.doi:10.1364/AO.29.004790.PMID20577468.
  17. ^Asanović, K.; Beck, J.; Feldman, J.; Morgan, N.; Wawrzynek, J. (January 1994)."Designing a connectionist network supercomputer".International Journal of Neural Systems.4(4). ResearchGate: 317–26.doi:10.1142/S0129065793000250.PMID8049794.RetrievedDecember 26,2023.
  18. ^"The end of general purpose computers (not)".YouTube.April 17, 2015.
  19. ^abGschwind, M.; Salapura, V.; Maischberger, O. (February 1995)."Space Efficient Neural Net Implementation".RetrievedDecember 26,2023.
  20. ^abGschwind, M.; Salapura, V.; Maischberger, O. (1996). "A Generic Building Block for Hopfield Neural Networks with On-Chip Learning".1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.pp. 49–52.doi:10.1109/ISCAS.1996.598474.ISBN0-7803-3073-0.S2CID17630664.
  21. ^abcdChen, Tianshi; Du, Zidong; Sun, Ninghui; Wang, Jia; Wu, Chengyong; Chen, Yunji; Temam, Olivier (April 5, 2014)."DianNao".ACM SIGARCH Computer Architecture News.42(1): 269–284.doi:10.1145/2654822.2541967.ISSN0163-5964.
  22. ^abcChen, Yunji; Luo, Tao; Liu, Shaoli; Zhang, Shijin; He, Liqiang; Wang, Jia; Li, Ling; Chen, Tianshi; Xu, Zhiwei; Sun, Ninghui; Temam, Olivier (December 2014). "DaDianNao: A Machine-Learning Supercomputer".2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.IEEE. pp. 609–622.doi:10.1109/micro.2014.58.ISBN978-1-4799-6998-2.S2CID6838992.
  23. ^abcDu, Zidong; Fasthuber, Robert; Chen, Tianshi; Ienne, Paolo; Li, Ling; Luo, Tao; Feng, Xiaobing; Chen, Yunji; Temam, Olivier (January 4, 2016). "ShiDianNao".ACM SIGARCH Computer Architecture News.43(3S): 92–104.doi:10.1145/2872887.2750389.ISSN0163-5964.
  24. ^abcLiu, Daofu; Chen, Tianshi; Liu, Shaoli; Zhou, Jinhong; Zhou, Shengyuan; Teman, Olivier; Feng, Xiaobing; Zhou, Xuehai; Chen, Yunji (May 29, 2015). "PuDianNao".ACM SIGARCH Computer Architecture News.43(1): 369–381.doi:10.1145/2786763.2694358.ISSN0163-5964.
  25. ^Chen, Yunji; Chen, Tianshi; Xu, Zhiwei; Sun, Ninghui; Temam, Olivier (October 28, 2016). "DianNao family".Communications of the ACM.59(11): 105–112.doi:10.1145/2996864.ISSN0001-0782.S2CID207243998.
  26. ^"Qualcomm Helps Make Your Mobile Devices Smarter With New Snapdragon Machine Learning Software Development Kit".Qualcomm.
  27. ^Rubin, Ben Fox."Qualcomm's Zeroth platform could make your smartphone much smarter".CNET.RetrievedSeptember 28,2021.
  28. ^Gschwind, Michael; Hofstee, H. Peter; Flachs, Brian; Hopkins, Martin; Watanabe, Yukio; Yamazaki, Takeshi (2006). "Synergistic Processing in Cell's Multicore Architecture".IEEE Micro.26(2): 10–24.doi:10.1109/MM.2006.41.S2CID17834015.
  29. ^De Fabritiis, G. (2007). "Performance of Cell processor for biomolecular simulations".Computer Physics Communications.176(11–12): 660–664.arXiv:physics/0611201.Bibcode:2007CoPhC.176..660D.doi:10.1016/j.cpc.2007.02.107.S2CID13871063.
  30. ^Video Processing and Retrieval on Cell architecture.CiteSeerX10.1.1.138.5133.
  31. ^Benthin, Carsten; Wald, Ingo; Scherbaum, Michael; Friedrich, Heiko (2006).2006 IEEE Symposium on Interactive Ray Tracing.pp. 15–23.CiteSeerX10.1.1.67.8982.doi:10.1109/RT.2006.280210.ISBN978-1-4244-0693-7.S2CID1198101.
  32. ^"Development of an artificial neural network on a heterogeneous multicore architecture to predict a successful weight loss in obese individuals"(PDF).Archived fromthe original(PDF)on August 30, 2017.RetrievedNovember 14,2017.
  33. ^Kwon, Bomjun; Choi, Taiho; Chung, Heejin; Kim, Geonho (2008).2008 5th IEEE Consumer Communications and Networking Conference.pp. 1030–1034.doi:10.1109/ccnc08.2007.235.ISBN978-1-4244-1457-4.S2CID14429828.
  34. ^Duan, Rubing; Strey, Alfred (2008).Euro-Par 2008 – Parallel Processing.Lecture Notes in Computer Science. Vol. 5168. pp. 665–675.doi:10.1007/978-3-540-85451-7_71.ISBN978-3-540-85450-0.
  35. ^"Improving the performance of video with AVX".February 8, 2012.
  36. ^Chellapilla, K.; Sidd Puri; Simard, P. (October 23, 2006)."High Performance Convolutional Neural Networks for Document Processing".10th International Workshop on Frontiers in Handwriting Recognition.RetrievedDecember 23,2023.
  37. ^Krizhevsky, A.; Sutskever, I.; Hinton, G.E. (May 24, 2017)."ImageNet Classification with Deep Convolutional Neural Networks".Communications of the ACM.60(6): 84–90.doi:10.1145/3065386.
  38. ^Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E (May 24, 2017)."ImageNet classification with deep convolutional neural networks".Communications of the ACM.60(6): 84–90.doi:10.1145/3065386.
  39. ^Roe, R. (May 17, 2023)."Nvidia in the Driver's Seat for Deep Learning".insideHPC.RetrievedDecember 23,2023.
  40. ^Bohn, D. (January 5, 2016)."Nvidia announces 'supercomputer' for self-driving cars at CES 2016".Vox Media.RetrievedDecember 23,2023.
  41. ^"A Survey on Optimized Implementation of Deep Learning Models on the NVIDIA Jetson Platform",2019
  42. ^abHarris, Mark (May 11, 2017)."CUDA 9 Features Revealed: Volta, Cooperative Groups and More".RetrievedAugust 12,2017.
  43. ^"Summit: Oak Ridge National Laboratory's 200 petaflop supercomputer".United States Department of Energy.2024.RetrievedJanuary 8,2024.
  44. ^Sefat, Md Syadus; Aslan, Semih; Kellington, Jeffrey W; Qasem, Apan (August 2019)."Accelerating HotSpots in Deep Neural Networks on a CAPI-Based FPGA".2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).pp. 248–256.doi:10.1109/HPCC/SmartCity/DSS.2019.00048.ISBN978-1-7281-2058-4.S2CID203656070.
  45. ^"FPGA Based Deep Learning Accelerators Take on ASICs".The Next Platform.August 23, 2016.RetrievedSeptember 7,2016.
  46. ^"Microsoft unveils Project Brainwave for real-time AI".Microsoft.August 22, 2017.
  47. ^"Google boosts machine learning with its Tensor Processing Unit".May 19, 2016.RetrievedSeptember 13,2016.
  48. ^"Chip could bring deep learning to mobile devices".sciencedaily.February 3, 2016.RetrievedSeptember 13,2016.
  49. ^"Google Cloud announces the 5th generation of its custom TPUs".August 29, 2023.
  50. ^"Deep Learning with Limited Numerical Precision"(PDF).
  51. ^Rastegari, Mohammad; Ordonez, Vicente; Redmon, Joseph; Farhadi, Ali (2016). "XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks".arXiv:1603.05279[cs.CV].
  52. ^Lucian Armasu (May 23, 2018)."Intel To Launch Spring Crest, Its First Neural Network Processor, In 2019".Tom's Hardware.RetrievedMay 23,2018.Intel said that the NNP-L1000 would also support bfloat16, a numerical format that's being adopted by all the ML industry players for neural networks. The company will also support bfloat16 in its FPGAs, Xeons, and other ML products. The Nervana NNP-L1000 is scheduled for release in 2019.
  53. ^Joshua V. Dillon; Ian Langmore; Dustin Tran; Eugene Brevdo; Srinivas Vasudevan; Dave Moore; Brian Patton; Alex Alemi; Matt Hoffman; Rif A. Saurous (November 28, 2017). TensorFlow Distributions (Report).arXiv:1711.10604.Bibcode:2017arXiv171110604D.Accessed May 23, 2018.All operations in TensorFlow Distributions are numerically stable across half, single, and double floating-point precisions (as TensorFlow dtypes: tf.bfloat16 (truncated floating point), tf.float16, tf.float32, tf.float64). Class constructors have a validate_args flag for numerical asserts
  54. ^Woodie, Alex (November 1, 2021)."Cerebras Hits the Accelerator for Deep Learning Workloads".Datanami.RetrievedAugust 3,2022.
  55. ^"Cerebras launches new AI supercomputing processor with 2.6 trillion transistors".VentureBeat.April 20, 2021.RetrievedAugust 3,2022.
  56. ^Abu Sebastian; Tomas Tuma; Nikolaos Papandreou; Manuel Le Gallo; Lukas Kull; Thomas Parnell; Evangelos Eleftheriou (2017)."Temporal correlation detection using computational phase-change memory".Nature Communications.8(1): 1115.arXiv:1706.00511.Bibcode:2017NatCo...8.1115S.doi:10.1038/s41467-017-01481-9.PMC5653661.PMID29062022.
  57. ^"A new brain-inspired architecture could improve how computers handle data and advance AI".American Institute of Physics.October 3, 2018.RetrievedOctober 5,2018.
  58. ^Carlos Ríos; Nathan Youngblood; Zengguang Cheng; Manuel Le Gallo; Wolfram H.P. Pernice; C. David Wright; Abu Sebastian; Harish Bhaskaran (2018)."In-memory computing on a photonic platform".Science Advances.5(2): eaau5759.arXiv:1801.06228.Bibcode:2019SciA....5.5759R.doi:10.1126/sciadv.aau5759.PMC6377270.PMID30793028.S2CID7637801.
  59. ^Zhong Sun; Giacomo Pedretti; Elia Ambrosi; Alessandro Bricalli; Wei Wang; Daniele Ielmini (2019)."Solving matrix equations in one step with cross-point resistive arrays".Proceedings of the National Academy of Sciences.116(10): 4123–4128.Bibcode:2019PNAS..116.4123S.doi:10.1073/pnas.1815682116.PMC6410822.PMID30782810.
  60. ^abMarega, Guilherme Migliato; Zhao, Yanfei; Avsar, Ahmet; Wang, Zhenyu; Tripati, Mukesh; Radenovic, Aleksandra; Kis, Anras (2020)."Logic-in-memory based on an atomically thin semiconductor".Nature.587(2): 72–77.Bibcode:2020Natur.587...72M.doi:10.1038/s41586-020-2861-0.PMC7116757.PMID33149289.
  61. ^abcFeldmann, J.; Youngblood, N.; Karpov, M.; et al. (2021). "Parallel convolutional processing using an integrated photonic tensor".Nature.589(2): 52–58.arXiv:2002.00281.doi:10.1038/s41586-020-03070-1.PMID33408373.S2CID211010976.
  62. ^"Photonic Chips Curb AI Training's Energy Appetite - IEEE Spectrum".
  63. ^"NVIDIA launches the World's First Graphics Processing Unit, the GeForce 256".Archived fromthe originalon February 27, 2016.
  64. ^"Intel to Bring a 'VPU' Processor Unit to 14th Gen Meteor Lake Chips".PCMAG.
  65. ^abcChen, Yu-Hsin; Emer, Joel;Sze, Vivienne(2017). "Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks".IEEE Micro:1.doi:10.1109/mm.2017.265085944.hdl:1721.1/102369.ISSN0272-1732.
  66. ^abHan, Song; Liu, Xingyu; Mao, Huizi; Pu, Jing; Pedram, Ardavan; Horowitz, Mark A.; Dally, William J. (February 3, 2016).EIE: Efficient Inference Engine on Compressed Deep Neural Network.OCLC1106232247.
  67. ^Reagen, Brandon; Whatmough, Paul; Adolf, Robert; Rama, Saketh; Lee, Hyunkwang; Lee, Sae Kyu; Hernandez-Lobato, Jose Miguel; Wei, Gu-Yeon; Brooks, David (June 2016). "Minerva: Enabling Low-Power, Highly-Accurate Deep Neural Network Accelerators".2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).Seoul: IEEE. pp. 267–278.doi:10.1109/ISCA.2016.32.ISBN978-1-4673-8947-1.
  68. ^Judd, Patrick; Albericio, Jorge; Moshovos, Andreas (January 1, 2017). "Stripes: Bit-Serial Deep Neural Network Computing".IEEE Computer Architecture Letters.16(1): 80–83.doi:10.1109/lca.2016.2597140.ISSN1556-6056.S2CID3784424.
  69. ^abcJouppi, N.; Young, C.; Patil, N.; Patterson, D. (June 24, 2017).In-Datacenter Performance Analysis of a Tensor Processing Unit.Association for Computing Machinery.pp. 1–12.doi:10.1145/3079856.3080246.ISBN9781450348928.S2CID4202768.
  70. ^"MLU 100 intelligence accelerator card"(in Japanese). Cambricon. 2024.RetrievedJanuary 8,2024.
  71. ^abChi, Ping; Li, Shuangchen; Xu, Cong; Zhang, Tao; Zhao, Jishen; Liu, Yongpan; Wang, Yu; Xie, Yuan (June 2016). "PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory".2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).IEEE. pp. 27–39.doi:10.1109/isca.2016.13.ISBN978-1-4673-8947-1.
  72. ^abcSong, Linghao; Qian, Xuehai;Li, Hai;Chen, Yiran (February 2017). "PipeLayer: A Pipelined ReRAM-Based Accelerator for Deep Learning".2017 IEEE International Symposium on High Performance Computer Architecture (HPCA).IEEE. pp. 541–552.doi:10.1109/hpca.2017.55.ISBN978-1-5090-4985-1.S2CID15281419.
  73. ^Shin, Dongjoo; Lee, Jinmook; Lee, Jinsu; Yoo, Hoi-Jun (2017)."14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks".2017 IEEE International Solid-State Circuits Conference (ISSCC).pp. 240–241.doi:10.1109/ISSCC.2017.7870350.ISBN978-1-5090-3758-2.S2CID206998709.RetrievedAugust 24,2023.
  74. ^Lee, Jinmook; Kim, Changhyeon; Kang, Sanghoon; Shin, Dongjoo; Kim, Sangyeob; Yoo, Hoi-Jun (2018)."UNPU: A 50.6TOPS/W unified deep neural network accelerator with 1b-to-16b fully-variable weight bit-precision".2018 IEEE International Solid - State Circuits Conference - (ISSCC).pp. 218–220.doi:10.1109/ISSCC.2018.8310262.ISBN978-1-5090-4940-0.S2CID3861747.RetrievedNovember 30,2023.
  75. ^Liu, Shaoli; Du, Zidong; Tao, Jinhua; Han, Dong; Luo, Tao; Xie, Yuan; Chen, Yunji; Chen, Tianshi (June 2016). "Cambricon: An Instruction Set Architecture for Neural Networks".2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).IEEE. pp. 393–405.doi:10.1109/isca.2016.42.ISBN978-1-4673-8947-1.
  76. ^abAmbrogio, Stefano; Narayanan, Pritish; Tsai, Hsinyu; Shelby, Robert M.; Boybat, Irem; di Nolfo, Carmelo; Sidler, Severin; Giordano, Massimo; Bodini, Martina; Farinha, Nathan C. P.; Killeen, Benjamin (June 2018). "Equivalent-accuracy accelerated neural-network training using analogue memory".Nature.558(7708): 60–67.Bibcode:2018Natur.558...60A.doi:10.1038/s41586-018-0180-5.ISSN0028-0836.PMID29875487.S2CID46956938.
  77. ^Chen, Wei-Hao; Lin, Wen-Jang; Lai, Li-Ya; Li, Shuangchen; Hsu, Chien-Hua; Lin, Huan-Ting; Lee, Heng-Yuan; Su, Jian-Wei; Xie, Yuan; Sheu, Shyh-Shyuan; Chang, Meng-Fan (December 2017). "A 16Mb dual-mode ReRAM macro with sub-14ns computing-in-memory and memory functions enabled by self-write termination scheme".2017 IEEE International Electron Devices Meeting (IEDM).IEEE. pp. 28.2.1–28.2.4.doi:10.1109/iedm.2017.8268468.ISBN978-1-5386-3559-9.S2CID19556846.
  78. ^Yang, J. Joshua; Strukov, Dmitri B.; Stewart, Duncan R. (January 2013)."Memristive devices for computing".Nature Nanotechnology.8(1): 13–24.Bibcode:2013NatNa...8...13Y.doi:10.1038/nnano.2012.240.ISSN1748-3395.PMID23269430.
  79. ^Shafiee, Ali; Nag, Anirban; Muralimanohar, Naveen; Balasubramonian, Rajeev; Strachan, John Paul; Hu, Miao; Williams, R. Stanley; Srikumar, Vivek (October 12, 2016). "ISAAC".ACM SIGARCH Computer Architecture News.44(3): 14–26.doi:10.1145/3007787.3001139.ISSN0163-5964.S2CID6329628.
  80. ^Ji, Yu Zhang, Youyang Xie, Xinfeng Li, Shuangchen Wang, Peiqi Hu, Xing Zhang, Youhui Xie, Yuan (January 27, 2019).FPSA: A Full System Stack Solution for Reconfigurable ReRAM-based NN Accelerator Architecture.OCLC1106329050.{{cite book}}:CS1 maint: multiple names: authors list (link)
  81. ^Nandakumar, S. R.; Boybat, Irem; Joshi, Vinay; Piveteau, Christophe; Le Gallo, Manuel; Rajendran, Bipin; Sebastian, Abu; Eleftheriou, Evangelos (November 2019). "Phase-Change Memory Models for Deep Learning Training and Inference".2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS).IEEE. pp. 727–730.doi:10.1109/icecs46596.2019.8964852.ISBN978-1-7281-0996-1.S2CID210930121.
  82. ^Joshi, Vinay; Le Gallo, Manuel; Haefeli, Simon; Boybat, Irem; Nandakumar, S. R.; Piveteau, Christophe; Dazzi, Martino; Rajendran, Bipin; Sebastian, Abu; Eleftheriou, Evangelos (May 18, 2020)."Accurate deep neural network inference using computational phase-change memory".Nature Communications.11(1): 2473.arXiv:1906.03138.Bibcode:2020NatCo..11.2473J.doi:10.1038/s41467-020-16108-9.ISSN2041-1723.PMC7235046.PMID32424184.
  83. ^"Nvidia claims 'record performance' for Hopper MLPerf debut".
  84. ^"Development of a machine vision system for weed control using precision chemical application"(PDF).University of Florida.CiteSeerX10.1.1.7.342.Archived fromthe original(PDF)on June 23, 2010.
  85. ^"Self-Driving Cars Technology & Solutions from NVIDIA Automotive".NVIDIA.
  86. ^"movidius powers worlds most intelligent drone".March 16, 2016.
  87. ^"Qualcomm Research brings server class machine learning to everyday devices–making them smarter [VIDEO]".October 2015.
edit
Retrieved from "https://en.wikipedia.org/w/index.php?title=AI_accelerator&oldid=1259252659"