PL3506095T3 - Optymalizacje komunikacyjne dla rozproszonego uczenia maszynowego - Google Patents

Optymalizacje komunikacyjne dla rozproszonego uczenia maszynowego

Info

Publication number
PL3506095T3
PL3506095T3 PL18209320.3T PL18209320T PL3506095T3 PL 3506095 T3 PL3506095 T3 PL 3506095T3 PL 18209320 T PL18209320 T PL 18209320T PL 3506095 T3 PL3506095 T3 PL 3506095T3
Authority
PL
Poland
Prior art keywords
machine learning
distributed machine
communication optimizations
optimizations
communication
Prior art date
Application number
PL18209320.3T
Other languages
English (en)
Inventor
Srinivas Sridharan
Karthikeyan Vaidyanathan
Dipankar Das
Chandrasekaran Sakthivel
Mikhail E. SMORKALOV
Original Assignee
Intel Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corporation filed Critical Intel Corporation
Publication of PL3506095T3 publication Critical patent/PL3506095T3/pl

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Image Generation (AREA)
  • Computer And Data Communications (AREA)
  • Small-Scale Networks (AREA)
PL18209320.3T 2017-12-29 2018-11-29 Optymalizacje komunikacyjne dla rozproszonego uczenia maszynowego PL3506095T3 (pl)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US15/859,180 US11270201B2 (en) 2017-12-29 2017-12-29 Communication optimizations for distributed machine learning

Publications (1)

Publication Number Publication Date
PL3506095T3 true PL3506095T3 (pl) 2025-07-07

Family

ID=64559624

Family Applications (1)

Application Number Title Priority Date Filing Date
PL18209320.3T PL3506095T3 (pl) 2017-12-29 2018-11-29 Optymalizacje komunikacyjne dla rozproszonego uczenia maszynowego

Country Status (5)

Country Link
US (4) US11270201B2 (pl)
EP (1) EP3506095B1 (pl)
CN (1) CN110135575B (pl)
ES (1) ES3032644T3 (pl)
PL (1) PL3506095T3 (pl)

Families Citing this family (189)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10334334B2 (en) 2016-07-22 2019-06-25 Intel Corporation Storage sled and techniques for a data center
US20180150256A1 (en) * 2016-11-29 2018-05-31 Intel Corporation Technologies for data deduplication in disaggregated architectures
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
US11373266B2 (en) 2017-05-05 2022-06-28 Intel Corporation Data parallelism and halo exchange for distributed machine learning
US20190044809A1 (en) 2017-08-30 2019-02-07 Intel Corporation Technologies for managing a flexible host interface of a network interface controller
US11263162B2 (en) * 2017-12-20 2022-03-01 Intel Corporation System decoder for training accelerators
US11270201B2 (en) 2017-12-29 2022-03-08 Intel Corporation Communication optimizations for distributed machine learning
US10579494B2 (en) * 2018-01-05 2020-03-03 Nec Corporation Methods and systems for machine-learning-based resource prediction for resource allocation and anomaly detection
US11044495B1 (en) 2018-02-13 2021-06-22 Cyborg Inc. Systems and methods for variable length codeword based data encoding and decoding using dynamic memory allocation
US11687762B2 (en) 2018-02-27 2023-06-27 Stmicroelectronics S.R.L. Acceleration unit for a deep learning engine
US11586907B2 (en) * 2018-02-27 2023-02-21 Stmicroelectronics S.R.L. Arithmetic unit for deep learning acceleration
US10977854B2 (en) 2018-02-27 2021-04-13 Stmicroelectronics International N.V. Data volume sculptor for deep learning acceleration
US11645507B2 (en) * 2018-03-23 2023-05-09 Vmware, Inc. Providing models to client devices
US11948073B2 (en) * 2018-04-20 2024-04-02 Advanced Micro Devices, Inc. Machine learning inference engine scalability
CN110503593A (zh) * 2018-05-18 2019-11-26 微软技术许可有限责任公司 多个图形处理单元的调度
US11431648B2 (en) 2018-06-11 2022-08-30 Intel Corporation Technologies for providing adaptive utilization of different interconnects for workloads
US10826851B2 (en) * 2018-07-18 2020-11-03 Ambeent Inc. Method and system for routing delay-tolerant communication messages to a destination device
US10909651B2 (en) * 2018-08-08 2021-02-02 International Business Machines Corporation Graphic processor unit topology-aware all-reduce operation
US10782685B1 (en) 2018-10-10 2020-09-22 Waymo Llc Planner system recovery for autonomous vehicles
US11769041B2 (en) 2018-10-31 2023-09-26 Advanced Micro Devices, Inc. Low latency long short-term memory inference with sequence interleaving
US11526759B2 (en) * 2018-11-05 2022-12-13 International Business Machines Corporation Large model support in deep learning
JP2020077300A (ja) * 2018-11-09 2020-05-21 日本電信電話株式会社 分散深層学習システムおよびデータ転送方法
TWI696961B (zh) * 2018-12-12 2020-06-21 財團法人工業技術研究院 深度神經網路硬體加速器與其操作方法
CN111353575B (zh) 2018-12-20 2025-01-10 超威半导体公司 用于卷积神经网络的图块化格式
US11520331B2 (en) * 2018-12-28 2022-12-06 Intel Corporation Methods and apparatus to update autonomous vehicle perspectives
US11353870B2 (en) * 2018-12-31 2022-06-07 Baidu Usa Llc Autonomous driving computing and storage expansion device with flexible host and client configuration
EP3889846A4 (en) * 2019-01-16 2022-06-01 Huawei Cloud Computing Technologies Co., Ltd. METHOD AND SYSTEM FOR TRAINING DEEP LEARNING MODELS
US10491240B1 (en) * 2019-01-17 2019-11-26 Cyborg Inc. Systems and methods for variable length codeword based, hybrid data encoding and decoding using dynamic memory allocation
US20200242464A1 (en) * 2019-01-29 2020-07-30 Sony Corporation Incremental ai firmware updates using in-device training and peer-to-peer updates
US11580371B2 (en) * 2019-03-13 2023-02-14 Roviero, Inc. Method and apparatus to efficiently process and execute Artificial Intelligence operations
CN111723918B (zh) 2019-03-18 2026-03-24 超威半导体公司 用于卷积内核的自动生成和调谐工具
WO2020210362A1 (en) * 2019-04-08 2020-10-15 Ares Technologies, Inc. Systems, devices, and methods for machine learning using a distributed framework
US10552121B1 (en) * 2019-05-07 2020-02-04 Capital One Services, Llc System and method for dynamic process flow control based on real-time events
US11521042B2 (en) * 2019-05-21 2022-12-06 Anil Ravindranath System and method to dynamically and automatically sharing resources of coprocessor AI accelerators
US11003501B2 (en) * 2019-07-03 2021-05-11 Advanced New Technologies Co., Ltd. Loading models on nodes having multiple model service frameworks
US11520592B2 (en) * 2019-07-15 2022-12-06 Microsoft Technology Licensing, Llc Executing large artificial intelligence models on memory-constrained devices
US11354579B2 (en) * 2019-07-15 2022-06-07 Microsoft Technology Licensing, Llc Dynamic multi-layer execution for artificial intelligence modeling
US11436019B2 (en) * 2019-07-15 2022-09-06 Microsoft Technology Licensing, Llc Data parallelism in distributed training of artificial intelligence models
CN110347636B (zh) 2019-07-15 2024-04-30 北京一流科技有限公司 数据执行体及其数据处理方法
US10956776B2 (en) 2019-08-06 2021-03-23 Alibaba Group Holding Limited 2D convolutional accelerator that generates 3D results
US12249189B2 (en) 2019-08-12 2025-03-11 Micron Technology, Inc. Predictive maintenance of automotive lighting
WO2021032497A1 (en) * 2019-08-16 2021-02-25 Telefonaktiebolaget Lm Ericsson (Publ) Methods, apparatus and machine-readable media relating to machine-learning in a communication network
EP4014171A1 (en) * 2019-08-16 2022-06-22 Telefonaktiebolaget Lm Ericsson (Publ) Methods, apparatus and machine-readable media relating to machine-learning in a communication network
US11042350B2 (en) 2019-08-21 2021-06-22 Micron Technology, Inc. Intelligent audio control in vehicles
US12497055B2 (en) 2019-08-21 2025-12-16 Micron Technology, Inc. Monitoring controller area network bus for vehicle control
US11526761B2 (en) * 2019-08-24 2022-12-13 Microsoft Technology Licensing, Llc Neural network training with decreased memory consumption and processor utilization
US20210065038A1 (en) * 2019-08-26 2021-03-04 Visa International Service Association Method, System, and Computer Program Product for Maintaining Model State
CN110503207A (zh) * 2019-08-28 2019-11-26 深圳前海微众银行股份有限公司 联邦学习信用管理方法、装置、设备及可读存储介质
US12210401B2 (en) 2019-09-05 2025-01-28 Micron Technology, Inc. Temperature based optimization of data storage operations
US11693562B2 (en) * 2019-09-05 2023-07-04 Micron Technology, Inc. Bandwidth optimization for different types of operations scheduled in a data storage device
CN110633796B (zh) * 2019-09-05 2022-04-08 北京达佳互联信息技术有限公司 模型更新方法、装置、电子设备及存储介质
US11915041B1 (en) * 2019-09-12 2024-02-27 Neureality Ltd. Method and system for sequencing artificial intelligence (AI) jobs for execution at AI accelerators
CN112529206B (zh) * 2019-09-18 2024-05-17 华为技术有限公司 一种模型运行方法和系统
US11941528B2 (en) 2019-09-30 2024-03-26 Amazon Technologies, Inc. Neural network training in a distributed system
US12518167B1 (en) * 2019-09-30 2026-01-06 Amazon Technologies, Inc. Neural network training in a distributed system
US11556450B2 (en) * 2019-10-11 2023-01-17 International Business Machines Corporation Hybrid data-model parallelism for efficient deep learning
EP3830764A4 (en) * 2019-10-12 2021-06-09 Baidu.com Times Technology (Beijing) Co., Ltd. METHOD AND SYSTEM TO ACCELERATE AI TRAINING WITH ADVANCED CONNECTION TECHNOLOGIES
US12511543B2 (en) * 2019-11-05 2025-12-30 Nvidia Corporation Distributed weight update for backpropagation of a neural network
CN110908799A (zh) * 2019-11-08 2020-03-24 浪潮电子信息产业股份有限公司 一种分布式训练中的通信方法、装置、设备、介质
US11507840B2 (en) * 2019-11-13 2022-11-22 International Business Machines Corporation Region constrained regularized adversarial examples for model interpretability
WO2021092890A1 (en) * 2019-11-15 2021-05-20 Baidu.Com Times Technology (Beijing) Co., Ltd. Distributed ai training topology based on flexible cable connection
CN111210020B (zh) * 2019-11-22 2022-12-06 清华大学 一种加速分布式机器学习的方法及系统
WO2021097784A1 (en) * 2019-11-22 2021-05-27 Huawei Technologies Co., Ltd. Method and system for constructing compiler intermediate representations from tensorflow graph
US11521062B2 (en) * 2019-12-05 2022-12-06 International Business Machines Corporation Neural network training using a data flow graph and dynamic memory management
US20220414464A1 (en) * 2019-12-10 2022-12-29 Agency For Science, Technology And Research Method and server for federated machine learning
CN112988367B (zh) * 2019-12-12 2024-05-28 中科寒武纪科技股份有限公司 资源的分配方法、装置、计算机设备及可读存储介质
CN112988229B (zh) * 2019-12-12 2022-08-05 上海大学 基于异构计算的卷积神经网络资源优化配置方法
US20210182660A1 (en) * 2019-12-16 2021-06-17 Soundhound, Inc. Distributed training of neural network models
US11521063B1 (en) * 2019-12-17 2022-12-06 Bae Systems Information And Electronic Systems Integration Inc. System and method for terminal acquisition with a neural network
US11250648B2 (en) 2019-12-18 2022-02-15 Micron Technology, Inc. Predictive maintenance of automotive transmission
CN111124696B (zh) * 2019-12-30 2023-06-23 北京三快在线科技有限公司 单元组创建、数据同步方法、装置、单元和存储介质
CN111208980B (zh) * 2019-12-31 2021-04-06 北京九章云极科技有限公司 一种数据分析处理方法和系统
US11580388B2 (en) * 2020-01-03 2023-02-14 Microsoft Technology Licensing, Llc Distributed processing architecture
CN111240743B (zh) * 2020-01-03 2022-06-03 格兰菲智能科技有限公司 人工智能集成电路
US12236343B2 (en) 2020-01-08 2025-02-25 Maxim Integrated Products, Inc. Systems and methods for reducing memory requirements in neural networks
CN113094168A (zh) * 2020-01-08 2021-07-09 北京奇虎科技有限公司 模型的分布式训练方法、装置和系统
CN111262917A (zh) * 2020-01-13 2020-06-09 苏州浪潮智能科技有限公司 一种基于fpga云平台的远端数据搬移装置和方法
CN113128686A (zh) * 2020-01-16 2021-07-16 华为技术有限公司 模型训练方法及装置
CN113138832B (zh) * 2020-01-17 2024-03-01 深圳致星科技有限公司 一种基于重置训练数据传输网络的分布式训练方法及系统
CN111292223B (zh) * 2020-01-22 2023-07-25 北京百度网讯科技有限公司 图计算的处理方法、装置、电子设备及存储介质
US12443831B1 (en) * 2020-01-23 2025-10-14 Nvidia Corporation Neural network execution streams
US11226859B2 (en) * 2020-01-26 2022-01-18 Microsoft Technology Licensing, Llc Systems and methods for error recovery
US11513860B2 (en) 2020-01-31 2022-11-29 Red Hat, Inc. Serverless function colocation with storage pools
DE102020202093A1 (de) 2020-02-19 2021-08-19 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren und Vorrichtung für maschinelles Lernen
CN110955530A (zh) * 2020-02-25 2020-04-03 深圳鲲云信息科技有限公司 深度学习引擎并行处理数据方法、装置、设备及储存介质
US11507486B2 (en) * 2020-03-04 2022-11-22 Dell Products L.P. Method and system for generating latency aware workloads using resource devices in a resource device pool
US11250538B2 (en) 2020-03-09 2022-02-15 Apple Inc. Completion signaling techniques in distributed processor
WO2021183135A1 (en) * 2020-03-13 2021-09-16 Hewlett-Packard Development Company, L.P. Transmitting node instructions
US20210304025A1 (en) * 2020-03-24 2021-09-30 Facebook, Inc. Dynamic quality of service management for deep learning training communication
CN113452541B (zh) * 2020-03-27 2023-02-03 上海商汤智能科技有限公司 网络带宽调整方法和相关产品
US11468325B2 (en) * 2020-03-30 2022-10-11 Amazon Technologies, Inc. Multi-model training pipeline in distributed systems
CN115335804B (zh) * 2020-03-31 2026-03-17 阿里巴巴集团控股有限公司 由第一计算节点实施的方法、第一计算节点及可读介质
CN111178522B (zh) * 2020-04-13 2020-07-10 杭州雄迈集成电路技术股份有限公司 一种软硬件协同加速方法、系统及计算机可读存储介质
CN111507257B (zh) * 2020-04-16 2023-09-08 奇安信科技集团股份有限公司 图片处理方法、装置、系统、介质及程序
US11681598B2 (en) 2020-04-16 2023-06-20 Texas Instruments Incorporated Method and apparatus to facilitate low latency fault mitigation, QoS management and debug of a processing pipeline
CN111582494B (zh) * 2020-04-17 2023-07-07 浙江大学 一种基于延迟处理的混合分布式机器学习更新方法
CN111552563B (zh) * 2020-04-20 2023-04-07 南昌嘉研科技有限公司 一种多线程数据系统、多线程消息传递方法及系统
KR102515188B1 (ko) * 2020-04-22 2023-03-31 한국전력공사 에지 컴퓨팅에서 다중 ami 데이터 스트림 가속처리를 위한 하이브리드 딥 러닝 스케줄링 방법
CN113570028A (zh) * 2020-04-28 2021-10-29 超威半导体公司 用于在神经网络中处理数据的静态生成的经编译表示
US11394799B2 (en) 2020-05-07 2022-07-19 Freeman Augustus Jackson Methods, systems, apparatuses, and devices for facilitating for generation of an interactive story based on non-interactive data
CN113705801B (zh) * 2020-05-22 2025-06-03 华为技术有限公司 一种神经网络模型的训练装置、方法及相关设备
CN113743567B (zh) * 2020-05-28 2025-07-08 平头哥(杭州)半导体有限公司 一种将深度学习模型部署到加速单元的方法
CN111752689B (zh) * 2020-06-22 2023-08-25 深圳鲲云信息科技有限公司 一种基于数据流的神经网络多引擎同步计算系统
CN112308205B (zh) * 2020-06-28 2025-03-18 北京沃东天骏信息技术有限公司 基于预训练模型的模型改进方法及装置
CN111813858B (zh) * 2020-07-10 2022-06-24 电子科技大学 基于计算节点自组织分组的分布式神经网络混合同步训练方法
CN111881345B (zh) * 2020-07-13 2023-06-09 汕头大学 基于位置上下文感知的神经协同过滤服务质量预测方法
US11915122B2 (en) * 2020-07-29 2024-02-27 Micron Technology, Inc. Gateway for distributing an artificial neural network among multiple processing nodes
KR20230051491A (ko) * 2020-08-18 2023-04-18 퀄컴 인코포레이티드 채널 상태 정보에 대한 구성 고려사항들
CN111949422B (zh) * 2020-08-26 2024-04-16 南京云灿信息科技有限公司 基于mq和异步io的数据多级缓存与高速传输记录方法
CN112001351A (zh) * 2020-09-01 2020-11-27 城云科技（中国）有限公司 多个视频流处理方法、系统、计算机设备和存储介质
KR20230061370A (ko) * 2020-09-03 2023-05-08 엘지전자 주식회사 복수의 디바이스들의 학습 결과에 기초하여, 서버가 최종 학습 결과물을 도출하는 분산학습을 위한 통신방법 및 서버
KR20230062822A (ko) * 2020-09-07 2023-05-09 엘지전자 주식회사 수신기의 신호 진폭 범위에서 aircomp에 대한 바이어스를 사용하여 그래디언트 기호를 정렬하는 방법
TWI775170B (zh) * 2020-09-30 2022-08-21 新漢股份有限公司 Cpu應用於人工智慧相關程序時的執行方法
US12159225B2 (en) * 2020-10-14 2024-12-03 Google Llc Queue allocation in machine learning accelerators
JP2023546145A (ja) 2020-10-15 2023-11-01 ドルビー・インターナショナル・アーベー 正弦波活性化を用いるニューラル・ネットワーク・ベースのオーディオ処理のための方法及び装置
US12229078B2 (en) 2020-11-02 2025-02-18 T-Head (Shanghai) Semiconductor Co., Ltd. Neural processing unit synchronization systems and methods
CN112465112B (zh) * 2020-11-19 2022-06-07 苏州浪潮智能科技有限公司 基于nGraph的GPU后端分布式训练方法和系统
CN112381218B (zh) * 2020-11-20 2022-04-12 中国人民解放军国防科技大学 一种用于分布式深度学习训练的本地更新方法
US12117908B2 (en) * 2020-12-04 2024-10-15 Intel Corporation Restoring persistent application data from non-volatile memory after a system crash or system reboot
US11847504B2 (en) 2020-12-16 2023-12-19 Nexcom International Co., Ltd. Method for CPU to execute artificial intelligence related processes
US11782757B2 (en) * 2021-05-07 2023-10-10 SiMa Technologies, Inc. Scheduling off-chip memory access for programs with predictable execution
US12067465B2 (en) 2020-12-17 2024-08-20 SiMa Technologies, Inc. Instruction streaming for a machine learning accelerator
WO2022138232A1 (ja) * 2020-12-23 2022-06-30 ソニーグループ株式会社 通信装置、通信方法、および通信システム
WO2022139230A1 (ko) * 2020-12-24 2022-06-30 엘지전자 주식회사 무선 통신 시스템에서 스플릿 포인트를 조정하는 방법 및 장치
KR20220096466A (ko) * 2020-12-31 2022-07-07 삼성전자주식회사 뉴럴 네트워크 연산을 위한 메모리 관리 방법 및 장치
US11729279B2 (en) * 2021-01-11 2023-08-15 Dell Products, L.P. System and method for remote assisted optimization of native services
US20220222129A1 (en) * 2021-01-12 2022-07-14 GM Global Technology Operations LLC System for parallel processing middleware node application algorithms using threads
CN112712171B (zh) * 2021-01-12 2022-08-12 湖南工业大学 深度卷积神经网络的分布式训练方法、设备和存储介质
CN112748998B (zh) * 2021-01-21 2023-10-03 中南大学 一种移动端的卷积神经网络任务调度方法及系统
US11516311B2 (en) * 2021-01-22 2022-11-29 Avago Technologies International Sales Pte. Limited Distributed machine-learning resource sharing and request routing
CN112799834B (zh) * 2021-01-26 2024-05-07 北京迈格威科技有限公司 训练数据分发方法、装置、电子设备及存储介质
US12430581B2 (en) 2021-02-15 2025-09-30 Bank Of America Corporation Machine learning training device
US12001511B2 (en) * 2021-03-11 2024-06-04 Hewlett Packard Enterprise Development Lp Systems and methods of resource configuration optimization for machine learning workloads
CN115307853A (zh) * 2021-03-23 2022-11-08 索尼集团公司 用于抑制车辆振动的系统、方法和计算机程序
US12450518B2 (en) * 2021-03-29 2025-10-21 Google Llc Evaluating on-device machine learning model(s) based on performance measures of client device(s) and/or the on-device machine learning model(s)
US11271843B1 (en) * 2021-04-20 2022-03-08 Netapp, Inc. Quality of service performance diagnostic mechanism
WO2022225530A1 (en) * 2021-04-23 2022-10-27 Nokia Technologies Oy Low latency machine learning
CN113222118B (zh) * 2021-05-19 2022-09-09 北京百度网讯科技有限公司 神经网络训练方法、装置、电子设备、介质和程序产品
CN113487025B (zh) * 2021-07-02 2023-09-08 浙江大学 面向神经网络检查点数据的智能压缩存储方法和系统
US11385925B1 (en) 2021-07-06 2022-07-12 Bank Of America Corporation System and method for provisioning hosted virtual desktop resources to remote users
US11789783B2 (en) 2021-07-06 2023-10-17 Bank Of America Corporation Hosted virtual desktop slicing using federated edge intelligence
US12387116B2 (en) 2021-07-06 2025-08-12 Bank Of America Corporation Synchronized data collection for use in hosted virtual desktop slicing
CN115599956B (zh) * 2021-07-07 2024-09-24 清华大学 数据处理方法、执行工作站、分布式计算系统和存储介质
EP4120078A1 (en) 2021-07-13 2023-01-18 Aptiv Technologies Limited Method for scheduling feature services within a distributed data flow service framework
US20230024130A1 (en) * 2021-07-23 2023-01-26 Advanced Micro Devices, Inc. Workload aware virtual processing units
CN115843037A (zh) * 2021-08-17 2023-03-24 华为技术有限公司 数据处理方法及装置
US12361268B2 (en) 2021-08-30 2025-07-15 Stmicroelectronics International N.V. Neural network hardware accelerator circuit with requantization circuits
CN113703980B (zh) * 2021-08-31 2024-09-06 西安电子科技大学 一种分布式机器学习系统及适用于其的通信调度方法
CN114936117A (zh) * 2021-09-02 2022-08-23 华为技术有限公司 模型训练的方法、服务器、芯片以及系统
CN113867983B (zh) * 2021-09-14 2025-06-24 杭州海康威视数字技术股份有限公司 图数据挖掘方法、装置、电子设备及机器可读存储介质
GB2605664B (en) 2021-09-30 2023-03-29 Imagination Tech Ltd Processor with hardware pipeline
GB2605665B (en) * 2021-09-30 2023-11-01 Imagination Tech Ltd Graphics processor
GB2605471B (en) 2021-09-30 2023-11-01 Imagination Tech Ltd Processor with hardware pipeline
US20230108177A1 (en) * 2021-10-06 2023-04-06 Google Llc Hardware-Aware Progressive Training Of Machine Learning Models
CN116017496A (zh) * 2021-10-21 2023-04-25 华为技术有限公司 通信方法及装置、系统、存储介质、程序产品
US12493789B2 (en) * 2021-10-21 2025-12-09 Rakuten Mobile, Inc. Cooperative training migration
CN113918351B (zh) * 2021-12-08 2022-03-11 之江实验室 深度学习框架与ai加速卡片内分布式训练适配方法和装置
CN114328105B (zh) * 2021-12-27 2025-11-14 中国船舶重工集团公司第七0九研究所 一种四叉树监控网络设计方法
CN114531669B (zh) * 2022-01-14 2024-06-07 山东师范大学 一种基于车辆边缘计算的任务卸载方法及系统
US12423578B1 (en) * 2022-01-26 2025-09-23 Amazon Technologies, Inc. Distributed training of machine learning models
US11902176B2 (en) 2022-02-24 2024-02-13 Bank Of America Corporation System for detecting multi-dimensional linkage and layering of resource transfers using a distributed server network
CN114840445B (zh) * 2022-03-02 2025-01-28 阿里巴巴（中国）有限公司 内存访问方法和装置
CN118922842A (zh) * 2022-04-07 2024-11-08 华为技术有限公司 分布式推理的方法和装置
CN114866429B (zh) * 2022-04-28 2023-07-18 中国科学技术大学苏州高等研究院 联合路由决策和网内聚合的分布式模型训练方法和装置
CN114820279B (zh) * 2022-05-18 2023-03-24 北京百度网讯科技有限公司 基于多gpu的分布式深度学习方法、装置及电子设备
US12074667B2 (en) * 2022-05-31 2024-08-27 Motional Ad Llc Antenna monitoring and selection
CN117395186A (zh) * 2022-06-29 2024-01-12 华为云计算技术有限公司 一种分布式训练方法、系统及装置
CN115357375B (zh) * 2022-07-15 2025-07-01 中国科学院计算技术研究所 一种面向MPI的Serverless并行计算方法及其系统
CN115314397B (zh) * 2022-08-05 2023-07-21 中科计算技术西部研究院 用于分布式训练的网络模拟方法、系统、装置及存储介质
CN115348320B (zh) * 2022-08-08 2024-07-02 航天科工火箭技术有限公司 一种通信数据的转换方法、装置、及电子设备
CN115473854B (zh) * 2022-08-23 2023-10-24 电子科技大学 一种多模态网络的流量智能控制方法
JP2025532570A (ja) * 2022-09-14 2025-10-01 ホアウェイ・テクノロジーズ・カンパニー・リミテッド 確率情報を使用した推論のための方法、システム、及び装置
CN115271102B (zh) * 2022-09-26 2022-12-16 太极计算机股份有限公司 一种机器学习引擎的任务导向的优先级方法及系统
CN115378818B (zh) * 2022-10-26 2023-02-24 西南民族大学 一种适用于大规模分布式机器学习的新型拓扑设计方法
KR20240103607A (ko) 2022-12-27 2024-07-04 삼성전자주식회사 통합 가상 메모리 관리 방법 및 장치
CN115936095B (zh) * 2023-02-20 2023-05-23 浪潮电子信息产业股份有限公司 一种参数梯度同步方法、装置、设备及存储介质
US20240311153A1 (en) * 2023-03-06 2024-09-19 Microsoft Technology Licensing, Llc Collective communication as a multi-commodity flow problem
US20240314151A1 (en) * 2023-03-16 2024-09-19 International Business Machines Corporation Detecting anomalous activity in a system-on-chip
CN116149759B (zh) * 2023-04-20 2023-07-14 深圳市吉方工控有限公司 Uefi驱动卸载方法、装置、电子设备及可读存储介质
CN119512827A (zh) * 2023-08-25 2025-02-25 华为技术有限公司 分布式计算系统和用于分布式计算系统的故障处理方法
US20250097526A1 (en) * 2023-09-20 2025-03-20 The Nielsen Company (Us), Llc Method and System for Internal Failover of Intermediary Gateway
WO2025131314A1 (en) * 2023-12-22 2025-06-26 Huawei Technologies Co., Ltd. Method for use in a parallel computing system based on message passing for distributing a workload
CN121233354A (zh) * 2024-06-27 2025-12-30 华为技术有限公司 一种集合通信方法及装置
US12423234B1 (en) 2024-07-23 2025-09-23 International Business Machines Corporation Cache governance in a computing environment with multiple processors
US20260032092A1 (en) * 2024-07-24 2026-01-29 Advanced Micro Devices, Inc. Prioritize the earlier step messages for collective algorithms
CN118838721B (zh) * 2024-09-23 2025-02-18 山东浪潮科学研究院有限公司 一种基于gpgpu的mpi集体通信传输系统及方法
US20260104966A1 (en) * 2024-10-11 2026-04-16 Nvidia Corporation Fast checkpointing of neural network training
CN119903015A (zh) * 2025-03-27 2025-04-29 苏州亿铸智能科技有限公司 环形通信方法、环形通信装置以及设备
CN121418294B (zh) * 2025-12-30 2026-03-31 上海数据港股份有限公司 一种数据中心中数据迁移处理方法和系统

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7363422B2 (en) * 2000-01-05 2008-04-22 Rambus Inc. Configurable width buffered module
US7412588B2 (en) * 2003-07-25 2008-08-12 International Business Machines Corporation Network processor system on chip with bridge coupling protocol converting multiprocessor macro core local bus to peripheral interfaces coupled system bus
DE10353532A1 (de) 2003-11-14 2005-06-16 Embigence Gmbh Verfahren und Vorrichtung zur Übertragung von Daten
US7873812B1 (en) 2004-04-05 2011-01-18 Tibet MIMAR Method and system for efficient matrix multiplication in a SIMD processor architecture
US7783822B2 (en) * 2007-07-25 2010-08-24 Hewlett-Packard Development Company, L.P. Systems and methods for improving performance of a routable fabric
US20090089078A1 (en) * 2007-09-28 2009-04-02 Great-Circle Technologies, Inc. Bundling of automated work flow
US8004977B2 (en) * 2007-11-28 2011-08-23 Alcatel Lucent Method of implementing packet-based resource allocation and persistent resource allocation in a wireless communication system
FR2925187B1 (fr) 2007-12-14 2011-04-08 Commissariat Energie Atomique Systeme comportant une pluralite d'unites de traitement permettant d'executer des taches en parallele,en mixant le mode d'execution de type controle et le mode d'execution de type flot de donnees
EP2294808B1 (en) * 2008-06-20 2013-05-01 Hewlett-Packard Development Company, L.P. Method and system for efficient video processing
US8442927B2 (en) 2009-07-30 2013-05-14 Nec Laboratories America, Inc. Dynamically configurable, multi-ported co-processor for convolutional neural networks
CN102708404B (zh) 2012-02-23 2016-08-03 北京市计算中心 一种基于机器学习的多核下mpi最优运行时的参数预测方法
US9571402B2 (en) * 2013-05-03 2017-02-14 Netspeed Systems Congestion control and QoS in NoC by regulating the injection traffic
US9753767B2 (en) * 2014-03-11 2017-09-05 Sas Institute Inc. Distributed data set task selection
CN104035751B (zh) 2014-06-20 2016-10-12 深圳市腾讯计算机系统有限公司 基于多图形处理器的数据并行处理方法及装置
US10223333B2 (en) 2014-08-29 2019-03-05 Nvidia Corporation Performing multi-convolution operations in a parallel processing system
US9785423B2 (en) 2015-04-23 2017-10-10 Google Inc. Compiler for translating between a virtual image processor instruction set architecture (ISA) and target hardware having a two-dimensional shift array structure
CN106297774B (zh) 2015-05-29 2019-07-09 中国科学院声学研究所 一种神经网络声学模型的分布式并行训练方法及系统
US10320695B2 (en) * 2015-05-29 2019-06-11 Advanced Micro Devices, Inc. Message aggregation, combining and compression for efficient data communications in GPU-based clusters
US10158575B2 (en) * 2015-06-17 2018-12-18 Citrix Systems, Inc. System for bandwidth optimization with high priority traffic awareness and control
CN107196877B (zh) * 2016-03-14 2021-07-20 华为技术有限公司 网络流量控制的方法及其网络设备
US10338953B2 (en) * 2016-03-18 2019-07-02 Intel Corporation Facilitating execution-aware hybrid preemption for execution of tasks in computing environments
US11086814B2 (en) * 2016-04-15 2021-08-10 Nec Corporation System and method for communication efficient sparse-reduce
CN107463990A (zh) 2016-06-02 2017-12-12 国家计算机网络与信息安全管理中心 一种卷积神经网络的fpga并行加速方法
US10546211B2 (en) 2016-07-01 2020-01-28 Google Llc Convolutional neural network on programmable two dimensional image processor
US10997496B2 (en) 2016-08-11 2021-05-04 Nvidia Corporation Sparse convolutional neural network accelerator
US10891538B2 (en) 2016-08-11 2021-01-12 Nvidia Corporation Sparse convolutional neural network accelerator
US20180075347A1 (en) * 2016-09-15 2018-03-15 Microsoft Technology Licensing, Llc Efficient training of neural networks
US10810492B2 (en) * 2017-01-27 2020-10-20 Hewlett Packard Enterprise Development Lp Memory side acceleration for deep learning parameter updates
US11373266B2 (en) 2017-05-05 2022-06-28 Intel Corporation Data parallelism and halo exchange for distributed machine learning
US11288575B2 (en) * 2017-05-18 2022-03-29 Microsoft Technology Licensing, Llc Asynchronous neural network training
US11270201B2 (en) 2017-12-29 2022-03-08 Intel Corporation Communication optimizations for distributed machine learning

Also Published As

Publication number Publication date
CN110135575A (zh) 2019-08-16
US20230376762A1 (en) 2023-11-23
US12450484B2 (en) 2025-10-21
US20190205745A1 (en) 2019-07-04
US20220245454A1 (en) 2022-08-04
ES3032644T3 (en) 2025-07-22
US11704565B2 (en) 2023-07-18
US20260119902A1 (en) 2026-04-30
CN110135575B (zh) 2025-11-11
US11270201B2 (en) 2022-03-08
EP3506095B1 (en) 2025-03-19
EP3506095A2 (en) 2019-07-03
EP3506095A3 (en) 2019-09-25

Similar Documents

Publication Publication Date Title
PL3506095T3 (pl) Optymalizacje komunikacyjne dla rozproszonego uczenia maszynowego
PL3396547T3 (pl) Optymalizacje obliczeń dla operacji uczenia maszynowego o niskiej precyzji
EP3631692A4 (en) QUATERNION-BASED MACHINE LEARNING SYSTEM THAT IS COMPUTING EFFICIENT POINT OF VIEW
EP3651503A4 (en) COMMUNICATION CONTROL METHOD
EP3618485A4 (en) COMMUNICATION SYSTEM
EP3565294A4 (en) COMMUNICATION SYSTEM
EP3562243A4 (en) COMMUNICATION SYSTEM
PL3665788T3 (pl) Kwazi-kolokalizacja dla formowania wiązki
EP3639451C0 (en) COMMUNICATION SYSTEM
DK3224972T3 (da) Passive optiskbaserede datacenternetværk
EP3678399A4 (en) COMMUNICATION DEVICE
EP3613940A4 (en) ROTARY GUIDANCE DEVICE
DK3612519T3 (da) Phenyl-2-hydroxy-acetylamino-2-methyl-phenyl forbindelser
EP3605867A4 (en) COMMUNICATION SYSTEM
EP3574061A4 (en) ENCAPSULATES
EP3484191A4 (en) Communications device
EP3836691A4 (en) COMMUNICATION DEVICE
EP3476150A4 (en) NON-ORIGINAL COMMUNICATION
EP3707914A4 (en) IMPROVED REPRODUCTION
PL3714627T3 (pl) Ustanowienie komunikacji
EP3836731A4 (en) COMMUNICATION DEVICE
EP3562112A4 (en) COMMUNICATION SYSTEM
DK3272129T3 (da) Kommunikationssystem
EP3713194A4 (en) RSSP-I SECURE COMMUNICATION PROCESS USING A SYNCHRONIZATION OF TWO SYSTEMS
EP3619920A4 (en) AUDIO OBJECT INTERACTIONS WITHOUT METADATA