CN116680296B - 一种基于单机的大规模图数据处理系统 - Google Patents

一种基于单机的大规模图数据处理系统

Info

Publication number
CN116680296B
CN116680296B CN202310695465.2A CN202310695465A CN116680296B CN 116680296 B CN116680296 B CN 116680296B CN 202310695465 A CN202310695465 A CN 202310695465A CN 116680296 B CN116680296 B CN 116680296B
Authority
CN
China
Prior art keywords
data
module
graph
sub
subgraph
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310695465.2A
Other languages
English (en)
Chinese (zh)
Other versions
CN116680296A (zh
Inventor
朱筱可
刘阳
刘书昊
樊文飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Computing Sciences
Original Assignee
Shenzhen Institute of Computing Sciences
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Computing Sciences filed Critical Shenzhen Institute of Computing Sciences
Priority to CN202310695465.2A priority Critical patent/CN116680296B/zh
Priority to PCT/CN2023/101407 priority patent/WO2024254894A1/fr
Publication of CN116680296A publication Critical patent/CN116680296A/zh
Application granted granted Critical
Publication of CN116680296B publication Critical patent/CN116680296B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/24569—Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24532—Query optimisation of parallel queries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G06F16/9024—Graphs; Linked lists
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/90335—Query processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/544—Buffers; Shared memory; Pipes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/546—Message passing systems or structures, e.g. queues
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
CN202310695465.2A 2023-06-12 2023-06-12 一种基于单机的大规模图数据处理系统 Active CN116680296B (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202310695465.2A CN116680296B (zh) 2023-06-12 2023-06-12 一种基于单机的大规模图数据处理系统
PCT/CN2023/101407 WO2024254894A1 (fr) 2023-06-12 2023-06-20 Système de traitement de données de graphe à grande échelle basé sur une unité unique

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310695465.2A CN116680296B (zh) 2023-06-12 2023-06-12 一种基于单机的大规模图数据处理系统

Publications (2)

Publication Number Publication Date
CN116680296A CN116680296A (zh) 2023-09-01
CN116680296B true CN116680296B (zh) 2026-01-13

Family

ID=87790518

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310695465.2A Active CN116680296B (zh) 2023-06-12 2023-06-12 一种基于单机的大规模图数据处理系统

Country Status (2)

Country Link
CN (1) CN116680296B (fr)
WO (1) WO2024254894A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241353A (zh) * 2020-01-16 2020-06-05 支付宝(杭州)信息技术有限公司 一种图数据的分区方法、装置以及设备
CN113392280A (zh) * 2021-06-10 2021-09-14 东北大学 一种面向跨区域的多主模型分布式图计算方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2003287815A1 (en) * 2002-12-02 2004-06-23 Mount Sinai Hospital Methods and products for representing and analyzing complexes of biological molecules
CN112352234B (zh) * 2018-06-15 2024-03-08 华为云计算技术有限公司 用于处理并发属性图查询的系统
WO2020019313A1 (fr) * 2018-07-27 2020-01-30 浙江天猫技术有限公司 Procédé de mise à jour de données de graphe, système, support d'informations lisible par ordinateur et dispositif
CN109308327A (zh) * 2018-09-19 2019-02-05 浙江天猫技术有限公司 基于子图模型兼容点中心模型的图计算方法装置介质设备
US20200249998A1 (en) * 2019-02-01 2020-08-06 Alibaba Group Holding Limited Scheduling computation graph heterogeneous computer system
CN111859027B (zh) * 2019-04-24 2024-09-27 华为云计算技术有限公司 图计算方法及装置
CN112988064B (zh) * 2021-02-09 2022-11-08 华中科技大学 一种面向并发多任务的磁盘图处理方法
CN113434702A (zh) * 2021-07-27 2021-09-24 支付宝(杭州)信息技术有限公司 一种用于图计算的自适应控制方法和系统
CN114756483A (zh) * 2022-03-31 2022-07-15 深圳清华大学研究院 基于核间存储访问的子图分段优化方法及应用

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241353A (zh) * 2020-01-16 2020-06-05 支付宝(杭州)信息技术有限公司 一种图数据的分区方法、装置以及设备
CN113392280A (zh) * 2021-06-10 2021-09-14 东北大学 一种面向跨区域的多主模型分布式图计算方法

Also Published As

Publication number Publication date
WO2024254894A1 (fr) 2024-12-19
CN116680296A (zh) 2023-09-01

Similar Documents

Publication Publication Date Title
CN110704360B (zh) 一种基于异构fpga数据流的图计算优化方法
Wang et al. Optimizing load balancing and data-locality with data-aware scheduling
CN110619595A (zh) 一种基于多fpga加速器互联的图计算优化方法
Bryk et al. Storage-aware algorithms for scheduling of workflow ensembles in clouds
US12314851B2 (en) Microservice-based training systems in heterogeneous graphic processor unit (GPU) cluster and operating method thereof
US20150066157A1 (en) Parallel Processing with Cooperative Multitasking
KR20210057184A (ko) 이종 cpu/gpu 시스템에서 데이터 흐름 신호 처리 애플리케이션 가속화
CN110262901A (zh) 一种数据处理方法及数据处理系统
CN111756802B (zh) 一种数据流任务在numa平台上的调度方法及系统
JP7050957B2 (ja) タスクスケジューリング
CN106164881A (zh) 异构计算系统中的工作窃取
TW201337751A (zh) 執行成型記憶體存取作業的系統和方法
CN118519768A (zh) 向共享缓存进行数据溢出的方法、装置、设备和存储介质
JP2025542316A (ja) ワークスケジューリングの動的制御
WO2016024508A1 (fr) Dispositif multiprocesseur
CN105988856B (zh) 解释器访存优化方法及装置
CN118277490B (zh) 数据处理系统、数据同步方法、电子设备和存储介质
CN107528871A (zh) 存储系统中的数据分析
CN116680296B (zh) 一种基于单机的大规模图数据处理系统
CN120335953A (zh) 基于强化学习的numa架构下多核处理器任务调度方法
CN119292741A (zh) 可重构处理器的任务管理和调度方法
CN114741166B (zh) 一种分布式任务的处理方法、分布式系统及第一设备
US8090762B2 (en) Efficient super cluster implementation for solving connected problems in a distributed environment
Singh Communication Coroutines For Parallel Program Using DW26010 Many Core Processor
CN119861974B (zh) 基于数据流核心架构的任务处理方法、装置、设备及介质

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant