CN105683967A - 基于大数据的网页抓取方法及系统 - Google Patents

基于大数据的网页抓取方法及系统 Download PDF

Info

Publication number
CN105683967A
CN105683967A CN201680000295.XA CN201680000295A CN105683967A CN 105683967 A CN105683967 A CN 105683967A CN 201680000295 A CN201680000295 A CN 201680000295A CN 105683967 A CN105683967 A CN 105683967A
Authority
CN
China
Prior art keywords
webpage
user
web page
big data
classification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201680000295.XA
Other languages
English (en)
Chinese (zh)
Inventor
马岩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Boxinnuoda Trading Consultation Co Ltd
Original Assignee
Shenzhen Boxinnuoda Trading Consultation Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Boxinnuoda Trading Consultation Co Ltd filed Critical Shenzhen Boxinnuoda Trading Consultation Co Ltd
Publication of CN105683967A publication Critical patent/CN105683967A/zh
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
CN201680000295.XA 2016-01-30 2016-01-30 基于大数据的网页抓取方法及系统 Pending CN105683967A (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/072923 WO2017128357A1 (fr) 2016-01-30 2016-01-30 Procédé à base de mégadonnées et système d'analyse de page web

Publications (1)

Publication Number Publication Date
CN105683967A true CN105683967A (zh) 2016-06-15

Family

ID=56215757

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201680000295.XA Pending CN105683967A (zh) 2016-01-30 2016-01-30 基于大数据的网页抓取方法及系统

Country Status (2)

Country Link
CN (1) CN105683967A (fr)
WO (1) WO2017128357A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107682382A (zh) * 2016-08-01 2018-02-09 汇仕电子商务（上海）有限公司 一种互联网大数据采集系统及其使用方法
WO2018027456A1 (fr) * 2016-08-08 2018-02-15 深圳市博信诺达经贸咨询有限公司 Procédé et système de spécification d'application à partager dans des mégadonnées

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101189604A (zh) * 2005-05-10 2008-05-28 微软公司 使用自动列表的组合查询
CN104077397A (zh) * 2014-07-01 2014-10-01 成都康赛信息技术有限公司 一种分布式的大数据分类检索网页的响应方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102915352A (zh) * 2012-10-08 2013-02-06 清华大学 一种自动检索并整合网络信息的装置
US9235638B2 (en) * 2013-11-12 2016-01-12 International Business Machines Corporation Document retrieval using internal dictionary-hierarchies to adjust per-subject match results

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101189604A (zh) * 2005-05-10 2008-05-28 微软公司 使用自动列表的组合查询
CN104077397A (zh) * 2014-07-01 2014-10-01 成都康赛信息技术有限公司 一种分布式的大数据分类检索网页的响应方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107682382A (zh) * 2016-08-01 2018-02-09 汇仕电子商务（上海）有限公司 一种互联网大数据采集系统及其使用方法
WO2018027456A1 (fr) * 2016-08-08 2018-02-15 深圳市博信诺达经贸咨询有限公司 Procédé et système de spécification d'application à partager dans des mégadonnées

Also Published As

Publication number Publication date
WO2017128357A1 (fr) 2017-08-03

Similar Documents

Publication Publication Date Title
CN105683966A (zh) 基于大数据的搜索方法及系统
CN107835498B (zh) 一种用于管理用户的方法与设备
CN110019263B (zh) 信息存储方法和装置
CN105849730A (zh) 数据抓取的方法及系统
CN110266505B (zh) 一种管理会话群的方法与设备
CN103501241B (zh) 垃圾文件清理方法、装置和系统
CN105701161A (zh) 实时大数据用户标签系统
CN103617267A (zh) 社交化扩展搜索方法及装置、系统
CN104462096B (zh) 舆情监测分析方法和装置
CN113704638A (zh) 一种在社交群聊中识别呈现信息的方法与设备
CN107070729A (zh) 网络链接中主设备推选方法及系统
US20120072359A1 (en) Location Based Friend Finding
CN110008462A (zh) 一种命令序列检测方法及命令序列处理方法
CN105683967A (zh) 基于大数据的网页抓取方法及系统
CN103841121B (zh) 一种基于本地文件的评论和互动系统及方法
WO2018228225A2 (fr) Procédé et dispositif de recommandation d'utilisateur
CN107291923A (zh) 信息处理方法和装置
CN111127232A (zh) 兴趣圈发现方法、装置、服务器和介质
CN113407889A (zh) 小说转码方法、装置、设备以及存储介质
CN107040454A (zh) 一种大数据量快速更新场景下的未读消息提醒方法及装置
CN108062277B (zh) 一种电子凭据数据访问方法、装置及系统
CN112069267A (zh) 一种数据处理方法和装置
US9251125B2 (en) Managing text in documents based on a log of research corresponding to the text
CN111125506B (zh) 兴趣圈主题确定方法、装置、服务器和介质
CN106462883A (zh) 房地产销售数据统计方法及系统

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20160615