EP1776643A2 - Protocoles de schemas d'objets et de chaines de paquets destines a la gestion du routage et de la distribution de contenus numeriques dans des structures de connexion dynamiques pair a pair - Google Patents

Protocoles de schemas d'objets et de chaines de paquets destines a la gestion du routage et de la distribution de contenus numeriques dans des structures de connexion dynamiques pair a pair

Info

Publication number
EP1776643A2
EP1776643A2 EP04795287A EP04795287A EP1776643A2 EP 1776643 A2 EP1776643 A2 EP 1776643A2 EP 04795287 A EP04795287 A EP 04795287A EP 04795287 A EP04795287 A EP 04795287A EP 1776643 A2 EP1776643 A2 EP 1776643A2
Authority
EP
European Patent Office
Prior art keywords
node
dcs
file
connection
content
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP04795287A
Other languages
German (de)
English (en)
Inventor
Vincent Addessi
Jamie Addessi
Chris Bick
Ken Hausam
Kirk Feathers
Greg Kerber
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wurld Media Inc
Original Assignee
Wurld Media Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wurld Media Inc filed Critical Wurld Media Inc
Publication of EP1776643A2 publication Critical patent/EP1776643A2/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/12—Applying verification of the received information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1044—Group management mechanisms 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1044—Group management mechanisms 
    • H04L67/1046—Joining mechanisms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
    • H04L67/1076—Resource dissemination mechanisms or network resource keeping policies for optimal resource availability in the overlay network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/61—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources taking into account QoS or priority requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W84/00—Network topologies
    • H04W84/18—Self-organising networks, e.g. ad-hoc networks or sensor networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1044—Group management mechanisms 
    • H04L67/1051—Group master selection mechanisms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W40/00—Communication routing or communication path finding
    • H04W40/02—Communication route or path selection, e.g. power-based or shortest path routing

Definitions

  • the present invention relates to the creation and maintenance of a dynamic centrally organized network topology and the use of that topology in conjunction with a routing scheme for multi-media file distribution via a multi-media protocol for transferring files, or Multi-Media Transfer Protocol (MMTP).
  • the invention organizes nodes into one or more types of Dynamic Connection Structure (DCS) such that the available bandwidth for each node, both upstream as well as downstream, is utilized to the fullest extent necessary in order to maximize the efficiency of file transfers.
  • DCS Dynamic Connection Structure
  • enabled nodes of the invention are adapted and configured so that they can both receive and send portions, or chunks, of a file virtually simultaneously.
  • FTP Transfer Protocol
  • Recent art makes use of peer-to-peer technology, in which a client receives a file and then sends it on to other clients. Use of this technology decreases the load on the original file server by utilizing the processing and transmitting capabilities of the clients. Unfortunately, conventional embodiments of such conventional technology require the entire file to be transmitted to a client before that file can be retransmitted to other clients, thereby producing undesirable delays.
  • File sharing programs such as Napster and Gnutella use peer-to-peer technology to transmit files between clients. In these systems, clients search for available files located on other client computers. A client is allowed to transfer a desired file via FTP from another client. These systems do not make provisions to transfer one file efficiently to a large number of clients. Instead, they merely give a client a wide selection of other clients from which a file may be received, to thereby increase the probability that an available source is found.
  • One method of transferring files by cascaded release describes an application of peer-to-peer technology that allows a file to be transmitted from a server to a number of clients. See U.S. Patent No. 5,995,982.
  • This method allows a group of clients to receive a file from the server and then send the file on to other clients.
  • This method arranges a list of clients into groups, and does not allow a client of the group to begin transmission until a client in the previous group has fully received the file.
  • the organization of connections between clients in the list remains static once the list is created.
  • this method does not make provisions for dynamically redefining the distribution list by adding, removing, and repositioning clients while files are being transmitted between clients in the list.
  • U.S. Patent Publication Number 2003/0139934 "Requesting and Providing Services via a Registry" to Mandera describes a method for transferring files in a peer-to- peer network wherein requests are split up into sub-requests and a registry server is used to find peers that can fulfill the sub-requests. However, no firewall provisions are described and the responding peer is responsible for further forwarding further sub-requests.
  • U.S. Patent Publication Number 2002/0007350 System and Method for On- Demand Data Distribution in a P2P system
  • U.S. Patent Publication Number 2003/0125964 System and Method for Controlling Distribution of Digital Copyrighted Material using a Multi-Level Marketing Model
  • U.S. Patent Publication Number 2003/0149620 System for Offering Services Using Network of Unowned Computers
  • Gaither unlike the present invention, an entire file must be distributed from one computer. Moreover, no firewall protection is provided by Gaither et al., and there is no attempt to prevent pirating of intellectual property.
  • U.S. Patent Publication Number 2002/0169700 Digital Content Subscription Conditioning System
  • Huffman et al. describes distribution services in a peer-to-peer system in which individual users store content on their local computers making the content available to other users for free or for a fee.
  • DSP Digital Service Provider
  • Kitze et al. describes a financial model for the digital file marketplace which includes generating revenue from the delivery of files and paying royalties to content owners from the revenue collected.
  • no topology is described and no routing is described.
  • the copyright owner must be proactive.
  • U.S. Patent 5,884,031 "Method for Connecting Client Systems into a Broadcast network" by Ice describes a private network built by allowing a predetermined number of clients to connect directly to a server. After this occurs, additional client systems requesting connection are furnished with the addresses of client systems already connected within the private network. Each of the additional client systems then makes connections with a multiple number of client systems to receive information from the server system. Each of these client systems subsequently accepts connections from up to a second pre- determined number of client systems to which it transmits information received from the server system.
  • the present invention describes a dynamic connection structure (DCS) in which nodes connect and disconnect dynamically.
  • DCS dynamic connection structure
  • the DCS of the present invention is highly intelligent and considers geography, firewalls, subnets, speed (upstream and downstream), bandwidth etc. in situating a node connecting to the network as well as in DCS (topology) re- organization/maintenance.
  • the present invention also dynamically re-organizes the topology of the DCS.
  • the present invention packetizes files and transfers those files as packets, and retransmits those packets virtually simultaneously, and as necessary, among and between the nodes of the DCS.
  • systems of the present invention perform data routing and message routing between nodes.
  • the present invention simultaneously routes files, data and messages between nodes of the DCS.
  • a special node communicates node instructions regarding connections to affected nodes.
  • Ice describes a redundant tree or pyramid architecture in contrast to the binary heaps and meshes of the present invention.
  • any node can be a server/seed node/root server, that is, any node of the present invention can distribute content. This is in stark contrast to the single server described by Ice.
  • the server described by Ice performs both network organization and routing functions with routing of data accomplished via broadcast in contrast to the routing of packetized files between nodes of the present invention.
  • U.S. Patent 6,584,073 Network Topologies
  • Steele et al. describes network topologies selected to improve network performance including adding nodes and removing nodes from the network.
  • the mesh of the present invention is not organized (or re-organized) in the same manner as the network topologies described by Steele.
  • the meshes of the present invention are organized using criteria analysis algorithms.
  • the traffic cop node of the present invention looks at a large number of nodes and selects those that fit the criteria best.
  • the nodes of O'Neal propagate data files linearly from start to finish.
  • the server described by O'Neal does not dictate the structure of the network but rather passes addresses of nodes within the network. Then, the nodes described by O'Neal manage the structure themselves (decentralized) by sending messages around. For example, if a node determined that it was not connected in an optimal way, it would send messages to its neighbors and facilitate connection changes itself to try to improve its situation in contrast to the centralized organization and re-organization of the topology of the present invention.
  • BitTorrent is a system in which the publisher of a file may effect peer-to-peer transfer.
  • a tracker helps peers find each other in order to obtain fixed size pieces of an available file by requesting the same pieces of files from multiple peers.
  • BitTorrent supports no central resource allocation.
  • the invention provides a method for transferring a file from one computer to many computers distributed across a network.
  • a program called a server is executed on a computer connected to the network.
  • a program called the root client is executed on a computer connected to the network which has direct access to the file which is to be transferred.
  • a program called the client is executed on each computer which requires receipt of the specified file.
  • the MMTP is an application layer protocol as is file transfer protocol (FTP).
  • FTP file transfer protocol
  • MMTP is a combination of the DCS (topology) and routing scheme described herein.
  • the combination of topology and routing address the problem of copyright infringement as well as better utilizing both unutilized and under-utilized upstream and downstream bandwidth of nodes connected, for example, to the Internet or cable.
  • the present invention is adaptable for use in related applications such as the re-engineering of business processes related to the distribution of audio, video, streaming media, interactive chat, interactive games, etc.
  • the present invention provides the enabling technology, which concurrently addresses the issues of efficient use of bandwidth and system resources and further provides for delivery of multiple files over the same DCS.
  • a DCS is a peer-to-peer network of nodes in communication with each other and with a traffic cop node, which organizes and maintains (re-qrganizes) the topology of the network.
  • One important aspect of the present topology systems is a data structure containing connection change information, which is described in greater detail hereinbelow with respect to the packet chain portion of the multimedia transport protocol (MMTP).
  • MMTP multimedia transport protocol
  • Routing employs Access Maps, which create an intelligent awareness among peer nodes of the proximity of each chunk of the requested multi media file, and request maps, which guide the data flow to maximize the efficiency of each chunk transmission.
  • QoS quality of service
  • the invention provides a method by which a requesting node in a DCS can locate and select which chunk of a file set to request, the method comprising the steps of: issuing a request for one or a plurality of the chunks of the file set to peer nodes to which the requesting node is connected via the DCS, in response to the request, locating the one or the plurality of chunks of the file set, wherein the one or plurality of chunks are stored in a memory or storage means of a second node of the dynamic connection structure, and using Access Maps and Request Maps, selecting randomly at least one of the file chunks from a pre-determined number of qualifying chunks of the second node, if the file set of the second node is streaming, and the predetermined number of chunks are the size of a streaming buffer.
  • the method may further comprise the steps of determining, with respect to the second node, which chunks of the requested file set are not in the second node, issuing one or more additional requests for one or a plurality of the chunks of the file set not in the second node, the additional requests being issued to additional peer nodes to which the requesting node is connected via the DCS, in response to the additional requests, locating the one or the plurality of chunks of the file set, wherein the one or plurality of chunks are stored in a memory or storage means of the additional nodes of the dynamic connection structure, and using Access Maps and Request Maps, selecting randomly at least one of the file chunks from a predetermined number of qualifying chunks of the additional nodes, if the file set of the second node is streaming, and the predetermined number of chunks are the size of a streaming buffer.
  • the present methods may further comprise the step or act of utilizing a plurality of object schemas to effect topology of the DCS and the routing and distribution of content via the DCS, and wherein the object schemas pertaining to the routing and distribution include a file set information (FSI) object schema and a file information (FI) object schema.
  • the FSI object schema may include also a type of transfer to use when delivering a file set, a digest of an entire file set and a list of files in the file set as well as includes a name of a file in a file set, a digest of the file, a size of the file, a relative location in which to save the file, a MIME type of the file and any additional information about the file.
  • the object schemas pertaining to the topology of the DCS may include a connection change information list (CCIL) object schema and a connection change information (CCI) object schema such that the CCI object schemas are passed to servants to connect to the DCS and to connected servants to reposition the connected servants within the DCS.
  • the CCIL object schema includes a timestamp, a signature and at least one CCI name, wherein the CCI includes a session ID.
  • the present methods may include wherein the CCI object schema includes a session ID, a source, a timestamp, a signature, a list of connections to which the source is connected, an indication of whether the connection is to be maintained, an indication of whether a current connection is to a root server and an indication of whether the current connection uses a buddy connection, and also wherein a message requesting insertion into the DCS is sent frorft a node to the traffic cop node.
  • the message requesting insertion into the DCS may include a URL, a request data header and a response body, and the URL may include a host address, a listening port and a URN of content, and wherein the host address is an IP address of the traffic cop node.
  • the listening port may be the port on which the traffic cop node is listening.
  • the URN of content may be a URN of hash information of ontent and the request data header may include a packet chain protocol (PCP) client version, a client ID, an indication of firewall status, a listening port, downstream speed and a host address and the PCP client version may be a protocol version.
  • PCP packet chain protocol
  • the present methods may include wherein the listening port is an IP address and a listening port of the node as well as wherein the host address is an IP address of the node forwarding the message requesting insertion into the DCS, and wherein the response body includes data, the data including a message having information about how the node/servant forwarding the message requesting insertion into the DCS can join the DCS and detailed information about the requested content.
  • the present methods include wherein a message requesting deletion from the DCS includes a URL, a request data header and a response body, wherein the URL includes a host address, a listening port and a URN of content, and wherein the host address is an IP address of the traffic cop node, as well as wherein the listening port is a port on which the traffic cop node is listening.
  • the methods include wherein the traffic cop node performs one or more of the following acts: validating a timestamp and a signature and disposing of the message requesting insertion into the DCS if either validation fails; locating the connection change information object which matches a local node ID; removing a connection list from the connection change information object; initiating a new connection for each ID in the connection list, where the node ID does not match an existing connection; extracting the node's connection change information and propagating the connection change information; and verifying the timestamp of the message is greater than a last applied child connection change message timestamp, if a no connection change flag is set, and forwarding the traffic cop node a connection update message.
  • the invention includes further wherein a plurality of object schemas/messages comprising object schemas are used to effect topology of a dynamic connection structure (DCS) and object schemas are further used to effect distribution of content via, that is, over or across or through the DCS.
  • DCS dynamic connection structure
  • the object schemas which pertain to routing and distribution may include a file set information (FSI) object schema and a file information (FI) object schema, wherein the FSI object schema includes a type of transfer to use when delivering a file set, a digest of an entire file set and a list of files in the file set, and wherein the FI object schema includes a name of a file in a file set, a digest of the file, a size of the file, a relative location in which to save the file, a MIME type of the file and any additional information about the file.
  • the plurality of object schemas of the invention include those wherein the content is digital, such as multi-media content.
  • a dynamic connection structure (DCS) of the invention comprises a plurality of nodes, wherein the plurality comprises at least one first node, the first node being connected to at least one second node; and a traffic cop node for overseeing, adapting and maintaining the topology of the DCS, wherein the DCS forms a peer-to-peer architecture.
  • the topology of a DCS of the invention refers to the nodes and the connections and interconnections between or among any two or more nodes of a DCS, or between any node and the nodes to which it is connected.
  • each of the nodes is enabled to perform the functions of self- insertion, that is, inserting itself into an existing DCS in accordance with instructions received from the traffic cop node; and of reorganizing itself positionally in the DCS with respect to other nodes of the.plurality, wherein the reorganization is in accordance with instructions received by that node from the traffic cop node.
  • each of the nodes is enabled to perform the self-insertion and the reorganization of the DCS simultaneously.
  • each of the nodes of a DCS of the invention is enabled to function as both a server and a servant, and in certain respects can further perform these functions simultaneously, and can function to provide content suitable for distribution to one or a plurality of the nodes in the DCS, and which content is distributed by way of the nodes in, over, by way of, or through the DCS.
  • any of the plurality of nodes such as at least one first node and at least one second node, are enabled to function both as a server to provide that content suitable for distribution to one or a plurality of the other nodes in the DCS, and as a servant to receive that content, or other content, which is distributed by way of the nodes in the DCS.
  • each or any of the plurality of nodes are enabled to function both as a server and as a servant with respect to first content and with respect to additional content and both the server functions and the servant functions can be performed simultaneously with respect to the first content and with respect to the additional content.
  • any of the nodes of the DCS, or any group of nodes of the DCS are continually organized and reorganized in accordance with instructions conveyed from the traffic cop node.
  • the organization and re- organization is centrally controlled and is a function of the software and information possessed or accessed by the traffic cop node.
  • Information necessary for the organization and the re-organization of the DCS can be provided to the traffic cop node in a number of ways, such as from a centrally maintained database in the traffic cop node itself, or which is stored in one or more of the traffic cop node, and one or more digital media accessible to the traffic cop node such as a traffic cop server or a central server functionally connected to the traffic cop.
  • information and communications necessary to organize and reorganize the DCS is decentralized and is among the plurality of nodes.
  • a buddy connection is effected.
  • the traffic cop node processes all connection change information.
  • the present invention is configured and arranged such that the traffic cop node maintains a connection list of all active transmission connections and, preferably, the connection list is grouped in accordance with one or more universal resource nodes (URN's).
  • the traffic cop node maintains a connection list of all active transmission connections, and wherein the connection list is grouped in accordance with one or more universal resource nodes (URN's).
  • connections between a first node and a second node of the DCS can be established simultaneously while connections between the first node and a second node of the DCS are broken.
  • DCS can be organized and re-organized simultaneously with the establishment and breaking of the connections.
  • peer level nodes of the invention are not necessarily always connected to the traffic cop, but can connect, disconnect and reconnect to the traffic cop intermittently as needed to send and receive information regarding a requested or acj al change in the status or position of one or more nodes.
  • a dynamic connection structure (DCS) of the invention can be of any topology, for example a binary heap, an incomplete binary heap, a binary tree, an m-ary tree, a graph, a chain, a mesh, a ring or a bus.
  • a DCS of the invention can be two or more interconnected systems selected from the above-mentioned list of topologies.
  • DCS systems of the invention are organized and re-organized according to a number of attributes to maximize efficiency by maximizing usage of upstream and downstream bandwidth capacity, for example, network connection speed, firewall status, geographic location, IP sub-net address, location of chunks of content by sub-net address and user account information.
  • a DCS of the invention may comprise almost any number of nodes, for example at least a third, fourth and fifth node, or at least dozens or hundreds of nodes, for example least three hundred nodes, or at least six hundred nodes or at least one thousand nodes, or more.
  • Methods of the invention include those for changing or adapting the topology or topologies to add or delete nodes, to connect nodes to additional nodes, and to disconnect a node from one node and connect that node to another node, and to disconnect one node from one of a group of connected nodes while maintaining its connections to the other nodes of the group.
  • the invention includes a method for processing a connection change in a dynamic connection structure (DCS), the method comprising the steps or actions of adding a new connection to a connection list for a specified universal resource node (URN); validating a timestamp to ensure the connection change is more recent than a last processed connection change; validating a security signature; locating a data object corresponding to the connection change, the data object having a matching local session ID for the specified URN and storing connection change information in the data object; and propagating a connection change message to a connection identified in the data object.
  • DCS dynamic connection structure
  • the method may further comprise the act or step of storing a null value in the data object if no valid entry is located, and the step or act of initiating a connection to a specified node if the data object is not null, and the local session ID does not match an existing connection.
  • the act or step of initiating may further include the initiation of an outgoing connection, and the addition of the outgoing connection to the connection list. Also, the step or act of initiating may further comprise initiating a buddy connection if the node is firewall-protected, that is, if the node is behind a software firewall. Further acts or steps of the invention may comprise the acts or steps of removing the connection from the connection list when, for any existing connection for the specified URN, there is no data object, and transmitting a message to the connection indicating connection deletion. [ ⁇ 46] The present invention also includes methods for creating a connection between a first node and a second node of a dynamic connection structure (DCS).
  • DCS dynamic connection structure
  • the method comprises the steps or acts of: sending, by the first node, a connection request to the second node; acknowledging, by the second node, receipt of the connection request from the first node; establishing the connection between the first and the second node; requesting, by the first node, access to a file set and forwarding, by the ⁇ first node, access information to the second node; validating, by the second node, the access information forwarded by the first node; granting, by the second node to the first node, access to the file set; and forwarding, by the first node, the access information of the first node to all other nodes in the dynamic connection structure that are affected by the creation of the connection between the first and second nodes.
  • the method may further comprise the additional steps or acts of: denying access to the file set if the access information is not validated, and deleting the established connection.
  • the first node and the second node can each be one of a server (or "seed node") and a servant, and each of the first and the second nodes may simultaneously be a server (or seed node) and a servant with respect to the same or different portions of one or more file sets.
  • a node of the invention can receive a first chunk, or portion, of a first file set from a neighboring or distant node in the DCS, and simultaneously send the same first chunk, or portion, of that file set to yet another node with which it is directly or indirectly connected in the DCS.
  • a node of the invention is enabled to send a portion, or chunk, of a first file set while simultaneously receiving a portion of a second, that is, different, file set.
  • the functions of receiving and sending can occur simultaneously in a particular node with respect to various related or unrelated file sets traveling across a DCS of the invention.
  • the present methods include also those for deleting one or more nodes from a dynamic connection structure.
  • the method comprises the acts or steps of receiving, by a traffic cop node, a request from the node to leave the DCS; and forwarding, by the traffic cop node, a connection list to nodes affected by disconnection of the node from the DCS.
  • disconnection of a particular node in a DCS of the invention includes communications conducted by and through the traffic cop node such that nodes to which the disconnecting node is connected are informed of the disconnection and reorganization.
  • Numerous nodes can be enabled in accordance with the present invention to participate in one or more dynamic connection structures.
  • a DCS of the invention comprises; at least one first node connected to a plurality of second nodes; and a traffic cop node configured and arranged for overseeing the topology of the DCS, wherein the DCS forms a peer-to-peer architecture, and wherein each of the at least one first node and each of the plurality of second nodes are enabled to perform the functions of i) inserting itself into the DCS in accordance with instructions received from the traffic cop node; and ii) reorganizing itself positionally in the DCS in accordance with instructions received from the traffic cop node.
  • each of the nodes of a DCS of the invention is enabled to perform the self-inserting and the reorganizing simultaneously.
  • each of the nodes is enabled to perform the functions of self-insertion, that is, the insertion of itself into an existing DCS in accordance with instructions received from the traffic cop node; and of reorganizing itself positionally in the DCS with respect to other nodes of the plurality, wherein the reorganization is in accordance with instructions received by that node from the traffic cop node.
  • each of the nodes is enabled to perform the self-insertion and the reorganization of the DCS simultaneously.
  • each of the nodes of a DCS of the invention is enabled to function as both a server and a servant, and in certain respects can further perform these functions simultaneously, and can function to provide content suitable for distribution to one or a plurality of the nodes in the DCS, and which content is distributed by way of the nodes in, over, by way of, or through the DCS.
  • any of the plurality of nodes such as at least one first node and at least one second node, are enabled to function both as a server to provide that content suitable for distribution to one or a plurality of the other nodes in the DCS, and as a servant to receive that content, or other content, which is distributed by way of the nodes in the DCS.
  • a dynamic connection structure (DCS) of the invention comprises a plurality of nodes, all of the nodes being adapted for efficiently routing content between and among the nodes of the DCS.
  • one embodiment of a DCS of the invention is where at least one first node is connected to at least one second node, and wherein the at least one first node and the at least one second node are adapted for efficiently routing content between and among the at least one first and the at least one second node of the DCS, and wherein the DCS forms a peer-to-peer architecture.
  • each of the at least one first node and the at least one second node is capable of simultaneously sending and receiving chunks of a file set over the DCS.
  • each of the at least one first node and the at least one second node creates and maintains an Access Map, the Access Map being useful for, and being used for locating chunks of the file set within the DCS.
  • each of the at least one first node and the at least one second node creates and maintains Neighbor Access Maps, the Neighbor Access Maps being useful for, and being used, for locating chunks of the file set within the DCS.
  • each of the at least one first node and the at least one second node creates and maintains a Request Map, the Request Map being useful for, and being used, to determine which chunks of the file set are needed by each the creating and maintaining node.
  • the at least one first node and the at least one second node create and maintain Neighbor Request Maps, the Neighbor Request Maps being useful for, and being used, to determine which chunks of the file set are needed by each of the creating and maintaining nodes, that is, the nodes which create and maintain the Request Maps.
  • a node In addition to generating its own Access Map and Request Map, a node generates an Access Map and a Request Map for each node (neighbor/subnode/peer) to which it is directly connected. These neighbor/subnode/peer Access Maps and Request Maps are communicated as appropriate to other nodes (neighbors/subnodes/peers) to which the node is connected. This is done in order to propagate access information and requests for chunks of a file farther than just the directly connected nodes. These Access Maps and Request Maps for neighbors/subnode/peers are called Neighbor Access Maps and Neighbor Request Maps.
  • each of the at least one first node and the at least one second node propagates its Access Map to nodes to wliich it is directly connected, that is, to its Neighbor Nodes, and each of the at least one first node and the at least one second node propagates its Request Map to nodes to which it is directly connected, that is, to its Neighbor Nodes.
  • each node which is enabled to function within a DCS of the invention is provided with a plurality of sockets over which chunks of a file set are communicated.
  • each of the at least one first node and the at least second node has a plurality of sockets over which chunks of a file set are communicated.
  • each of the sockets is enabled to operate simultaneously with each other socket, that is, each socket of each of the at least one first node and the at least one second node operates simultaneously with each other socket of each of the at least one first node and the at least one second node.
  • a node in a DCS of the invention is enabled such that, as messages arrive on one or more sockets of the respective nodes, the messages are added to a message queue, and the messages can -include one or a plurality of Access Maps, Request Maps and chunks of a file set forwarded from other nodes.
  • each node of a DCS is provided with at least one message handling thread which repeatedly removes messages from the message queue and processes those messages.
  • each node of the at least one first node and the at least one second node has a message handling thread which repeatedly removes messages from the message queue and processes the messages.
  • each Access Map is transmitted as a compressed table of chunk ranges and each Access Map is transmitted as changes to a last Access Map.
  • each Request Map preferably is transmitted as a compressed table of chunk ranges and is transmitted as changes to a last Request Map.
  • each node preferably regenerates and transmits its Access Map at a predetermined interval or whenever a change occurs, updated Access Maps are transmitted.
  • each node regenerates and transmits its Request Map at a predetermined interval, or whenever a change occurs to the Request Map, updated Request Maps are transmitted.
  • an Access Map is maintained in a storage system of each of the at least one first node and of the at least one second node as a table, and a size of the table is equal to a number of chunks in the file set.
  • the size of the table is equal to, or directly related to, the number of chunks in the file set.
  • a Request Map is maintained in a storage system of each of the at least one first node and of the at least one second node as a table, and a size of the table is equal to a number of chunks in the file set.
  • the size of the table is equal to, or directly related to, the number of chunks in the file set.
  • a storage system is generally defined as any device in or on which information can be kept, where the term kept includes placing the information in or on the device and updating or maintaining such information as required.
  • a storage system includes anything on or in which data/information/content can be stored. This would include memory and storage means such as hard drives, any of the forms of RAM (SRAM, DRAM, PCRAM, CAM, etc.) as well as any form of removable media such as floppies, magnetic tapes, CDs, flash memory, sticks, etc.
  • each Access Map is uncompressed by a receiving node upon receipt by that node, wherein the receiving node is one of the at least one first node and the at least one second node to which a transmitting node is directly connected, and wherein the transmitting node is one of the at least one first node and the plurality of second nodes.
  • each Request Map is uncompressed by a receiving node upon receipt, wherein the receiving node is one of the at least one first node and the at least one second node to which a transmitting node is directly connected.
  • each entry of the Access Map table indicates whether there is access to a particular chunk of the file set and, if there is access to a particular chunk of the file set, the distance in hops to that particular chunk of the file set.
  • each entry of the Request Map table indicates whether there is a need for a particular chunk of the file set.
  • updated Access Maps and updated Request Maps are each generated by each node of the at least one first node and the at least one second node at periodic intervals, where the periodic intervals are predetermined intervals.
  • the invention provides a method for enabling nodes to generate one or more Access Maps.
  • an Access Map generating node can generate an Access Map to be transmitted from that node to a node directly connected to the generating node.
  • the method comprises the steps of 1) initializing all entries in the Access Map table, where the entries each represent a chunk of a file set, 2) determining which, if chunks of the file set are available on the generating node; 3) re-setting any entries in the Access Map table corresponding to chunks of the file set maintained on the generating node; 4) determining, using Access Maps of other nodes maintained on the generating node, which, if any, chunks of the file set are available on those others nodes, and also the identity of nodes on which the chunks of the file set are available, and a distance from the generating node, and 5) re-setting any entries in the Access Map table to a number corresponding to the distance from the generating node.
  • the distance is a
  • the present methods include also a method for generating a Request Map table to be transmitted from a node generating the Request Map table to a node directly connected to the generating node.
  • a Request Map generating node can generate a Request Map to be transmitted from that node to a node directly connected to the generating node.
  • the method comprises the steps of a) validating all incoming Request Map tables; b) using an Access Map table for the generating node, setting each Request Map table entry to a first predetermined value for each chunk of a file set that is maintained on the generating node; and c) using an Access Map table for the generating node, setting each Request Map table to a second predetermined value for each chunk of a file set that is not maintained on the generating node.
  • a method for requesting a chunk of a file set comprises the steps of i) verifying pending requests for chunks of the file set; ii) determining if any new requests for chunks of the file set overlap pending requests for chunks of the file set; and iii) forwarding an intersection of the pending requests and the new requests if there is an overlap.
  • This method for requesting a chunk of a file set may also include the step of iv) one of forwarding new requests for chunks of the file set to a first node that can be reached in a minimum number of hops and forwarding new requests for chunks of the file set to a second node that has a most needed chunk of the file set.
  • This embodiment includes also the optional step of v) removing requests when one of all chunks of the file set in the request have been received and all nodes contributing to the request have been disconnected.
  • the present invention includes also methods for transmitting a file set over, across or through a Dynamic Connection Structure (DCS) from one or more nodes to one or more other nodes.
  • DCS Dynamic Connection Structure
  • This method comprises the steps of A) determining a size and a number of chunks of the file set based on a size of the file set; B) generating an Access Map for a first node of the DCS based on an Access Map forwarded to the first node by at least one second node of the DCS upon establishment of a connection between the first node of the DCS and the at least one second node of the DCS; C) generating a Request Map for the first node of the DCS based on the first node's Access Map; and D) requesting at least one chunk of the file set from the at least one second node of the DCS based on one of requesting a most needed chunk of the file set first and requesting a closest chunk of the file set first.
  • This embodiment may include also the further steps of E) updating the first node's Access Map; F) forwarding the first node's updated Access Map to the at least one second node; G) updating the first node's Request -Map; and H) forwarding requests for additional chunks of the file set based on the first node's updated Request Map.
  • Updates to the various Maps of the invention can be effected in any manner which contributes to the efficiency of the performance of the DCS. For example, updates can be provided to the first node's Access Map and the first node's Request Map upon the occurrence of one or more events.
  • Such events include the establishment of a new connection between the first node of the DCS and another node of the DCS such as a connection to an additional second node, the disconnection of one of the second nodes from the DCS, the reception of the requested chunk by the first node of the DCS, the response by the first node to a successful receipt of the chunk of the file set, reception of a requested chunk of the file set, the reception by the first node of the DCS of the updated Request Map from the second node of the DCS, the reception by the first node of the DCS of the updated Access Map from the second node of the DCS.
  • updates are made to the first node's Access Map and the first node's Request Map at one or more predetermined intervals.
  • the present invention provides also for the validation of Request Maps. Validation can be provided by any means or method which contributes to the operational efficiency of the DCS.
  • one method to validate a Request Map forwarded to a first node of a Dynamic Connection Structure (DCS) by at least one second node of the DCS includes the steps of: setting a request to null and setting a forwarding target to null if the request is empty; determining if a chunk of a file set which fulfills the request is held locally; determining if forwarding target is not null; determining if an overlapping request exists and forwarding the request to the same node for which there is an overlapping request; and forwarding a non-overlapping request to a second node of the DCS based on one of requesting a most needed chunk of a file set first and requesting a closest chunk of the file set first.
  • the present invention provides also for the generation of Access Maps which show the inventory of chunks of the file set in neighboring nodes.
  • Generation of such maps can be provided by any means or method which contributes to the operational efficiency of the DCS.
  • one method for generating an Access Map by a first node of a Dynamic Connection Structure (DCS) for a second node of the DCS comprises the steps of: initializing each entry of the Access Map for the second node of the DCS; resetting each entry of the Access Map that corresponds to a chunk of a file set that is maintained locally; and setting each entry of the Access Map that corresponds to a distance based on a number of hops from the first node.
  • DCS Dynamic Connection Structure
  • the present invention provides also for the generation of Request Maps which show the chunks of the file set needed by neighboring nodes.
  • the methods of the present invention include a method for generating a Request Map by a first node of a Dynamic Connection Structure (DCS) for at least one second node of the DCS, the method comprising the steps of 1) initializing each entry of the Request Map for the at least one second node of the DCS; 2) resetting each entry of the Request Map to indicate whether a chunk of a file set is needed by the at least one second node; 3) determining if the Request Map for the at least one second node of the DCS has overlapping requests with the Request Map of the first node of the DCS; 4) updating the Request Map accordingly to produce a Resulting Request Map; and 5) forwarding the resulting Request Map to a target node.
  • DCS Dynamic Connection Structure
  • the invention uses two Java programs, a j-server and a j-client program.
  • Any reference to a j-server implies an executing instance of the j- server program.
  • Any reference to a j-client implies an executing instance of the j-client program.
  • Any reference to a root j-client implies an executing instance of the j-client program configured to be a root.
  • Any reference to a general j-client implies an executing instance of the j-client program not configured to be a root.
  • a user of a computer system makes a decision to use tlie DCS of H e present invention and the corresponding routing scheme of the present invention. The user needs to have special software to use the present invention.
  • This special software may be downloaded from a website, such as a multi-media vendor, or may be distributed free on a CD when a video, e.g., DVD, or music CD is purchased, or by any other convenient means. Such means include a display in a multi-media vendor where the software may have a nominal cost attached thereto, which upon installation and registration, is rebated or credited back towards purchases.
  • the node software to operate the present invention will be hereinafter referred to as Network Interface Software (NIS).
  • NIS Network Interface Software
  • the present invention comprises two major components - the DCS topology and the routing scheme. While these two components may each be able to be used with other corresponding components, the present invention works best when they are used together.
  • the DCS can be a mesh or graph, a binary heap, a ring, a chain, a bus or of any desired topology.
  • Salient features include wherein the structure is dynamic, and thus changes when a node disconnects/connects from the topology for any reason, for example, when a download is complete, failure of the node's link to the DCS, or when requesting a service or product.
  • the DCS is centrally organized and centrally re-organized whenever an event occurs such as a node connecting/disconnecting.
  • a DCS of the invention is a peer-to-peer network of nodes in communication with each other and with a traffic cop node, which organizes and maintains (re-organizes) the topology of the network as needed or as required.
  • the underlying network may be the Internet, a cable network, fiber, hybrid-fiber- coax, wireless satellite communications or any other technology capable of supporting topologies of the DCS, and the many possible routing schemes described herein.
  • nodes may be mobile. That is, a node may be in a moving vehicle, such as a car, bicycle, RV, ship, boat, laptop computer, train, airplane, or handheld device.
  • a node can be a servant in which case it requests content from the network via the routing scheme and the associated Request Maps and Access Maps.
  • a servant is a client and server which perform tasks normally associated with both clients and servers.
  • a node can also be a subnode (neighbor or peer) to another node.
  • a node can be a root server (server/seed node or seed server). In this case the node is the provider of content to the network.
  • a seed server is a central place for getting specific content. Its only job is to deliver the data that is requested by the servants.
  • a client is a node that requests services or data of another node. That is, a client is a node that receives shared services and data from a server.
  • a peer, or peer node is any of the devices in a layered communications network that operate on the same protocol level.
  • a peer is any functional unit that is in the same layer as another entity.
  • a peer is a corresponding node or entity.
  • a peer may manifest in the form of a processor, a process or a device.
  • a peer may be anything with a digital heartbeat, including sensors, servers, PC's, manufacturing and medical equipment, phones and cellular phones.
  • the peer In order to interact with other peers (e.g. to form or join peer groups), the peer needs to be connected to some kind of network (wired or wireless), such as IP, Bluetooth cable, or Havi, among others.
  • peer entities are entities in the same or different open systems that are in the same layer, where a layer is a network protocol layer or level.
  • a peer node is a node that is a member of the same peer group as a given node.
  • a peer group is a set of logical nodes, which are grouped for purposes of creating a routing hierarchy.
  • Peer protocol is the set of rules defining the procedures for communication between like entities. The identical entities may be devices or software modules at specific layers in an architecture implementation. Peer-to-peer communications pertains to data communication between two nodes that have equal status in the interchange, where either node can begin the exchange and where equal status means nodes that are in the same protocol layer or level.
  • a traffic cop is a software component that manages the topology of the DCS.
  • the primary functions of a traffic cop are to provide information to servants who wish to join the DCS, and to adjust the topology of the DCS when servants leave.
  • the traffic cop also helps a servant to determine if that servant is behind a firewall or not.
  • the traffic cop (or j-server) is a software program used to: 1. Organize dynamic connection structures ("DCS ' s"); 2. Reorganize dynamic connection structures; and 3. Communicate organizational changes to affected nodes in the DCS.
  • DCS ' s Organize dynamic connection structures
  • To organize a Dynamic Connection Structure the traffic cop utilizes algorithms that allow it to intelligently organize a DCS. Some examples of organizational determinants include: 1. Is the client high-speed?
  • ISP connection/identity information can be obtained, for example, via lookup in an ISP database by Internet Protocol address. Preferably, clients having the same ISP are connected to one another. 5. What is the client's geographic location? This information can be obtained, for example, via lookup in a geographic database by Internet Protocol address. Connect the client to other clients that are in nearby geographic locations.
  • the traffic cop is adapted and configured to communicate organizational changes to affected nodes.
  • organizational changes are made in response to an action/event that is transmitted to the traffic cop.
  • the client notifying the traffic cop of the action/event will receive the instruction message(s) from the traffic cop, make connections that apply to itself, and propagate the message to other nodes that are affected by the instructions.
  • the other nodes will similarly make connections that apply to them, and propagate the message farther if necessary.
  • Examples of actions/events to which the Traffic Cop responds include: 1. Client wishes to be added to a specific DCS. 2. Client wishes to be removed froth a specific DCS. 3. Client in a DCS disappeared. 4. Client attributes changed. For example, a client may notify the traffic cop that the client has received the entire file, and the traffic cop may decide to give the client additional connections.
  • a client interprets a Traffic Cop instruction message in accordance with specific criteria, including: 1. Does this message have a valid security signature? If yes, proceed. 2. Does this message contain directions for any of the peers that I am currently connected to? If yes, forward the message to those peers. 3. Does this message dictate connection changes for me? If yes, make new connections as specified and sever existing connections as specified. 4. Does this message contain directions for any of the new peers that I have just connected to? If yes, forward the message to those peers. 5. Does this message contain any additional instructions for me (called non-topology messages)? If so, handle them appropriately.
  • the invention provides means and methods for assembling a DCS comprising a plurality of nodes which can perform the desired file transfer and file re-transfer functions even when not connected to a server or seed node.
  • This partial autonomy is accomplished, in pertinent part, by means of software (NIS) which are provided to each node prior to or as that node is connected to the DCS.
  • NIS software
  • the following acts or functions are enabled in a node possessing the NIS:
  • a dynamic connection structure of clients is established comprising all clients involved in the file transfer and their connections to each other.
  • This dynamic connection structure is generated using an incomplete binary heap algorithm (defined below) as follows. For each client added to the dynamic connection structure, a node is added to the heap. For each client removed from the dynamic connection structure, the node corresponding to that client is removed from the heap. There is always a direct mapping between nodes in the heap and clients in the dynamic connection structure.
  • Nodes in a binary heap can have a parent and up to two children. Henceforth, any reference to a given client's parent implies the client which corresponds to the parent node of the node which corresponds to the given client.
  • Node A in the heap corresponds to Client A
  • Node B corresponds to Client B
  • Node A is the parent of Node B
  • Client A may be referred to as the parent of Client B.
  • the server program is responsible for creating and maintaining the heap.
  • the client contacts the server and requests addition.
  • the server then adds a node representing the client to the heap using the incomplete binary heap addition algorithm, and sends the client an encoded representation of its corresponding resultant subheap.
  • the client then inserts itself into the dynamic connection structure by contacting necessary clients, providing them with connection information derived from the server's message, and cooperatively establishing the specified connections.
  • a removal from the dynamic connection structure is accomplished similarly.
  • the parent of the client contacts the server, the server removes the corresponding node from the heap, and sends information which includes a description of the new subheap to the parent.
  • the parent then makes the necessary communications to enable the dynamic connection structure to reorganize itself in a way which contains no connections to the removed node.
  • the server program facilitates the file transfer by organizing and specifying appropriate alterations to the connections over wliich the packets are transferred.
  • the root client is responsible for dividing the file into packets, and these packets are passed sequentially over the open connections in the dynamic connection structure. Packets are always passed from a parent client to a child client. The root client passes a packet to its child clients. Each of these clients passes the packet to their child clients, and the process continues until every client has received the packet. Packets are transferred between two connected clients without intervention. In other words, if two clients have an open connection, they will always repeatedly transfer packets until disconnected.
  • the present invention represents a dramatic improvement on current file transfer technologies because it allows client programs to send packets to their children while receiving packets from their parents. As soon as a client has received a packet, it is able to receive the next packet. Essentially, every client in the dynamic connection structure can be receiving the file simultaneously.
  • this embodiment employs a modified binary heap data structure as the most appropriate type of binary tree.
  • the dynamic connection structure is considered dynamic because additions and deletions are permitted any time after the heap is initialized and before the heap is destroyed.
  • the embodied heap sorting requirement is that any j-client in the dynamic connection structure must always have a slower network connection speed than its parent. Each j-client' s network connection speed is determined prior to the insertion of a node corresponding to that client in the heap, and is used as the sort key for the heap.
  • the heap properties are modified as follows: the heap is no longer required to satisfy the completeness property; the repositioning of nodes in the heap following an addition or deletion of a node, should never cause a node corresponding to a j-client to be repositioned more than one step away from its previous position.
  • the completeness property of a binary tree requires that every level except the last is full, and that on the last level, all internal nodes are to the left of external nodes. In a standard binary heap, the completeness property must always be satisfied. Following the addition or removal of nodes, reorganization of the heap is often required to maintain the completeness property.
  • the first open position in the heap is specified in the insert algorithm as the leftmost position not occupied by a node in the highest level in the tree which is not full.
  • the upHeap and downHeap algorithms remain the same as in a standard binary heap algorithm. These simplifications ensure that tlie addition or removal from the heap of a node corresponding to a single j-client will never cause a node corresponding to another j-client to be moved further than one step along a path in the tree. Thus, these simplified algorithms satisfy the second requirement that a node corresponding to a j-client is never repositioned more than one step away from its previous position in the heap.
  • the modified binary heap is used to model the dynamic connection structure of j- clients by using a node in the heap to represent a j-client in the dynamic connection structure.
  • Each node in the heap can have a parent and up to two children.
  • each j-client can have a parent and up to two children.
  • the parent of a j-client is a j-client which is responsible for sending data packets to that j-client.
  • the children of a j -client are j -clients to which that j -client will send data packets .
  • the representation of the heap at a given time can be used to determine which network connections for a given j-client should be actively sending and receiving data packets at that time.
  • incomplete binary heap or incomplete binary heap structure or incomplete binary heap data structure will be defined as a data structure possessing the characteristics of a binary heap modified in the manner described above.
  • incomplete binary heap algorithm will be defined as an algorithm possessing the characteristics of a binary heap algorithm modified in the manner described above.
  • incomplete binary heap addition algorithm and incomplete binary heap deletion algorithm will be defined as the addition (insertion) algorithm and the removal (deletion) algorithms described above.
  • a data packet or packet shall be defined as an individual segment of information resulting from the division of a larger set of information into one or more parts.
  • a data packet or packet shall be defined as an individual segment of data bytes resulting from the division of a file.
  • any references to transferring a file at the packet level shall imply dividing the file into packets, and transferring the packets individually.
  • Socket implementations in a programming language are common in the art.
  • a socket is used in a computer program to connect to, receive connections from, send information to, and receive information from other computers. All communications between computers on a network in this embodiment are accomplished using sockets. Other utilities used to transmit data packets from one computer to another, would not fall outside of the scope and spirit of the present invention.
  • any reference to connections or active connections indicates the set of open network connections between a j-client or j-server and another j-client or j- server. Any connection described in this embodiment indicates open TCP socket connection between two programs, which implies open TCP connection between the computers on which the programs are running. If the two programs are running on the same computer, this still holds true.
  • any reference to a program opening a connection, opening a network connection, or connecting to another program implies the following steps:
  • the computer on which the program is running uses a socket to contact a computer on a specified port, and requests that a connection be established.
  • the contacted computer accepts, rejects, or does not respond to the request.
  • connection is considered open and the sockets are considered connected. If the computer to which contact is attempted rejects or does not respond to the request within a given amount of time, the connection is considered closed, and the sockets remain unconnected.
  • a network connection is open between the sender and receiver.
  • the source program transforms the message into data bytes as specified by a protocol.
  • the source program transmits the message over the open connection.
  • a dynamic connection structure of clients or a dynamic connection structure is defined for the purposes of the specification of the present invention as a set of clients and a set of open network connections between these clients, of which the following are true: information is capable of being transmitted across included network connections at any time; the set of network connections may be modified at any time.
  • All programs described herein imply computer programs or processes executing on a computer connected to a network.
  • the specific computers may vary in any way, but they must all be connected to the same network.
  • This network will henceforth be referred to as Network A.
  • a data file may be transmitted from a source computer to a number of other computers over a network using dynamic connection structure means.
  • the methods described pertain to transfer of a specific file, which may or may not be identified prior to initiation of these mechanisms.
  • the file to be transferred will be predetermined, and this file will henceforth be referred to as File A.
  • the definition of binary heap is consistent with the commonly understood definition in the art.
  • the term binary heap or heap designates an incomplete binary heap structure containing one node for each j-client in the dynamic connection structure. The heap provides a model of the dynamic connection structure and is used to determine connection information..
  • Node A in the heap which has " children Node B and Node C, corresponds to j- client A in the dynamic connection structure
  • Node B and Node C in the heap correspond to j-client B and j-client C in the dynamic connection structure, respectively
  • j-client A is responsible for sending data packets to j-client B and j-client C.
  • Information stored in the heap for each j-client is: Unique j-client ID used to identify each j-client in the dynamic connection structure.
  • j-client IP address used to identify each j-client in the dynamic connection structure.
  • j-client IP address j-client IP address.
  • j-client network port base defined below.
  • a client is defined as a computer program capable of connecting to, receiving connections from, sending information, and receiving information from other computer programs.
  • An external client is a client that is not included in a specified dynamic connection structure.
  • a j-client shall be customized to refer to a program executing on a computer connected to Network A that is responsible for receiving packets from a source and sending those packets on to its children, and is also responsible for listening for connection change information, and making the specified connection changes when they are received.
  • the root j-client is a specific type of j-client. This program must be executing on a computer that has direct access to File A. It is preferred that the root j-client is executed on a computer that holds File A on its local disk. Because it is the reference source of File A, the node corresponding to the root j-client is always positioned at the top of the tree that models the dynamic connection structure.
  • the root j-client is responsible for extracting packets from the local copy of File A, and sending them to its children. In this embodiment, only one root j-client will be used. However, additional root j -clients could be added; for instance, to provide failover protection.
  • a root j-client is always assigned a connection speed of 0, which the binary heap sort key recognizes as the fastest defined network connection speed. This ensures that the no general j-client will ever be repositioned such that the root j-client is its child.
  • a general j-client refers to any j -client that is not a root j -client.
  • a server is defined as a computer program which is capable of connecting to, receiving connections from, sending information to, and receiving information from other computer programs, and that performs administrative tasks necessary to the overall implementation.
  • the distinction between a client and a server should not be construed to mean that clients are incapable of performing administrative tasks, the distinction is drawn simply for the purpose of clarifying the specification.
  • An external server is a server that is not included in a specified dynamic connection structure.
  • a j-server or file j-server shall be customized to refer to a program executing on a computer connected to Network A, which may or may not be executing on the same computer as the root j-client, and which is responsible for the following:
  • a j-server program maintains one socket and one network port.
  • a j-server must listen for connections from j-clients, and process the messages received.
  • the j-server listens for these connections on a network port on the computer it runs on. This port is referred to as the j-server port.
  • the socket which facilitates communication on the j-server port is referred to as the j-server socket.
  • a general j-client maintains four sockets and four network ports.
  • the first port is used to listen for connections, and to receive messages regarding connection changes. This port is referred to as the info port.
  • the second is used to connect to a parent j-client, and to receive packets from the parent j-client. This port is referred to as the parent port.
  • the next two are used to receive connections from up to two child j-clients, and to send packets to these child j-clients. These ports are referred to as child ports.
  • the sockets that facilitate communication on the info, parent and child ports are referred to as the info socket, the parent socket, and the child sockets respectively.
  • the four ports used by a j-client will be sequentially assigned starting with a base port number.
  • the info port is assigned the base port
  • the parent port is assigned the base port + 1
  • the child ports are assigned the base port +2 and base port + 3.
  • a j-client program is responsible for finding 4 adjacent open ports prior to its initial contact to the server.
  • connection information or connection information for a client is defined as information which indicates to a client the appropriate connections which the client should establish with other clients and connection information for other clients.
  • the root j-client which acts as the source of File A, is responsible for partitioning the file into one or more manageable chunks, referred to as data packets.
  • the packet size is arbitrary, but should be chosen to be relatively small, so that no message takes excessively long to be decoded by j-clients.
  • Each packet is assigned a packet number (packet num).
  • a new j-client is inserted in the dynamic connection structure, it does not request any packets from its parent until it receives packet requests from all of its children. It then requests the earliest packet required to be transferred to its children. If a j-client is inserted to a position where it has no children, it requests ANYJPACKET from its parent. If the parent has no other child, it will begin transmitting with the earliest packet it possesses. If the parent has another child, it will begin by transmitting the packet that it is currently transmittihg to the other child. This technique insures that child j-clients do not have to wait for an inserted parent to receive packets that they already have. When a j- client is inserted, its children will have to wait a minimal amount of time before they can continue receiving their packets.
  • the root j-client will begin packet transmissions to its first child with packet number 0. If File A contains n packets, the packets are numbered 0 to n- ⁇ . The root j- client will send each packet sequentially as they are requested until packet n- ⁇ is sent. The root j-client will then return to packet 0 and repeat the whole process.
  • a j-client in the dynamic connection structure always receives packets in sequential order, but not necessarily starting with packet 0. After packet n- ⁇ is received by a j-client, the j-client will wrap back around to packet 0, and begin receiving packets. If a j-client starts receiving packets with packet m, it will receive sequentially packets m through n- ⁇ , then 0 through m- ⁇ . The j-client will then transmit a final packet to its children, close connections with its children, and be removed from the dynamic connection structure.
  • This technique insures that a j-client in the dynamic connection structure will never need to request a packet that each of its children already has. In addition, a j-client is able to leave the dynamic connection structure (and disconnect from Network A) almost immediately following receipt of the last packet.
  • the protocol implemented in this embodiment provides a means of communication between j-client and j-server programs.
  • a message transmitted from one program to another in this system is structured as follows:
  • the message ID is a numeric id that is incremented each time a message is sent on a specific connection. This ID is used to insure that messages are not processed more than once, and to ensure that messages do not arrive out of sequence. [00124] In certain situations, a message is used to inform a j-client that its active connections should be changed.
  • This string can be used by any included j-client to determine the manner in which their connections should be changed.
  • the encoded tree is the encoded version of the subheap starting at j-client B.
  • a j-client needs to send new connection information to a new child, that j-client will generate an encoded tree for its child, which is the encoded version of the subheap starting at that child.
  • the REQUEST JNSERT and REQUEST NSERTJROOT message types allow a general j-client or a root j-client to request insertion into the dynamic connection structure.
  • the unique j-client ID is included in each, and it includes the j-client's connection speed and network port base.
  • YOU_ARE_THE_ROOT allows a j-server to notify a root j-client that it has been added to the root of the heap.
  • the message ALREADY_HAVE_ROOT allows a j-server to notify a root j- client that it was not added to the tree, because a root j-client already exists.
  • the INSERTJNFO message is sent from a j-server to a newly added j-client. This message includes the encoded tree of the j-client's new parent, and information about the file being served.
  • the NEW_PARENT_TREE message is used to inform a j-client that it must change its child and parent connections.
  • the encoded tree contains the j-client's new connection information, and new connection information for all j-clients included in the given j-client's representative subheap.
  • the parent port specifies the network port on which to contact the new parent.
  • the message SET_CHILDREN is identical to the NEW_PARENT TREE message, except it implies that only the j-client's children are changing, and does not need to send a parent port.
  • the message REQUEST PACKET allows a child to request a specific packet from its parent. PacketNum is the unique identifier of the packet needed.
  • the message PACKET_HERE is used to send a packet from a parent j-client to a child j-client. It includes the packet number, the packet size, and the data packet as an array of bytes.
  • the message INVALID_PACKET_REQUEST allows a parent j-client to notify a child j-client that it does not have the requested packet.
  • the message INVALID_PACKET_REQUEST BYE is identical to the message
  • INVALID_PACKET_REQUEST except it implies that the connection is being closed after acknowledgement of its receipt.
  • the message DELETE CHELD allows a j-client to notify a j-server that its child j-client requires deletion from the connection structure.
  • the message DELETEJNFO is sent from a j-server to a j-client which requested deletion of a j-client.
  • This message includes a new encoded tree for the j-client.
  • the message OK_BYE is used to acknowledge receipt of a message, and to notify the recipient that the connection is being closed.
  • a j-client contacts a j-server to request addition to the dynamic connection structure
  • J-client uses info port
  • j-server uses j-server port
  • J-client sends REQUESTJNSERT or REQUEST_INSERT_ROOT.
  • J-server responds YOU_ARE_THE_ROOT, ALREADY_HAVE_ROOT, or
  • a j-client contacts a j-server to request that one of its child j-clients be removed from the dynamic connection structure
  • J-client uses info port
  • j-server uses j-server port
  • J-client sends DELETE_CHILD.
  • J-server responds DELETE_INFO.
  • J-client A transmits information about changing connections to J-client B
  • J-client A uses info port or parent port
  • J-client B uses info port
  • J-client A sends SET CHILDREN, or NEW_PARENT_TREE.
  • J-client B responds OK_BYE or CHILD_INFO_RECIEVED.
  • J-client A attempts data packets transmission to J-client B
  • J-client A uses a child port
  • J-client B uses parent port
  • J-cliert B sends REQUEST_PACKET.
  • J-server responds PACKET_HERE, PACKET_HERE_BYE, INVALID JPACKET JREQUEST, or INVALID_PACKET_REQUEST BYE.
  • This protocol is obviously just one example of a protocol that could be used to transmit data packets and to facilitate changes to the dynamic connection structure described herein. Any other protocol used to facilitate similar changes to a dynamic connection structure would not fall outside the scope of the present invention.
  • the j-server program does the following when executed:
  • the j-server program adds a node corresponding to the j-client to the heap in its memory, generates an encoded tree for the j- client's highest level ancestor for which connections change, and sends an appropriate reply.
  • the j-server program removes the nod corresponding to the specified j-client from the heap in its memory, generates an encoded tree for the j-client, and sends an appropriate reply.
  • the j-client program does the following when executed:
  • Generate a unique j-client ID which includes IP address, connection speed, base port, and a timestamp. Use the local info port to connect to the remote j-server port and request addition to the tree.
  • any of the above steps refers to processing a message, it implies the following: If the message contains a data packet, the data packet is saved to disk. If the message contains connection information, this connection information is saved to memory. If the message contains a BYE, the connection is immediately closed. [00148]
  • Each of the steps from 10 to 18 are sequence independent. In other works, any step can be run at any time, and it will not disturb the integrity of the process.
  • the j-server program is executed on a computer connected to Network A.
  • a j- client program is executed on a computer connected to Network A and with local access to File A, and it is configured to run as the root j-client.
  • This program contacts the j-server, and requests addition to the dynamic connection structure, as the root j-client.
  • the j-server adds the root j-client to the heap, and replies, signifying that the addition was successful, and closes the connection.
  • Any number of j-client programs are then executed any number of times on computers connected to Network A. Each time a j-client program is executed, it contacts the j-server and requests addition to the dynamic connection structure.
  • the j-server adds a node corresponding to the j-client to the heap, which results in the repositioning of a number of nodes along a path in the heap.
  • the j-server replies, signifying that the addition was successful, and sends to the added j-client an encoded tree relating to the j-client at the highest level in the tree whose connections are affected by the addition (the ancestor).
  • the j-client contacts this ancestor, sends the new connection information generated by the j- server, and closes the connection.
  • the ancestor modifies its own connection information, by closing appropriate connections and contacting new children. It sends its new children their new connection information, which is extracted from the encoded tree, and then closes the appropriate connections. This process repeats; each j-client whose connections are affected closes the appropriate connections, contacts its new parents if necessary, and sends new connection information to its children.
  • a j-client receives a message from a child signifying that the child has received all packets in File A, or if a child or parent does not communicate for a certain amount of time (crashed or hung), the hung j-client is removed from the tree. To accomplish this, the j-client formerly connected to the hung j-client contacts the j-server and indicates to the j- server which j-client must be deleted. The j-server deletes the node corresponding to the j- client requiring deletion from the heap, which results in repositioning of a number of nodes along a path in the heap.
  • the j-server replies, signifying that the deletion was successful, and sends to the j-client a new encoded tree. Similarly to j-client addition, new connections are established as the information is passed down the heap. [00153] While all of this is happening, all j-clients that have open parent and child connections are sending and receiving data packets of File A. [00154] Because changes to the heap in the j-server' s memory do not immediately take effect (they are passed down the heap over time), the actual dynamic connection structure does not always exactly match the heap representation in the j-server's memory. In addition, because many additions and deletions may be performed by the j-server sequentially, and these changes are constantly filtering down the heap, the actual dynamic connection structure may never exactly match that of the heap.
  • the heap is thus considered to always represent an ever-changing goal state that the dynamic connection structure is working towards.
  • the j-server affixes a timestamp to every encoded tree it generates.
  • the newly generated encoded tree is assigned the same timestamp as the encoded free which was used to generate it. If a j -client ever receives connection information that includes a timestamp which is earlier than the timestamp on the last applied connection changes, the information is ignored.
  • FIGs. 11(a) through 11(d) Node N requests authorization to enter/connect to the DCS for purposes of file transfer. In this case as will be described below the DCS is a mesh.
  • node/peer/client N connects to traffic cop S.
  • peer N requests access to a file.
  • Traffic cop S then grants access and forwards access information, file information, a signature and a connection list to peer N as shown in Fig. 11(c). Peer N then disconnects from traffic cop S as shown in Fig. 11(d).
  • peer N inserts itself into the DCS by connecting to one or more peers that are already in the DCS. Peer N forwards access information and connection list information that was received from traffic cop S. As shown in Fig. 12(a), peer N connects to peers A and B. Connection list information is then sent to all affected peers. Affected peers are peers that must change their connections as a result of the newly established connection with peer N.
  • FIG. 12(b) show peer B forwarding connection list information to peer G because that connection will be broken. Also shown is peer A forwarding connection list information to peer R because that connection will also be broken as a result of the entry of peer N into the DCS.
  • Fig. 12(c) shows the breaking or disconnection of peer R from peer A and peer G from peer B.
  • Fig. 12(d) shows the formation of a new connection between peer R and peer G as a result of peer N entering the DCS. The information for the new connection was sent to peers R and G along with the disconnection information. All of the above access information and connection lists are further described below in the description of the Packet Chain Protocol and the object schemas. [00157] With respect to a smaller DCS, Fig.
  • FIG. 13(a) illustrates peer A notifying each neighbor by sending Access Maps of all chunks of the file that peer A has and what chunks of the file that it and its neighbors need.
  • peers B, C, D and E request chunks of the file that they and their neighbors need from peer A via Request Maps sent to peer A.
  • Peer A requests chunks from its neighbors (not shown) and forward chunks of the file that it has that are needed by peers B, C, D and E to them.
  • peer C might have neighbors Cl and C2 (not shown) and peer C forwards a Request Map to peer A that includes the chunks that peers Cl and C2 need as well.
  • peer A if peer A does not have any chunk of the file that peer B needs then peer A notifies peer C, for example, of chunks of the file that both peer A and peer B need via updated Request Maps.
  • peer N Upon the completion of a file transfer, peer N connects to traffic cop S and notifies traffic cop S that the file transfer is complete and that peer N wishes to disconnect from the DCS. Traffic cop S then sends connection list information to peer N. Peer N then disconnects from traffic cop S. Connection list information is sent to all affected peers/nodes that must change their connection because peer N wishes to disconnect from the DCS. Connections that are not needed are broken including all of peer N's connections. Peer N exits the DCS and any new connections required as a result of peer N leaving the DCS are formed.
  • Fig. 1 illustrates an exemplary DCS having a mesh topology, which is an extension of a binary heap. That is, in a mesh topology, a node may have any number of children (subnodes, called “children” or “child nodes” in a binary heap) to which it is connected, and is not limited to two children nor is there any limit on completeness. In fact, the number ofsubnodes and connections may vary from node to node.
  • This parent— >child relationship as described above also includes parent ⁇ child, making the relationship more equal.
  • a mesh or binary tree DCS may be organized and re-organized by the traffic cop node according to node attributes such as network connection speed, firewall status, geographic location, the IP address in the sub-network, where the parts (chunks) of the files are by sub-network class, user account information (e.g. most money in account, greatest number of shared files, spending habits, always online etc.). That is, the nodes of the DCS are organized to maximize efficiency using each node's available upstream as well as downstream capacity.
  • the above attributes may also be useful to the traffic cop node in the creation of a connection list, which will be described in greater detail herein below.
  • the traffic cop node (also called j-server) is a computer executing specialized software, which is designed, used and formulated to fulfill three primary functions: 1) organize dynamic connection structures; 2) reorganize dynamic connection structures, and 3) communicate organizational changes to affected nodes in a DCS.
  • a traffic cop node utilizes software algorithms that allow it to intelligently organize dynamic connection structures in accordance with certain criteria, including the speed of the node, the node's firewall status, and the node's geographic location. If the node is high-speed, then the node is given a higher number of connections. If the node is low-speed, then the node is given a lower number of connections.
  • the traffic cop will use Internet Service Provider (ISP) and geographic information to optimally manage meshes.
  • the traffic cop node will maintain three attributes for each node: ISP, state, and city. The order of priority will be ISP, state, city.
  • the ISP for a node is determined by looking in a database of ISPs by Internet Protocol Address and nodes are connected to each other, if possible and to the extent possible, through the same ISP.
  • the geographic location of a node is determined by looking in a database of ISPs by Internet Protocol Address and nodes are connected to each other, if possible and to the extent possible, in nearby geographic areas. For example, two nodes using the same ISP in different cities will be connected before two nodes using different ISPs in the same city.
  • one or more messages are constructed which contain topology instructions. These instructions can be interpreted by the affected nodes (by special node instruction software (NIS) installed on the nodes' computers), and these nodes will carry out the instructions communicated by the traffic cop node.
  • NIS special node instruction software
  • organizational changes are made in response to an action/event that is transmitted to the traffic cop node.
  • the node notifying the traffic cop node of the action/event, will receive the instruction message(s) from the traffic cop node, make connections that apply to itself, propagate the message to other nodes that are affected by the instructions.
  • the other nodes will similarly make connections that apply to them, and propagate the message farther, if necessary.
  • Actions/events communicated to the traffic cop node by a node of the DCS include a node that wants to be added to a DCS, a node that wants to be removed (deleted) from a DCS, a node that is reported by another node to be missing (disappeared) or non- responsive (lost a connection) from a DCS and attributes of a node that have changed.
  • the node has received the entire file so the traffic cop node may give that node additional connections so that the file may be disseminated more quickly and efficiently.
  • a node Upon receiving a topology instruction message from the traffic cop node, a node will verify that the message has a valid security signature. If the security signature is invalid the message will be ignored. If the security signature is valid then the node will establish new connections and break/sever existing connections as specified in the topology instruction message. If any of the directions in the message pertain to any peers to which the node receiving the topology instruction message is connected then the topology instruction message is forwarded/communicated to the affected peers. If the topology instruction message contains any direction for new peers with which the node receiving the topology instruction message was instructed to establish a connection, then the topology instruction message is forwarded/communicated to the new peers. If the message contains any non-topology instructions then the non-topology instructions are handled appropriately.
  • Non-topology messages include an invitation/directive to join another DCS, a chat message, a configuration message or a promotional message/advertisement.
  • a node might be invited to join another DCS because the node is needed as a file upload source in the other DCS or the other DCS is transferring content in which the node may be interested or the other DCS may be a chat group in which the node may be interested.
  • a DCS can be used for more than file transfers.
  • a DCS can be used in any scenario that resources of nodes are needed.
  • the configuration/topology of the DCS is organized specially depending on the types of resources needed. For example, a DCS can be used to facilitate message delivery into computers/nodes that are behind a firewall.
  • a DCS would be structured as illustrated in Fig.2.
  • a DCS may be created for a group of nodes that wish to chat together, and chat messages can be routed between these nodes.
  • a DCS may be created for a group of nodes and information passed among the nodes in order to utilize the CPU processing power of the nodes. This is called grid/utility computing.
  • a DCS may be created for a group of nodes and information passed among the nodes in order to utilize the aggregate storage (e.g., hard-drive) power of the nodes. This is called distributed storage.
  • aggregate storage e.g., hard-drive
  • a ratio of 1 indicates sufficient upstream bandwidth within the mesh to meet the desired quality of service for all the downloaders. Nonetheless, it makes sense to provide higher capacity, however, so that if any of the nodes fail, or if any of the nodes need to be pulled into another mesh, the current mesh DCS will not suffer.
  • the traffic cop node calculates the balance ratio of each mesh periodically. Periodically may be as frequently as each time an addition or deletion to the DCS is made, and each time a node reports itself completed. When the balance ratio of the DCS increases above a certain threshold, completed nodes are removed. When the balance ratio of the DCS decreases below a certain threshold, new upload sources are added to the mesh. Balance ratio thresholds are set using an algorithm in the traffic cop node. A range for the balance ration might be a minimum of 1.20 and a maximum might be 1.40. More accurate balance ratio calculations can be made by summing the average bandwidths reported by each node rather than using generic averages for each speed group. [00169] When the mesh balance ratio exceeds the upper threshold, nodes will be removed.
  • a popularity index for each node could also be used.
  • the popularity index is calculated as the sum of the popularity indices of each piece of content that node had shared.
  • a popularity index for a piece of content would be proportional to the number of nodes sharing it.
  • the traffic cop communicates with nodes via mesh specific CCIL messages (described in greater detail below), XML documents specifying lists of connections that need to be established/broken within that mesh.
  • CCIL messages allow the traffic cop to relay instructions to any number of nodes within a mesh regarding how their connections should change.
  • On-demand meshes are supported by performing the following steps: Every node, when logging on to the system, will automatically insert itself into a holding mesh (see Fig.3).
  • the holding mesh is a non-transfer mesh (no content is being transferred within it), and it exists only to provide message propagation routes to nodes. In this way, messages can be communicated to a node when it is time to transfer a node into a mesh/DCS.
  • Non-firewalls will be given connections to firewalls as buddies are needed, but many non- firewalls will not need to be given connections in the holding mesh, (since the non-firewall nodes can be reached directly with future messages)
  • an insert message is embedded in a holding mesh CCEL.
  • the traffic cop node can build this CCIL message just like any existing topology message in the holding mesh, but the insert message is embedded in the XML section intended for the target node.
  • a DCS is initiated when a node requests access to a resource, e.g. file, stream, data etc., for which no DCS currently exists.
  • a node using NIS and content provider administration tools is directed to an appropriate traffic cop node of the plurality of traffic cop nodes.
  • the content provider administration tool uses and Content Management Database (CMDB) to determine which traffic cop node is appropriate. If no traffic cop node exists then a new traffic cop node is cloned to handle the topology of the new DCS.
  • the traffic cop node makes a determination that it needs to start a new DCS and insert the requesting node into the new DCS.
  • CMDB Content Management Database
  • the traffic cop node In the case of a file transfer DCS, the traffic cop node also determines that one or more servers/root nodes/seed nodes are also needed in the new DCS.
  • the traffic cop node has access to the CMDB wliich tracks servers and their locations for all known/available content.
  • the traffic cop node communicates with the CMDB and requests new seed nodes/servers whenever necessary. Seeds (content) can be a web/http/ftp server which will be used to seed data content and provide data to the mesh. In this case, seed nodes are optional. Seed nodes/servers can be other nodes. In this case, the nodes need to be invited (directed to join) the new DCS (mesh) in order to upload the requested file. DCSs frequently overlap.
  • a list of sources is obtained via communication with the Content Management Database (CMDB) by specifying the content desired by ID and the number of sources (nodes) having that content. It is desirable to obtain many more potential sources than are actually needed, so that an optimal set can be chosen from those returned. For example, if three new sources are needed in a mesh, a query to the CMDB might request thirty candidates having the specified content be returned, so that the three best candidates/nodes can be added to the mesh. A request to the CMDB for even more candidates could be made so that a list is available in the traffic cop node and a subsequent query to the CMDB need not be made.
  • CMDB Content Management Database
  • the chosen source is in less than the maximum number of meshes (e.g., 5) then add the source/node to the current mesh, otherwise:* Determine which of the node's current upload-only meshes has the highest balance ratio, and determine what that mesh's balance ratio would be if the node were removed.
  • step 1 If the desired number of sources has not been able to be located, then obtain another (non- overlapping) set of sources from the CMDB and go back to step 1.
  • step 2a Repeat the entire above process, but ignoring the maximum mesh requirement of step 2a.
  • step 2a Extend step 2a to balance out nodes that are in high demand by adding part iii as follows:
  • the Topology Service (resident in the traffic cop node) is responsible for processing all connection change Information (CCI) Lists (CCILists) as they arrive. It will operate in its own thread, and it will run a repeating loop, which waits for a CCIMessage to arrive, processes it once it arrives, checks firewall timeouts, and repeats itself. [00175] The Topology Service must maintain the following values in memory: Last Processed CCIList - The last valid CCIList message processed. Connection List - A list of all active transmission connections. Because more than one content transmission can be active at a time, the list is grouped by Universal Resource Node (URN). For example:
  • Connection n URNConnectionList 1 URN 1 - identifier of the download 1 Connection 0 Connection 1
  • Validate the timestamp by checking if it is greater than lastProcessedTimestamp. If it is not, perform the Connection Cleanup Algorithm and dispose of the message. Validate the signature using the RSN public/private key. If it fails, dispose of the message.
  • connection list If the node is specified as outgoing, initiate a direct connection, and add it to the connection list.
  • connection code itself.
  • the topology manager assumes that all connections are successful, because unsuccessful connections will trigger other events (deletions) which will lead to a new CCI object being received.
  • Fig. 4 is a topology .. service flowchart depicting the creation of a connection between a traffic cop node and a node specifically highlighting the communications between the traffic cop node and the node from the perspective of the traffic cop node.
  • the traffic cop node receives a connection request from a node.
  • the traffic cop node responds to the node at step 210 by sending the node an acknowledgment of the connection request.
  • a connection is established at step 215.
  • the traffic cop node then receives an access request from the node as well as the node's access information, such as speed, upstream and downstream bandwidth, id, speed, subnet classification, firewall status etc. at step 220.
  • the traffic cop node validates the received access information at step 225 and grants or denies access to the node at step 230.
  • the traffic cop node sends the node access information for the node's connection list including when the node becomes a peer or neighbor node.
  • the above flowchart assumes a grant of access. If access is denied then the connection will be deleted.
  • Fig. 5 is a topology service flowchart depicting the creation of a connection between a traffic cop node and a node specifically highlighting the communications between the traffic cop node and the node from the perspective of the node.
  • the node sends a connection request to the traffic cop node and then receives an acknowledgment of the connection request from the traffic cop node at step 310.
  • a connection is established at step 315.
  • the node requests access to a file set and provides its access information, such as id, speed, bandwidth, firewall status, subnet classification etc. to the traffic cop node at step 320.
  • the traffic cop node then grants or denies access upon validation of the node's access information. Based on a grant of access, the node receives access at step 325.
  • the above flowchart assumes a grant of access. If access is denied then the connection will be deleted.
  • Fig. 6 is a topology service flowchart of the creation of a connection between a node and a peer node from the perspective of the peer node.
  • a node and' a peer node are both nodes. The terminology is used merely to distinguish between the two nodes. A "first node” and a “second node” could just as easily serve to distinguish between Hie -two nodes.
  • the peer node receives a connection request from a node and responds at step 410 with an acknowledgment of the connection request.
  • a connection is established at step 415.
  • the peer node receives the node's access information, such as id, speed, bandwidth, firewall status, subnet classification etc. at 420.
  • the peer node validates the node's access information at step 425 and based on validation and a grant of access, propagates the access information and connection list to its peer nodes at step 430. Connections to the peer node's peer nodes are established or broken as needed based on business rules and the establishment of a connection between a node and this peer node. The above flowchart assumes a grant of access. If access is denied then the connection will be deleted.
  • Fig.7 is a topology service flowchart of the creation of a connection between a node and a peer node from the perspective of the node.
  • a request is made to establish a connection and a response is then received acknowledging receipt of the connection request at step 510.
  • a connection is established at step 515.
  • a determination of the chunk size of the file set is determined at step 520 and an Access Map is generated at step 525.
  • a Request Map is generated at step 530.
  • the Access Map, access information and connection list including file set information is sent to affected peer nodes at step 535.
  • Fig. 8 is a topology service flowchart of the deletion of a node from the DCS from the perspective of the traffic cop node.
  • Fig. 9 is a topology service flowchart of the deletion of a node from the DCS from the perspective of the node.
  • the node sends a request for permission to leave/disconnect from the DCS to the traffic cop node.
  • the traffic cop node sends the node the connection list for the peer nodes that will be affected by the nodes disconnection at step 710.
  • the node thereafter propagates the connection list for the affected peer nodes at step 715 and disconnects from the DCS at step 720.
  • the routing service is responsible for initiating the generation and transmission of Request Maps and Access Maps to neighboring peer nodes whenever necessary. Access Maps are propagated between nodes to map out the locations of different data chunks in the DCS. Request Maps are propagated to maximize the data flow throughout the DCS.
  • the routing service will run in its own thread, and it is capable of handling routing for multiple transmissions simultaneously. Thus, every node in the DCS can receive the file or portions of the file simultaneously.
  • the software implementing the present invention is structured so as to run or process multiple threads simultaneously. This allows communication over the ports to occur simultaneously so that different messages over different ports occur simultaneously and on a non-interfering basis. When an applicable message arrives (Access Map, Request Map, data chunk), it will be added to a message queue. The routing service thread will repeatedly pulls messages from the queue and process messages from the queue.
  • peer-to-peer In all known peer-to-peer (P2P) systems conceived prior to the present invention, a file (piece of content) could be transmitted from one client to another client ("peer-to- peer" or “P2P") in its entirety.
  • the present invention adapts and configures peer-to-peer file transfer to the packet, or "chunk" level.
  • the packet level is a logical breakdown of a file into many packets/chunks so that each chunk may be retransmitted by a client as soon as it is received.
  • a client would have to receive an entire file before it could retransmit the file to another node. Because the efficiency of a P2P system is often limited by the amount of client upstream bandwidth that is utilized in the transfer, and the amount of time necessary to disseminate each file, these earlier systems are considerably less efficient than those employing the present invention.
  • EXAMPLE 1(a) The old way: S transmits the entire file to A, seconds 0 through 100; A transmits the entire file to B, seconds 100 through 200; Total time: 200 seconds.
  • EXAMPLE 1(b) The new way: Choose a logical chunk size of 100 KB;
  • S transmits 100 KB of the file ("chunk 1") to A ), during second 1; S transmits a different 100 KB ("chunk 2") to A, while simultaneously A transmits chunk 1 to B, during second 2;
  • S transmits a different 100 KB ("chunk 3") to A, while A transmits chunk 2 to B, during second 3; ... and so on;
  • S transmits the final 100 KB ("chunk 100") to A, while A transmits chunk 99 to B, during second 100; A transmits chunk 100 to B, during second 101.
  • Fig. 10 is a state transition diagram of the routing process. Commencing at state SO four events can occur. If a new node connects to the DCS then the response is to send the new peer/node an Access Map and proceed to state SI. From state SI a server can receive an Access Map from the new node.
  • Access Map Information is transmitted as a compressed table of chunk ranges in one exemplary embodiment, Access Map Information may be implemented as follows:
  • Chunk sizes are determined using a common algorithm based on file set size. The larger the file set, the larger the chunks are.
  • An Access Map table is maintained in memory (includes disk, hard drive etc.) such that the size of the Access Map table is equal to the number of chunks in the file set.
  • Access Map table is compressed prior to transmission and uncompressed upon receipt. All Access Map updates are transmitted as differences from the previous transmitted Access Map state to conserve bandwidth. Nodes, therefore, effectively transmit only changes to the Access Map.
  • Access Map Information is transmitted among nodes to convey to which chunks (portions) of data in the file set that node has known access. Each node will keep an Access Map representation in memory wliich will reflect the last map sent to that node. When two nodes connect, each should transmit its Access Maps to the other node as soon as possible. Updates (patches) should be sent whenever changes occur. [00192] In addition to generating its own Access Map and Request Map, a node generates an Access Map and a Request Map for each node (neighbor/subnode/peer) to which it is directly connected.
  • These neighbor/subnode/peer Access Maps and Request Maps are communicated as appropriate to other nodes (neighbors/subnodes/peers) to which the node is connected. This is done in order to propagate access information and requests for chunks of a file farther than just the directly connected nodes.
  • These Access Maps and Request Maps for neighbors/subnode/peers are called Neighbor Access Maps and Neighbor Request Maps.
  • a Request Map The goal of a Request Map is to facilitate the transmission of chunks of a file set between nodes via connections so that each node receives every chunk in minimal time. Chunk transmission is driven by tl e Request Map messages, which can be interpreted as
  • Request Map Information is transmitted as a compressed table of chunk ranges, and may be implemented as follows: Chunk sizes are determined using a common algorithm based on file set size. The larger the file set, the larger the chunks are.
  • a Request Map table is maintained in memory such that the size of the Request Map table is equal to the number of chunks in the file set.
  • the Request Map table is compressed prior to transmission and uncompressed upon receipt.
  • Request Map Information is transmitted among nodes to convey which chunks of data in the file set the node needs. Each node will keep a Request Map representation in memory which will reflect the last map sent to that node. Once an initial handshake protocol is complete, any node may send Request Maps to another node. Updates (patches) should be sent whenever changes occur.
  • the default Request Map is generated using the local access map. For each chunk that is held locally, the Request Map entry will have a value of 0, and for each chunk that is not held locally, the Request Map entry will have a value of 1. [00198] A node must store the most recent Request Map received from each of its neighbor nodes. These Request Maps should be frequently validated against the local Access Map to determine if some needed chunks are available locally. If a request cannot be fulfilled locally, then it must be forwarded to another node. [00199] To determine which node to which to forward a request, a check is first made for any pending forwarded requests. If any current forwarded requests intersect the new one (have any common needed chunks), then forward the intersection of the requests instead. If there are no current forwarded requests that overlap, choose the connection that can fulfill the request in the minimum number of hops (determined from its Access Map) or the most needed chunk. There may be additional schemes used to select the next requested file chunk.
  • each node When storing forwarded maps, each node also tracks which connection or connections contributed to the forwarded request. A forwarded request is removed when all chunks in the request have been received or all nodes contributing to the request have disconnected.
  • the routing service must maintain the following values in memory: Local Access Map List (including a list of local Access Map/Universal Resource Name (URN) combinations) and a Peer Map List (a list of each known node, and its information, for example, Session ID, URN, Last sent Access Map, Last sent Request Map, Last received Access Map, Last received Request Map, Forwarding target, and Last chunk requested)
  • URN the peer's session ID
  • URN the peer's URN
  • a peer is to be disconnected via disconnecting/ 'eer by removing the disconnectingPeer' s record from the peer map list and checking all remaining entries in the peer map list, and if any forwarding target equals disconnectingPeer, set the forwarding target to null.
  • Access Map newAccessMap
  • the validation passes (already forwarded). Otherwise the validation fails. If the validation fails, then the request must be forwarded: a. Look for an overlapping forwarded request: i. Loop through all entries in the Peer Map List for this URN in which the forwarding target is not null. It is unnecessary to check the peer from which this request originated. ii. For each non-null URN found, check to see if its incoming Request Map overlaps the Request Map that is to be forwarded. (It overlaps if any common index exists such that both Request Maps are -1 at that index.) iii. If a non-null URN is found, then forward the current Request Map to the same peer. b.
  • Access Map updates should be sent immediately to all peers except the peer from which that chunk was downloaded (rather than waiting until the next periodic, e.g. two second, interval). This method is preferred because it allows peers to become aware of chunks that can be downloaded as soon as chunks become available.
  • LQtpeerMap be the last Access Map received from the peer.
  • peerMap be the last Request Map received from the peer.
  • newRequestMap[index] 0 (note that this is the Request Map intersection routine)
  • Chunk requests can be made in two ways:
  • a peer has the chunk locally. (lastReceivedAccessMap[index] is 0). The chunk is not pending from any other node. (lastRequestedChunk is not index for any peer with the current URN)
  • a chunk at random from the subset of possibleChunks that includes only chunks that are within streamingBuffer a predetermined number of chunks of the first element of possibleChunks, for example within ten chunks.
  • topology and routing can be used independently of one another or more correctly with other routing and topology schemes, as indicated earlier they are most effective when used together as will be illustrated below.
  • Node N requests authorization to enter/connect to the DCS for purposes of file transfer. In this case as will be described below the DCS is a mesh.
  • node N connects to traffic cop S.
  • peer N requests access to a file. Traffic cop S then grants access and forwards access information, file information, a signature and a connection list to peer N as shown in Fig. 11(c).
  • Peer N then disconnects from ( traffic cop S as shown in Fig. 11(d).
  • peer N inserts itself into the DCS by connecting to one or more peers that are already in the DCS.
  • Peer N forwards access information and connection list information that was received from traffic cop S.
  • peer N connects to peers A and B. Connection list information is then sent to all affected peers.
  • Affected peers are peers that must change their connections as a result of the newly established connection with peer N.
  • Fig. 12(b) show peer B forwarding connection list information to peer G because that connection will be broken.
  • peer A forwarding connection list information to peer R because that connection will also be broken as a result of the entry of peer N into the DCS.
  • FIG. 12(c) shows the breaking or disconnection of peer R from peer A and peer G from peer B.
  • Fig. 12(d) shows the formation of a new connection between peer R and peer G as a result of peer N entering the DCS. The information for the new connection was sent to peers R and G along with the disconnection information. All of the above access information and connection lists are further described below in the description of the Packet Chain Protocol and the object schemas.
  • Fig. 13(a) illustrates peer A notifying each neighbor by sending Access Maps of all chunks of the file that peer A has and what chunks of the file that it and its neighbors need.
  • peers B, C, D and E request chunks of the file that they and their neighbors need from peer A via Request Maps sent to peer A.
  • Peer A requests chunks from its neighbors (not shown) and forward chunks of the file that it has that are needed by peers B, C, D and E to them.
  • peer C might have neighbors Cl and C2 (not shown) and peer C forwards a Request Map to peer A that includes the chunks that peers Cl and C2 need as well.
  • peer A if peer A does not have any chunk of the file that peer B needs then peer A notifies peer C, for example, of chunks of the file that both peer A and peer B need via updated Request Maps.
  • peer N Upon the completion of a file transfer, peer N connects to traffic cop S and notifies traffic cop S that the file transfer is complete and that peer N wishes to disconnect from the DCS. Traffic cop S then sends connection list information to peer N. Peer N then disconnects from traffic cop S. Connection list information is sent to all affected peers/nodes that must change their connection because peer N wishes to disconnect from the DCS. Connections that are not needed are broken including all of peer N's connections. Peer N exits the DCS and any new connections required as a result of peer N leaving the DCS are formed. [00212] A node is capable of efficiently downloading a file set from one or more sources.
  • the node will facilitate the download by connecting into a network DCS (dynamic connection structure), in which it can receive non-overlapping chunks of the file set from its neighbors/peers and send chunks to its peers/neighbors.
  • DCS dynamic connection structure
  • the protocol of the present invention to facilitate this process is denominated Packet Chain Protocol (PCP) and is part of the MMTP.
  • PCP Packet Chain Protocol
  • a file set refers to one or more files which should be downloaded. It is assumed that the files have a predetermined sequential order (used for indexing data chunks), but they do not necessarily need to arrive in that order.
  • a file set is data/content which includes content that is digital and/or multi-media.
  • a specification of the file set should include the number of files in the file set, their order, their names, their hash values, their relative paths, their sizes and a flag indicating whether the file set should be delivered sequentially (streaming) or segmented (downloads).
  • Chunks of data will be transmitted in chunks of a predetermined size. Receiving nodes will always tell transmitting clients which chunk should be sent next. To determine which chunk to request next:
  • the last Request Map was not a forwarded request, find all chunks that the peer has locally that the node does not have locally, choose one at random. If streaming, choose either of the above at random from only the lowest n qualifying chunks, where n is the streaming buffer size. None request a chunk that is pending from another peer.
  • connection change messages are used to communicate the changes to the affected nodes.
  • a non-firewall node's IP address and listening port can be extracted from the node's ID to conserve bandwidth. Because firewalled nodes cannot accept incoming connections, their node IDs are arbitrary and are used for identification purposes only.
  • connectionChaneelnformationList includes a timestamp, a signature and a connection list which contains connection change information (ConnectionChangelnf ⁇ ) for each node that is affected by the current changes.
  • ConnectionChangelnfo includes a session ID
  • Client Connectionlnfo includes, for example, for each of the node's connections, a session ID which is the unique identifier of the corresponding node's download session, a source URL which is the URL location of the node (not specified for firewall nodes) and buddy information which is the session ID and URL of the node's buddies if the node is firewalled.
  • This object is always generated by the traffic cop node in response to an event notification, and it is passed to the node notifying the traffic cop node of the event. Because a connection change message contains CCI objects for each affected node, only one message is sent by the traffic cop node. The message is then propagated through the affected nodes. When forwarding a connection change message, each node saves bandwidth by retransmitting only the necessary parts of the message.
  • An exemplary Connection Change Algorithm/process is invoked when a node receives a CCI message, as follows:
  • Forming connections includes both initiating a connection and accepting a connection initiated by another node.
  • initiating a connection if the connection is already pending as an incoming request, it can be taken directly from the incoming connection manager.
  • the node's IP and port are extracted from its ID, and the connection is formed directly.
  • the buddy is contacted, the connection change message is transmitted to the buddy, and the buddy is disconnected.
  • Traffic cop node protocol messages are messages that are exchanged between the traffic cop node and a node.
  • a Request Add Message is sent to the traffic cop node by a node desiring to connect to the network.
  • a Request Add Message includes, for example, the node ID which is the unique identifier of the node, the URN which is the unique identifier of the file set being requested, firewall status (firewalled or non-firewalled), the node URL which is the node's URL location and is not necessary for firewalled nodes, the average upstream bandwidth which is the node's average upstream bandwidth and the logon which is the ID/Password combination.
  • the Successful Add Information Message is sent from the traffic cop node to any node after a successful Request Add.
  • the Successful Add Information Message includes, for example, the session ID which is a unique identifier generated by the traffic cop node and which will identify the node during the transmission, the signature which is an authorization code for the node to enter the dynamic connection structure (DCS), file set information and connection change information.
  • the session ID which is a unique identifier generated by the traffic cop node and which will identify the node during the transmission
  • the signature which is an authorization code for the node to enter the dynamic connection structure (DCS)
  • file set information and connection change information.
  • the Can Not Add Message is sent from the traffic cop node to a node when an add request fails.
  • the Can Not Add Message includes, for example, an error code which is a number representing the type of error which prevented insertion of the node into the DCS and an error description which is a description of the error.
  • the Request Connection Update Message is sent to the traffic cop node by a node, whose connections are outdated or have been lost. This is a request for a current ConnectionChangelnfo object and includes, for example, the session ID and the URN.
  • the Connection Update Message is sent from the traffic cop node to any node after a connection update request and includes, for example, connection change information.
  • the Delete Node Message is sent to notify the traffic cop node that a node should be deleted from the DCS.
  • the Delete Node Message includes, for example, the delete ID which is the ID of the node being deleted from the DCS, the node ID which is the ID of the node requesting the deletion from the DCS, the delete code which is the number representing the type of situation which caused the deletion and the delete description which is a description of the conditions which caused the deletion from the DCS.
  • the Delete OK Message is sent from the traffic cop node to a node after a delete request has been completed and includes Connection Change Information.
  • Every node-to-node connection should start with a handshake in which each node transmits their Node/Client ID and signature. Each node has the ability to validate that the other node's signature was originally generated by the traffic cop node. [00231] Encryption mechanisms are optionally available, which guarantee that transmitted data could not be interpreted by an outside party. [00232] The dynamic connection structure is always maintained by only the authorized traffic cop node, and nodes are never able to cause unauthorized connection changes. Every ConnectionChangelnfo object is signed by the traffic cop node, and nodes will ignore messages without valid signatures.
  • the ConnectionChangelnfo object On an initial connect, the ConnectionChangelnfo object is always be transmitted from the connecting node to the node accepting the connection. The accepting node will then respond with its Access Map. Updates (patches) are sent periodically whenever changes occur, with a minimum frequency (e.g. every ten seconds).
  • a minimum frequency e.g. every ten seconds.
  • the Packet Chain Protocol (PCP) of the present invention can reduce the amount of hardware and bandwidth needed to distribute content to millions of people on the Internet without reducing/diminishing losing any QoS. This is accomplished by forming a dynamic content structure (DCS) network of computers and utilizing their unused bandwidth (upstream as well as downstream) and resources. By linking computers together maximum efficiency for content distribution over the Internet can be achieved. Use of the PCP of the present invention, thus, allows users to receive content faster and allows content providers to use fewer resources.
  • DCS dynamic content structure
  • File Set Info contains information about the specified content.
  • the content can contain one file or a list files. If the FSI has more than one file it is considered a bundle and the node will receive all files that are contained in the bundle. It is assumed that the files have a predetermined sequential order (used for indexing data chunks), but the files do not necessarily need to arrive in that order.
  • An exemplary FSI object schema is illustrated below.
  • Fig. 27 shows a system of ports and sockets adapted for the simultaneous/ contemporaneous reception and transmission of files according to the invention
  • Fig. 28 shows the relative positions and transport layer relationships between and among the Multimedia Transport Protocol and applications of the present invention.
  • ⁇ delivery> is the type of transfer to use when delivering a FSI, for example, download and streaming.
  • ⁇ hash> is the digest of the entire FSI. If there is only one file in the FSI it is equal the file's hash.
  • ⁇ file> is the list of files in the file set.
  • An exemplary FI object schema is illustrated below.
  • ⁇ index> index of this file in the FSI ⁇ name> is the name of the file.
  • ⁇ hash> is the digest of the file.
  • ⁇ size> is the size of the file.
  • ⁇ rpath> is the relative location to save the file.
  • ⁇ type> is the MIME type of the file.
  • ⁇ chunkHashes> the file is broken up into small pieces and each piece is hashed . This field is the concatenation of those hashes.
  • ⁇ metadata> is any additional information about the file. (For example, if the MIME type is mp3 this field might contain ID3 tag information.)
  • ⁇ sessionid> is a unique id that identifies a new session with the traffic cop network element.
  • ⁇ src> is the source URL For example, pcp://123.456.44.44:5457 ⁇ connection> is a list of connections that ⁇ src> is connected too.
  • ⁇ sessionid> is a unique id that identifies a new session with the traffic cop for the specified connection.
  • ⁇ src> is the source URL For example, ⁇ cp://123.456.44.44:5457 ⁇ root> identifies that this connection is to a root server node.
  • ⁇ buddy> identifies that this connection uses a buddy connection, not a direct connection.
  • a Connection Change Info List (CCIL) has a list of Connection Change Infos that a servant uses to connect into the DCS
  • the CCIL also contains other CCIs that are passed to the connected servants so they can be properly connected into the DCS.
  • He CCI and CCIL object schemas pertain to the topology of the DCS.
  • the traffic cop network element accepts requests via HTTP(S).
  • HTTP HyperText Transfer Protocol
  • the response to one of the above requests contains an XML message with the appropriate information. Both the request and response will contain standard HTTP headers along with custom headers. The value for the content-type response header will always be of MIME type xml/text.
  • a servant makes an insert request to the traffic cop node in order to obtain information on how to be added into the DCS, so that the servant can start receiving the specified content.
  • the firewall test is the connection back that is described above.
  • the traffic cop node attempts to connect back to the node, if it is successful, it assumes that the node is not behind a firewall. If it is a unsuccessful, it is assumed that the node is behind a firewall.
  • the body of the response will contain an XML message that will have information about how the requesting node can join the DCS and detailed information about the specified content. Once the servant receives this message it can then make the necessary TCP/IP connections to other servants that can deliver the specified content.
  • the body of the response will contain an XML message that will have information about how the other peers/nodes that the node being deleted is connected to should form other connections in the DCS.
  • a servant makes a reposition request to the traffic cop network element when it finds itself in a non-optimal position in the DCS.
  • the response information contains information on where a servant can connect to be in a more optimal position for receiving the specified content.
  • the body of the response will contain an XML message that will have information about how the requesting node can reposition in the DCS. Reposition elsewhere in the DCS Response Schema
  • the body of the response will contain an XML message that will have information about new connection information in the DCS.
  • the PCP Client is capable of efficiently downloading a file set from one or more sources.
  • the node will facilitate the download by connecting into a network DCS, in which the node can receive non-overlapping "chunks" of the file set from its neighbors and send "chunks" to its neighbors.
  • a network DCS in which the node can receive non-overlapping "chunks" of the file set from its neighbors and send "chunks" to its neighbors.
  • a PCP client connects itself to the DCS by making an "insert into the DCS" request to traffic cop node for information on what node to connect to and detailed information about the desired content.
  • the traffic cop node responds with an IDCSR message (response body schema) containing this information.
  • the node then extracts the information from the message and goes through the follow steps: 1. Validate the timestamp and signature and dispose of the message if it fails either validation. Find the ConnectionChangelnfo object in the structure which matches the local node ID. Remove the Connection List from the object.
  • Fig. 14 is a schematic diagram illustrating connecting to a firewall servant. Servant E needs to connect to Servant A, but a direct connection cannot be established because Servant A is behind a firewall.
  • Servant E then connects to Servant A's buddy Servant B sending a message which should be routed to Servant A.
  • Servant A receives the message it makes a direct connection to Servant E.
  • PCP handshake occurs to validate the two nodes. The handshake in structured much like a HTTP request:
  • Data messages contain the "chunks" of the desired digital content.
  • the header Payload length defines how much raw data is in a message.
  • Nodes use their neighbor's Access Map and Request Map to decide what "chunks" of data to send to each other. These messages are sent with the payload containing the sequential data. Access Map and Request Map updates are also sent if there are any changes. If the data range changes, a Data Range Change message is sent telling the receiving node that a new range of data is coming.
  • New Range is the new starting offset into the data stream for the next data message.
  • the Data Range Change message tells the receiving node what the next starting offset into the data stream it is going to send will be. The offset in stored in the New Range section of the message. After the Data Range Change message, data messages can be sent with the appropriate range of data.
  • New Connection Information Payload Null term [00259] Payload is a null terminated XML string that represents the new connection information. New Connection Info is sent to a node when its connections need to be changed.
  • CMDB Content Management Database
  • the content provider administration tools allow providers to manage/monitor their content distribution via a web interface.
  • Content provider administration tools use JSP pages which run on an Apache/Tomcat system hosted by the present invention.
  • a MySQL database can be used to store all provider/content related information.
  • Many traffic cop components will run using JSP pages, but these components will run on processors separate from the content provider administration tools, and they may or may not be hosted as part of the present invention.
  • An exemplary database is modeled as depicted in Fig. 16 and includes the following exemplary tables:
  • the Provider Table (10005) lists information about each provider. This table includes provider names, billing information, contact information, etc.
  • the User Table (10010) lists information about each user. This table is necessary for user security. Each user is associated with a provider.
  • the Role Table (10015) lists the available roles that the Content Management system uses for security. These include:
  • Provider - The provider role only provides access to add, edit and remove content for the users own provider. Providers also have access to edit the details of their own user account.
  • Provider-admin - The provider-admin role provides the same functionality as the provider role plus the ability to add, remove and edit user information under their own provider.
  • System-admin - The system-admin role provides the same functionality as a provider- admin but with the ability to work under any provider in the system.
  • the User Role Table (10020) associates a user with a particular role.
  • the Content Table (10025) lists information about a specific piece of content on the network.
  • URN - for static content this is the SHA hash of the content, for streaming content, this is a unique identifier automatically assigned to the stream.
  • Provider ID - the provider responsible for the content.
  • Default Delivery Type specifies whether the content should be delivered static or streaming.
  • a negative one (-1) implies static.
  • a positive number implies streaming, where the number is the streaming buffer size.
  • Default HS QOS the default quality of service for high speed users.
  • Promo Link - this field is used to specify promotional links which display to users while the download is in progress.
  • Metadata metadata associated with the content. Name - a name given to the piece of content. For content with only one file, it defaults to the filename.
  • Active content is available to users. Inactive content may not be downloaded. Pending content is unavailable to the user because it is being processed by the system (SHA generation, for example).
  • the Root Server Table (10030) lists information about each known root server/seed, node on the network.
  • a physical server may be referenced by more than one record if the server hosts more than one content object.
  • Content ID - a link to the content that the root server hosts.
  • Type - the type of server. This is usually zero for all servers, implying HTTP reliable server. This field may, however, represent failover servers, or even track user machines that allow uploads. Description - a text description of the root server.
  • the File Table (10040) contains information about each file that is accessible on the network. Content ID is specified as the specific piece of content to which this file belongs.
  • Each content object contains exactly one file.
  • a content object may contain multiple files (bundles) which allows multiple file records to exist for the same content ID.
  • Size the size of the file in bytes (initialized to 0 until the file is processed by the system).
  • Metadata metadata associated with the file.
  • Content ID the ID of the associated content.
  • the Chunk Table (10045) holds information about the individual chunks into which the file is broken.
  • Chunk Index the index that this chunk holds in a file.
  • URN the SHA hash of the chunk.
  • File ID the ID of the associated file.
  • the Mesh Table (10050) tracks all meshes that are created. When a traffic cop starts a mesh, a row will be inserted in this table recording the following:
  • Finish Time the time the mesh finished.
  • Status either active(O), finishing(l) or closed(2). Meshes will always start as active, and move to closed when they finish. If a piece of content is edited, the status for all active meshes will be changed to finishing. Additional active statuses specify that a mesh is full
  • Traffic Cop ID the ID of the traffic cop controlling the mesh.
  • File ID the serving file.
  • Delivery Type - specifies whether the content should is delivered static or streaming.
  • a negative one (-1) implies static.
  • a positive number implies streaming, where the number is the streaming buffer size.
  • the Traffic Cop Table (10055) tracks all known traffic cops on the network. When a traffic cop starts, it adds a record for itself to this table (if not already present), and set the status to active. When a traffic cop finishes, it updates its record to reflect that it is inactive.
  • URL the URL location of the traffic cop.
  • Status active or inactive.
  • the Download Stats Table (10060) records statistics of PCP transmissions. Most provider reports will be run against data in this table. Data is stored with respect to set time slices.
  • One hour time slices are the most commonly used. Records are added to the table for each active mesh at intervals of one time slice. Each record will reflect all statistics gathered by the mesh's traffic cop during that time slice.
  • the content provider administration tool allows providers to manage/monitor their content distribution via a web interface.
  • Content provider administration tools use JSP pages which run on an Apache/Tomcat system hosted by the present invention. This interface is secure (SSL). Its components may be categorized as follows: Login
  • the Login screen serves as the entry point to the web system, allowing providers to log in to the Provider Administration System. Unsuccessful logins display an error message to the user and allow them to try again. Successful logins initiate a session for the user, and take the user to the Content Management screen. Multiple independent sessions may exist for the same Provider ID simultaneously.
  • An exemplary Login screen is illustrated in Fig. 17. It should be noted that the User ID for the Login screen does not correspond to User ID in the database, which is a unique numeric key. The User ID for the Login screen corresponds to Login ID in the database, which is a text login assigned manually to new users.
  • the content management main screen lists all content for the provider in table format.
  • buttons allow the user to edit, activate/deactivate, or view statistics for the content.
  • the edit button takes the user to the Content Modification screen.
  • Active content has a deactivate button.
  • Inactive content has an activate button. Pressing the activate button takes the user to the Activate Content Verification screen. Pressing the deactivate button takes the user to the Deactivate Content Verification screen. Pressing the stats button takes the user to the Content Distribution Statistics screen.
  • an Add New Content button takes the user to the New Content Creation screen.
  • Fig. 18 is an exemplary content management main screen.
  • the activate content verification screen verifies that a user wants to activate a piece of inactive content. Both OK and Cancel take the user to the Content Management Main screen, but OK activates the content first.
  • Fig. 19 is an exemplary activate content verification screen.
  • the deactivate content verification screen verifies that a user wants to deactivate a piece of active content. Both OK and Cancel take the user to the Content Management Main screen, but OK deactivates the content first.
  • Fig. 20 is an exemplary deactivate content verification screen.
  • the new content creation screen allows a user to add new content to the system.
  • the user will be able to enter the content name, URL, delivery type, high-speed QOS, and description. Successful entry will take the user to the Content Management Main screen. Unsuccessful entry will return the user to this screen, and display an error message.
  • Fig. 21 is an exemplary new content creation screen.
  • the contents' Status is set to Pending, and an asynchronous process is started that downloads the actual file and computes the appropriate SHA Hash value(s) for the file.
  • the Status is set to Active and an email is sent to the user who created the content to alert them of its availability.
  • the content modification screen allows a user to edit existing content on the system.
  • the screen pre-populates with the current values of the selected content. The user is able to change any of these values except the content name. If the user clicks the apply button and the URL does not change, the changes are applied and the user returns to the content management main screen. If the URL changed, the user is taken to the URL change verification screen. Selecting the cancel button takes the user to the content management main screen.
  • Fig. 22 is an exemplary content modification screen.
  • the URL Change Verification screen is necessary because changing root server URLs is potentially dangerous to the system. If the file is different then the user must let the system know.
  • Fig. 23 is an exemplary URL change verification screen.
  • the account maintenance main screen allows a user to modify user information. Links to the other areas of the system are provided. Initially, only password change functionality will be available. Once we begin tracking more specific provider information (billing, contact info, etc.), it will become editable here. Pressing apply changes saves the changes and returns the user to the content Management Main screen, unless errors occur which returns the user to this screen with an error description. Pressing cancel returns the user to the Content Management Main screen.
  • the Provider ID for the account maintenance main screen does not correspond to Provider ID in the database, which is a unique numeric key.
  • the provider ID for the account maintenance main screen corresponds to Login ID in the database, which is a text login assigned manually to new providers. Fig.
  • the Provider Distribution Statistics screen allows users to view statistics for all of their content. This screen will summarize all data delivered for the provider. This screen also includes a detailed provider reporting component.
  • Fig. 25 is an exemplary Provider Distribution Statistics screen. Additionally, these same statistics may be generated for a specified time period.
  • the Content Distribution Statistics screen allows users to view statistics for a specified content object. This screen also includes a detailed content reporting component.
  • Fig. 26 is an exemplary content distribution statistics screen. Additionally, these same statistics may be generated for a specified time period.
  • the tree representing the dynamic connection structure may be organized using a different algorithm.
  • an ordered binary tree may be implemented in such a way that j-clients located geographically near one another would be grouped together on branches of the tree. This may be shown to further minimize file transfer time.
  • Another embodiment of the invention could include a checksum feature to ensure the accuracy of the file transfer, or include provisions for packet encryption or other security measures.
  • the method could be modified to stream an audio or video file in such a way that each packet is played on the j-client computer as it is received.
  • a DCS of the present invention can be used in any scenario or configuration where resources of clients are needed.
  • the configuration/ topology of the present Dynamic Connection Structures can be organized specially depending on the types of resources needed. These include, for example, message delivery across firewalls, chat message routing, on-line game playing, grid computing/ utility computing and distributed storage. 105. Message Delivery EXAMPLE - Many firewall type security provisions currently are used which block connections into computers.
  • a DCS can be used to facilitate message delivery into firewalled computers.
  • the DCS would be structured as: Firewall Firewall Firewall Firewall Firewall Firewall
  • Chat - A DCS is created for a group of clients that wish to chat together, and chat messages are routed between these clients.
  • Grid computing/utility computing - a group of clients are connected into the DCS and information is passed among them in order to utilize the CPU processing power of the clients.
  • Distributed storage - a group of clients are connected into the DCS and information is passed among them in order to utilize the aggregate hard- drive storage power of the clients.
  • Non-topology messages can also be effected through systems utilizing the present Dynamic Connection Structures.
  • a DCS is used to facilitate message delivery to a set of clients.
  • Many types of messages could be delivered, such as: 1. An invitation/directive to join another DCS a. Maybe the client is needed as a file upload source in the other DCS; b. Maybe the other DCS is transferring content that the client may be interested in; c. Maybe the other DCS is a chat group that the client may be interested in
  • a chat message 3. A configuration message; or 4. A promotional message/advertisement.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computing Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Theoretical Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Communication Control (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

L'invention concerne des structures de connexion dynamiques (DCS) faisant intervenir plusieurs schémas d'objets pour effectuer les topologies et le routage afin de transférer des fichiers de données vers des pluralités d'ordinateurs ou de noeuds sur un réseau. Lesdites structures font intervenir plusieurs types de schémas d'objets, tels que des FSI (informations d'ensembles de fichiers) et des FI (informations de fichiers) servant à faciliter les mailles, graphiques, anneaux et algorithmes d'arbres binaires des DCS, et faciliter l'adaptation et le changement des topologies et schémas de routage dans des architectures pair à pair en fonction de divers attributs spécifiques des noeuds, tels que leurs positions relatives, leurs capacités, leurs états de contenu, et leurs relations respectives par rapport à d'autres noeuds dans les DCS. Les schémas selon l'invention contiennent également des schémas d'objets de listes d'informations de changements (CCIL) et des schémas d'objets d'informations de changements de connexion (CCI) de telle manière que les schémas d'objets CCI peuvent être passés à des serveurs pour connexion aux DCS et à des serveurs connectés afin de repositionner les serveurs connectés dans les DCS. Par conséquent, les noeuds selon l'invention peuvent être conçus pour changer efficacement leurs topologies et acheminer des contenus entre les noeuds des DCS.
EP04795287A 2004-05-19 2004-10-18 Protocoles de schemas d'objets et de chaines de paquets destines a la gestion du routage et de la distribution de contenus numeriques dans des structures de connexion dynamiques pair a pair Withdrawn EP1776643A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US57222904P 2004-05-19 2004-05-19
PCT/US2004/034100 WO2005119477A2 (fr) 2004-05-19 2004-10-18 Protocoles de schemas d'objets et de chaines de paquets destines a la gestion du routage et de la distribution de contenus numeriques dans des structures de connexion dynamiques pair a pair

Publications (1)

Publication Number Publication Date
EP1776643A2 true EP1776643A2 (fr) 2007-04-25

Family

ID=35463215

Family Applications (2)

Application Number Title Priority Date Filing Date
EP04795288A Withdrawn EP1752002A1 (fr) 2004-05-19 2004-10-18 Topologies de structures de connexion dynamiques et procede simplifiant le transfert pair a pair de fichiers numeriques
EP04795287A Withdrawn EP1776643A2 (fr) 2004-05-19 2004-10-18 Protocoles de schemas d'objets et de chaines de paquets destines a la gestion du routage et de la distribution de contenus numeriques dans des structures de connexion dynamiques pair a pair

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP04795288A Withdrawn EP1752002A1 (fr) 2004-05-19 2004-10-18 Topologies de structures de connexion dynamiques et procede simplifiant le transfert pair a pair de fichiers numeriques

Country Status (2)

Country Link
EP (2) EP1752002A1 (fr)
WO (3) WO2005120102A1 (fr)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2431814A (en) * 2005-10-31 2007-05-02 Hewlett Packard Development Co Distribution of data in a network
US7881315B2 (en) 2006-06-27 2011-02-01 Microsoft Corporation Local peer-to-peer digital content distribution
US9026654B2 (en) * 2006-10-26 2015-05-05 Avaya Inc. Peer-to-peer overlay graph construction
GB2446170A (en) * 2006-12-01 2008-08-06 David Irvine Shared access to private files in a distributed network
GB2446200A (en) * 2006-12-01 2008-08-06 David Irvine Encryption system for peer-to-peer networks which relies on hash based self-encryption and mapping
US8996723B2 (en) * 2007-06-04 2015-03-31 Microsoft Technology Licensing, Llc ISP-aware peer-to-peer content exchange
CN103188279B (zh) * 2011-12-27 2016-06-01 中国电信股份有限公司 通过对等网络从多个邻居节点下载文件的方法和装置
CN103064285B (zh) * 2012-12-29 2015-08-26 杭州电子科技大学 一种基于模型的热泵供暖多目标优化控制方法
EP3353952B1 (fr) * 2016-01-29 2021-05-05 Hewlett Packard Enterprise Development LP Gestion de groupes de serveurs
US11257155B2 (en) * 2018-08-27 2022-02-22 Chicago Mercantile Exchange Inc. Apparatuses, methods and systems for a computationally efficient volatility index platform
CN110569186B (zh) * 2019-08-13 2022-04-19 中核控制系统工程有限公司 一种基于核电厂dcs平台逻辑算法块间连线的维护方法
CN112910936B (zh) * 2019-11-19 2023-02-07 北京金山云网络技术有限公司 数据处理方法、装置、系统、电子设备及可读存储介质
CN112637067A (zh) * 2020-12-28 2021-04-09 北京明略软件系统有限公司 基于模拟网络广播的图并行计算系统和方法
CN114268635B (zh) * 2021-12-02 2023-12-15 珠海迈科智能科技股份有限公司 一种p2p流媒体文件定位与节点选择的系统及其方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1994025913A2 (fr) * 1993-04-30 1994-11-10 Novadigm, Inc. Procede et appareil destines a la gestion d'entreprise assistee par ordinateur
US6584073B1 (en) * 1999-06-02 2003-06-24 Sun Microsystems, Inc. Network topologies
EP1297440B1 (fr) * 2000-05-12 2008-08-27 Niksun, Inc. Camera de securite pour reseau
US6636854B2 (en) * 2000-12-07 2003-10-21 International Business Machines Corporation Method and system for augmenting web-indexed search engine results with peer-to-peer search results
US7209973B2 (en) * 2001-04-09 2007-04-24 Swsoft Holdings, Ltd. Distributed network data storage system and method
US7574488B2 (en) * 2002-05-31 2009-08-11 Hitachi, Ltd. Method and apparatus for peer-to-peer file sharing
US7613796B2 (en) * 2002-09-11 2009-11-03 Microsoft Corporation System and method for creating improved overlay network with an efficient distributed data structure
US7774495B2 (en) * 2003-02-13 2010-08-10 Oracle America, Inc, Infrastructure for accessing a peer-to-peer network environment
US20040172336A1 (en) * 2003-02-27 2004-09-02 Peter Forsell Method and apparatus for advertising objects

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2005119477A3 *

Also Published As

Publication number Publication date
WO2005120102A1 (fr) 2005-12-15
WO2005119477A3 (fr) 2006-04-20
WO2005119476A2 (fr) 2005-12-15
WO2005119477A2 (fr) 2005-12-15
WO2005119476A3 (fr) 2006-06-01
EP1752002A1 (fr) 2007-02-14

Similar Documents

Publication Publication Date Title
CN1961558B (zh) 改善对等网络通信的方法
Heinzelman et al. Adaptive protocols for information dissemination in wireless sensor networks
US7493363B2 (en) Peer-to-peer group management and method for maintaining peer-to-peer graphs
Buchegger et al. PeerSoN: P2P social networking: early experiences and insights
Yang et al. An efficient hybrid peer-to-peer system for distributed data sharing
CN101068245B (zh) 共享文件的发布、下载方法及文件共享可控系统
CN102907065B (zh) 用于管理对等网络中的数据传递的系统和方法
Kashyap et al. Efficient gossip-based aggregate computation
EP1776643A2 (fr) Protocoles de schemas d'objets et de chaines de paquets destines a la gestion du routage et de la distribution de contenus numeriques dans des structures de connexion dynamiques pair a pair
CN110754070A (zh) 最近交易在区块链网络上的快速传播
JP2006294009A (ja) ピアツーピアメッセージングアプリケーションを構築するためのａｐｉ
CN101606143A (zh) 用于对等网络的增强体验的系统和方法
Shen IRM: Integrated file replication and consistency maintenance in P2P systems
JP7770322B2 (ja) 特定ネットワークデバイス並びに特定ローカルエリアネットワークの接続、コンテンツ発見、データ転送、及び制御方法
Li et al. Distributed dataset synchronization in disruptive networks
US8086629B2 (en) Content delivery apparatus, content delivery method, and content delivery program
US20080247400A1 (en) System and method for increasing the efficiency in the delivery of media within a network
Sunaga et al. Technical trends in P2P-based communications
CN114553886B (zh) 数据传输方法和通信装置
CA2595438C (fr) Procede permettant d'ameliorer la communication dans un reseau poste a poste
Mavromoustakis et al. A Gossip-based optimistic replication for efficient delay-sensitive streaming using an interactive middleware support system
Namgung et al. Self-organizing P2P overlay network applying dynamic landmark mechanism for contents delivery network
Rajasekhar Quality of service routing with performance considerations in peer-to-peer systems
Tseng et al. Peer‐assisted content delivery network by vehicular clouds: Algorithm and evaluation
Bejan et al. Self-optimizing DHTs using request profiling

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20061219

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 15/16 20060101AFI20070330BHEP

Ipc: G06F 15/173 20060101ALI20070330BHEP

Ipc: G06F 3/00 20060101ALI20070330BHEP

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20080503