ES3032063T3 - A method for safe train remote control, with processing image frames via two processing lines - Google Patents

Abstract

In summary, the present invention proposes a secure remote control for automated trains (ATR). The leading track section is monitored by a video camera system (CAM) that generates high-resolution images (HRI). These images are converted into low-resolution images (LRI) and transmitted by a mobile communications network (MCN) to a remote control console located in a track control center (COC) via a first data channel (CH1). Furthermore, the high-resolution images (HRI) are subjected to pattern recognition, and for identified objects that correspond to reference objects in a map database (MDB), reconstruction information is transmitted by the mobile communications network (MCN) to the train remote control console via a second data channel (CH2). At the train remote control console, the high-resolution object appearances (HROA) of the identified objects are reconstructed using the reconstruction information and the corresponding map database (MDB). High-resolution object appearances (HROAs) are superimposed on low-resolution imagery (LRI) received via the first data channel (CH1) and displayed on the train's remote control console. This allows the remote train driver to access safety-relevant details and an overview of the track section, while consuming minimal bandwidth from the mobile communications network. (Automatic translation using Google Translate, no legal value)

Description

DESCRIPTION

A procedure for safe remote train control, with image frame processing via two processing lines

The invention relates to a method of safe remote control of trains, wherein an automatic train comprises

- an on-board ADS (Automatic Driving System) unit, which includes a camera system that monitors a section of track ahead of the automatic train, and

- an automatic train mobile communications router for forwarding and receiving data via a mobile communication network,

where an ATO (Automatic Train Operation) ground rail traffic operation and management center comprises

- a train remote control console for remotely controlling the automated train, including a display showing an image of the track section in front of the train to a remote driver on the train remote control console, and in particular including a moving map display for indicating a train position of the automated train, and

- a mobile router of the management centre for forwarding and receiving data via the mobile communication network, wherein the ATO Operation and Management Centre (COC) and the Automatic Train (ATR) each comprise a Map Data Base (MDB) containing common reference objects.

This procedure is known from document JP 2019001203 A.

Trains are an important means of transport, both for passengers and freight, in a fast, efficient, and environmentally friendly manner. Typically, an onboard train driver, usually in the locomotive at the front of the train, observes the section of track ahead of the train through a window and drives the train using the local (onboard) driver's cab.

However, efforts have been made to automate train driving. At Grade of Automation (GoA) level 4, there is generally no longer a train driver on board. However, there may remain situations in which human interaction is necessary or desirable. Such situations may include emergencies or technical defects in the automatic train control system, or more generally, situations where the automatic train control system recognizes a situation that it cannot safely handle in automatic mode. To safely complete the train's journey, a human driver must take control, at least temporarily.

For this purpose, a driver can be brought onto the train, who can enter the driver's cab and assume control of the train on board. However, this can be cumbersome and cause significant delays. More convenient is remote control of the train by a human train driver in an Automatic Train Operation (ATO) rail traffic management and operation center (also simply called a control center). The train driver in the control center ("remote driver") requires basically all the information that an on-board train driver would have, and requires access to basically all the driving functions that an on-board train driver would have.

An important piece of information required by the remote train driver at the ground control center is a view of the section of track ahead, particularly to identify railway signs and signals, as well as potential objects ahead of the train, including obstacles on the track.

Document EP 3 220 613 A1 suggests that a train driver at a ground control center temporarily take control of a driverless train, where the ground control center is connected to a train camera that monitors the section of track ahead of the train. Communication takes place via a packet-based communication network.

The transmission of image information between the automated train or its onboard automated driving system unit and the ground control center or the remote train control console where the remote operator sits requires significant bandwidth in a mobile communications network. In general, however, bandwidth is often quite limited.

To save bandwidth, it is possible to use a low pixel resolution for image data and apply image compression. However, if the pixel resolution is low, safety-relevant details on the section of track in front of the train may not be recognizable. Image compression can also blur image details if applied excessively. For example, if the switching state of a railway signal can no longer be reliably identified by the remote driver, the safety of the train can immediately be jeopardized by collisions. Furthermore, the time resolution of the image data can be chosen to be low to ensure bandwidth, but this can lead to irregular images. In general, delays in image data transfer can also endanger the train.

Document EP 1597 130 B1, also published as WO 2004/074068 A1, describes a remote control for an unmanned locomotive on a track layout in a shunting yard. A handheld communications device is used to command a desired destination for the locomotive and control its movement.

Document EP 2765053 B1 describes a railway train diagnostic system, in which a wireless interface onboard a train transmits diagnostic data from an onboard control unit to a ground-based server. The diagnostic data is transmitted on a plurality of channels, including a live channel and a filler channel.

Document WO 2006/028318 A1 describes an apparatus for wireless remote control of a magnetic levitation train, which implements two-way packet-based communication.

Alstom recently announced the operation of metro trains with passengers on level 3 of GoA, and during the maneuver with level 4 of GoA, without personnel on board but with the possibility of remote control of the train, compare

https://www.alstom.com/press-releases-news/2020/5/world-first-automatic-train-operation-regional-passengertrains-be

on 05/11/2021.

Document JP 2019001203 A describes a railway operation support system, wherein video signals from a camera on a train are transmitted using a video signal compression unit via a wireless network to a video display in a ground control camera. In one variant, an object detection part is connected to the output of a camera on the train, and if the object detection part finds an object to be viewed, then a video signal control part maintains high visibility without reducing the resolution or time frame rate of the neighboring area that includes the object. In a further variant, the video compression unit imparts train position information as metadata of the video data to the control camera. A storage on the train and a storage in the control camera contain high-resolution past video data of the neighborhood area. The transmitted video signal contains only a difference between the video data from the camera and the storage on the train, and at the control camera, the transmitted difference is added to the video data from the storage in the control center, using the position information.

Document DE 697 31 009 T2 describes automatic obstacle identification, particularly for a railway collision avoidance system, in which a camera is automatically aligned. The automatic alignment can use GPS position information.

Document DE 10 2012 215 544 A1, also published as US 2015/201165 A1, describes a method for monitoring a railway track, wherein a railway vehicle is equipped with a recording unit, and the recording is stored on the railway vehicle or at a central location. The recorded images can be subjected to image analysis, can be provided with time or position information, and can be compared with previously recorded image data. The railway vehicle can communicate with the central location via a wireless network.

Object of the invention

The object of the present invention is to provide a method for remote control of an automatic train, wherein video images of the track section ahead of the train, including sufficient details of the track section ahead, can be provided for remote control of the train to be saved, in the railway traffic operation and management center at the ATO on the ground, which only requires low bandwidth consumption of a mobile communication network.

Brief description of the invention

This object is achieved, according to the invention, by a process as introduced at the beginning, characterized by

that said reference objects each represent typical types of objects on Earth, wherein for each reference object at least one object ID and high-resolution object appearance information representative of the corresponding typical type of object on Earth are stored, wherein the high-resolution object appearance information is vectorized,

that the camera system generates high-resolution image frames, which are assigned image frame numbers, and which are processed in at least two processing lines,

where in a first processing line,

- high-resolution image frames are downsampled, compressed and encoded in the integrated ADS unit into low-resolution image frames, and

- the low-resolution image frames and their image frame numbers are transmitted via a first data channel from the on-board ADS unit to the remote control console of the train via the mobile communication network,

where in a second processing line,

- the high-resolution image frames are subjected to a pattern recognition algorithm, which identifies objects in the high-resolution image frames, maps them to reference objects stored in the map database of the integrated ADS unit, and determines the corresponding object ID and relative object image embedding properties of each identified object, including at least an image frame number, an insertion vector point, and scale/rotation parameters, and

- the determined object IDs and image insertion properties are transmitted via a second data channel from the on-board ADS unit to the train's remote control console via the mobile communication network,

that at the ATO ground rail traffic management and operation centre, the object IDs and image insertion properties of the second data channel are translated into high-resolution object appearances based on corresponding stored reference objects in the ATO ground rail traffic management and operation centre map database,

and that these high-resolution object appearances are seamlessly superimposed on the low-resolution image frames with matching image frame number received via the first data channel, resulting in the image of the track section in front of the train being displayed to the remote driver at the ATO's ground-based rail traffic management and operation centre.

According to the present invention, video images of the track section ahead of the train that are displayed on the remote control console of the train to the remote driver are based on image data originating from (at least) two respective processing lines and data channels.

In order to capture all relevant safety information, the automated train's camera system generates a video signal comprising high-resolution image frames. However, these high-resolution image frames are not transmitted over the mobile communications network as such, as this would require a large bandwidth.

In a first processing line, the high-resolution image frames (pixels) are downsampled in each imaging direction, typically by resampling the image with known techniques such as bilinear or bicubic interpolation, thereby generating low-resolution image frames (pixels). Furthermore, the images are compressed using known techniques such as run-length coding or discrete cosine transforms or other video encoders. These low-resolution image data frames are transmitted via a first data channel from the automatic train (or its onboard ADS unit) to the ground rail traffic management and operation center (ATO) (also simply called the "control center") via the mobile communication network. This requires very little bandwidth. The low-resolution image frames are sufficient to give the remote driver an overall impression of the track section, particularly for general orientation. For example, road vegetation, such as grass, trees, or buildings on public roads, do not require high resolution to be recognizable, and are not important for train safety. In the first data channel, familiar video image compression techniques (such as i-frame/p-frame/b-frame prediction) can be applied to further reduce the required bandwidth.

In a second processing line, the high-resolution image frames (pixels) are subjected to pattern recognition. The pattern recognition is based on a comparison with reference objects stored in a map database of the automatic train. These reference objects represent different types of objects that can typically be encountered in front of a train, including objects of all kinds on the ground that could be relevant to the safe operation of the train, in particular railway signals (including their switching status), railway signals, tracks, track switches (including their switching status), and railway crossing bars. The reference objects preferably also include typical obstacles that may appear on a track, such as train cars and locomotives, fallen trees, disabled vehicles, people (in particular track workers and walkers), and large animals (such as cows). For each reference object, at least one object ID (typically an alphanumeric code) and representative high-resolution appearance information are stored. The high-resolution appearance information is vectorized, comprising a plurality of anchor points, lines, polylines, polygons, or curves connecting the anchor points, and color information for the lines, curves, and/or surfaces enclosed by the lines and curves. If a reference object sufficiently matches a structure contained in a high-resolution image frame, an object in the high-resolution image frame is then identified. An identified object is assigned the object ID of the matched reference object and relative object image insertion properties, including the image frame number where it was identified, an insertion vector point ("center of gravity" of the identified object), and scale/rotation parameters. The scale/rotation parameters typically include a scale value (the size of the identified object, typically relative to a stored standard size of the reference object), a first rotation value (rotation about a first axis, e.g., a vertical axis), and a second rotation value (rotation about a second axis of rotation, e.g., a horizontal axis). The object IDs and relative object image insertion properties of identified objects are transmitted via a second data channel from the onboard ADS unit to the control center over the mobile communications network. Transmission of this information also requires very little bandwidth.

From the second data channel, the identified (safety-relevant) objects can be reconstructed into high-resolution image data (pixels) in the control center. For this purpose, the control center also comprises a map database with reference objects, corresponding to the reference objects stored in the map database of the automatic train. In other words, the map databases of the automatic train and the control center contain essentially the same information about the reference objects. Based on the high-resolution object appearance information contained in the map database of the control center, the high-resolution object appearance for each identified object can be reconstructed for each image frame, based on the respective transmitted object ID, the insertion vector point and scale/rotation parameters, and the image frame number.

The reconstructed high-resolution object appearances, based on the second data channel information, can then be superimposed on the low-resolution image frame received via the first data channel with the corresponding image frame number. The resulting superimposed image frame then contains high-resolution image information where safety-relevant objects are identified and low-resolution image information where only the safety-irrelevant landscape of the train is present. In this way, for example, railway signals, including their switching status, which are reconstructed via the second data channel and displayed in high resolution, can be easily recognized by the remote driver, while at the same time, the remote driver also obtains an approximate impression of the landscape at low resolution.

It should be noted that the low-resolution image frames may be interpolated to the high resolution of the high-resolution object appearance at the control center to simplify the overlay procedure, according to the invention, but this does not increase the depth of information received via the first data channel.

The ATO's ground-based rail traffic management and operation center map database and the automatic train map database are regularly synchronized, so that both contain essentially the same reference objects ("common reference objects"). Note that in the simplest implementation, the map database does not require georeferencing information, although georeferencing information is useful for plausibility checks (see below).

Furthermore, please note that for remote control of the automatic train, the ATO's ground-based rail traffic operation and management center and the automatic train are connected via the management center's mobile router and the automatic train's mobile router through the mobile communication network, which may be a railway-specific mobile network or a public mobile network, or a combination of networks, e.g., GSM-R or 5G, 4G, or LTE. Communication includes, in addition to image information, for example, also driving commands and sensor readings.

Preferred variants of the invention

A highly preferred variant of the inventive process is characterized in that

on a third processing line,

- high-resolution image frames are subjected to a vectorization algorithm, which assigns vector elements, such as points, polylines, or polygons, to the high-resolution image frames, and determines the corresponding vector properties of each assigned vector element, including an image frame number, and

- the vector properties determined for each assigned vector element are transmitted via a third data channel from the on-board ADS unit to the train's remote control console via the mobile communication network,

that in the ATO ground rail traffic operation and management centre, the vector properties of the assigned vector elements of the third data channel are translated into vector element appearances,

and that the vector element appearances originating from the third data channel are used for plausibility checks and/or are included in the overlay of the high-resolution object appearances originating from the second data channel and the low-resolution image frames received via the first data channel with a matching image frame number.

In this variant, high-resolution image frames (pixels) are subjected to a vectorization algorithm in the ADS unit integrated into a third processing line. In other words, the structures found in the high-resolution image frames are abstracted. Note that this abstraction does not require defined reference objects, such as railway signs or railway signals, but can be based on general geometry, especially polygons. Consequently, the Map database is not required for this third processing line. The abstraction results in the assignment of vector elements, particularly points, polylines, and polygons. Polygons can contain color information, particularly for polylines and for surfaces enclosed within polygons. The information required to reconstruct an assigned vector element is/are the corresponding vector properties; These include the image frame number of the frame where the assignment was made, and information about the location and/or orientation and/or size and/or color of the points and/or lines and/or surfaces included in the vector element. Typically, the locations of all points included in a vector element are provided, with lines defined by their endpoints, and surfaces defined by their boundary lines, with color information provided for each line and surface.

The determined vector properties are transmitted from the on-board ADS unit to the control center via the mobile communication network through a third data channel, which requires only a small bandwidth. At the control center, the appearances of the vector elements can be reconstructed based on the transmitted vector properties. The appearances of the vector elements can be included in the image (i.e., the overlaid image frames) of the track section ahead of the train displayed to the remote driver, if desired. For example, an average of the reconstructed vector element appearance data and the reconstructed high-resolution object appearance data can be overlaid on the low-resolution image frames of the first channel. Note that, due to its vector-like nature, the appearance of the vector element is inherently high-resolution, although it is, of course, still the result of an abstraction of the original camera video data (high pixel resolution). Image details can remain visible/recognizable in the appearances of the vector elements that are not visible/recognizable in the low-resolution image frames. It is also possible to use the reconstructed vector element appearances for a plausibility check of the superimposed image frames resulting from the first and second data channel information. If there is an insufficient match between the vector element appearance (based on the third data channel) and the superimposed image (based on the first and second data channels) or the tits source data, the remote controller can be warned that no reliable image information is currently available. In both cases, increased reliability or confidence in the (regular) superimposed image frames or train safety can be achieved.

An alternative variant, also preferred, is characterized in that

on a third processing line,

- high-resolution image frames are subjected to a vectorization algorithm, which assigns vector elements, such as points, polylines, or polygons, to the high-resolution image frames, and determines the corresponding vector properties of each assigned vector element, including an image frame number, and

- the vector properties determined for each assigned vector element are also transmitted via the second data channel from the on-board ADS unit to the train's remote control console via the mobile communication network,

that in the ATO ground rail traffic operation and management center, the vector properties of the assigned vector elements from the second data channel of the third processing line are translated into vector element appearances,

and that the vector element appearances originating from the second data channel of the third processing line are used for plausibility checks and/or are included in the overlay of the high-resolution object appearances originating from the second data channel of the second processing line and the low-resolution image frames received via the first data channel with a matching image frame number. This variant corresponds to the previous variant, with the only difference that the determined vector properties are sent to the control center or its remote train console also via the second data channel, instead of a third data channel. In this way, the third channel can be done without; instead, the second data channel transmits somewhat more information, but still, only a little bandwidth is required by the second data channel.

Another preferred variant is characterized by

that the relative object image embedding properties of each identified object also include a georeference,

that the map database of the ATO and/or automatic train ground traffic operation and management centre also contains a register of known objects defined by a georeference and a reference object assigned in each case,

and that the ATO's land-based rail traffic management and operation centre and/or the on-board ADS unit verify the plausibility of the assignment of the identified objects to the reference objects based on a comparison of the georeferences and assigned reference objects of the identified objects, on the one hand, and the georeferences and attributed reference objects of the known objects, on the other hand.

In this variant, the map database in the automated train and/or control center also contains georeferences (geographic locations) of known objects and their object type (attributed reference object, typically represented by the object ID); the map database may also contain additional information about the known object, such as its orientation in space or its size, or a unique name, if desired. Objects identified in the second processing line are additionally assigned their georeference, i.e., their geographic location where they were found. The geographic location of an identified object can generally be calculated from the geographic location (usually known via GPS) of the automated train at the time of the camera recording, preferably corrected by an estimated distance of the object from the train for improved accuracy.

An identified object is considered plausible if it sufficiently matches a known object in a map database; a match can generally be confirmed if the georeferences of the identified object and the known object differ by less than a threshold (e.g., less than 3 m, depending on the accuracy of the train and object location) and if the object IDs of the identified object and the known object are identical. Note that for objects that may have different switching states (such as track switches or railway signals), all possible switching states are preferably included in the map database as separate known objects with the same georeference, for simple match verification. Alternatively, only a known object with the object ID belonging to a particular object switching state can be stored for the corresponding georeference, and an identified object with the assigned object ID belonging to any of the possible object switching states is considered compatible to approve a match.

If a matching known object has been found for an identified object, and the known object has been assigned a unique name (such as a signal number), the unique name of the known object can be displayed to the remote controller. If no match can be found for the identified objects in the known object register, the remote controller can then be warned that there is no reliable overlay image data available. Note that a match between identified and known objects is typically only checked (or required to avoid a warning message) for specific types of identified objects, generally those of particular relevance to railway safety, e.g., railway signals. By checking the plausibility of the identified object assignment, the reliability of the second processing line or the image data based on the second data channel or train safety can be improved.

In another preferred variant, the ATO's trackside rail traffic management and operation center verifies the plausibility of the assignment of the identified objects to the reference objects based on a correlation of the appearances of the high-resolution objects and the appearances of the corresponding objects in the low-resolution image frame to match the frame numbers. If there is no minimum conformity of the appearances of the high-resolution object and the corresponding structures in the low-resolution image frame, the object assignment is questionable, and the remote controller may be warned that a reliable overlay image is not currently available. With this variant, the reliability of the overlay image frames or the safety of the train may be improved.

A variant is additionally preferred which is characterized by

that the on-board ADS unit and the rail traffic operation and management centre ATO each comprise at least one UTC clock, preferably two UTC clocks, to provide UTC time,

that the communication between the auto-train mobile communications router and the management center mobile communications router through the mobile communication network is packet-based,

where each data packet is provided with a timestamp in a packet header section, indicating the UTC time of the packet being sent, which a Handshake protocol flow takes note of the UTC times as follows:

T<i s>UTC time when the packet was sent, T<1C>UTC time acknowledged when the packet was received, T<2S>UTC time when the acknowledgment was sent, and T<2C>UTC time when the acknowledgment was received,

that a round trip delay AT<IdayVuelta> is determined, with

AT Roundtrip = T<1>C-T<1>S+T<2>C-T<2>S,

that an average round trip delay ATIdayVuelta_AVG is determined over a plurality of passed packets, and that a warning message is delivered if

AT<IdayVuelta>-AT<IdayVuelta_PROM.>>T<UmbralRT>, or T<UmbralR T>: Threshold value for the round trip delay,

in particular with T<RTThreshold>=K<conf>*o(AT<RoundtripDelay>), where K<conf>: confidence estimate value; o: standard variance. If the data required for train control, in particular image data belonging to the track section ahead of the train, cannot be delivered in real-time synchronization, then the remote driver may not be able to react in a timely manner in order to ensure train safety. By applying the above variant according to a round-trip delay, delays in information or command transfer can be reliably determined. In case of rare or unacceptable delays, the remote driver can put the automatic train into a less dangerous state, for example by reducing the train speed or even stopping it. The warning message is typically delivered on the display of the remote train control console, for example by dimming the displayed image of the track section ahead, in particular if these are packets belonging to the first data channel or the second data channel. If the threshold value for round-trip delay is compared with the standard deviation of previous round-trip delays, common delays can be more easily distinguished from unusual delays. Alternatively, applying a fixed threshold for round-trip delay ensures a desired absolute safety level.

A preferred further development of this variant is characterized by

that a clock difference AT<clocksync> is determined, with

AT<clocksync>= 0.5*(T<1C>-T<1 S>+T<2S>-T<2C>),

that an average clock difference ATclocksync_AVG is determined over a plurality of passed packets, and that the warning message is also delivered if

ATclocksync- ATclocksync_AVG.>TUmbralCD, with TUmbralCD: Threshold value for the clock offset. If the UTC clocks on the automatic train and in the control center are not correctly synchronized, this can obscure and aggravate the identification of data transmission delays. By checking the clock offset, asynchronous UTC clocks can be identified, improving train safety.

Furthermore, within the scope of the present invention is a system for safe remote control of the train, comprising

a) for arrangement in an automated train:

- an on-board ADS (Automatic Driving System) unit, including a camera system for monitoring a section of track ahead of the automatic train, and

- an auto-train mobile communications router for forwarding and receiving data via a mobile communication network, and

b) for provision at an ATO (Automatic Train Operation) ground rail traffic management and operation centre:

- a remote train control console for remotely controlling the automated train, including a display for displaying an image of the track section in front of the train to a remote driver on the remote train control console, and in particular including a moving map display for indicating a train position of the automated train (ATR) and

- a mobile router of the management center for forwarding and receiving data via the mobile communication network,

characterized in that the system is adapted to perform an inventive method as described above. The inventive system enables safe remote control of an automated train via a mobile communication network, requiring only a low bandwidth. More specifically, an image or visual information of the section of track ahead can be provided on the display of the train's remote control console, containing high-resolution details about safety-relevant objects ahead (such as railway signals and railway signs) and a low-resolution overview of the future environment (such as vegetation and buildings on the ground).

Additional advantages can be derived from the description and the accompanying drawing. The features mentioned above and below can be used according to the invention, either individually or collectively in any combination. The mentioned embodiments should not be understood as an exhaustive list, but rather serve as an example for describing the invention.

Drawing

The invention is shown in the drawing.

Fig. 1 shows a schematic overview of a remote train control configuration, for the invention;

Fig. 2 schematically illustrates the ATO/on-board ground remote logic functions of the invention; Fig. 3 schematically illustrates an exemplary embodiment of a typical hardware implementation of the invention;

Fig. 4 schematically illustrates an example of train authentication, for the invention;

Fig. 5 schematically illustrates an example of channel processing, for the invention;

Fig. 6 schematically illustrates an exemplary embodiment of a remote train control display; Fig. 7 schematically illustrates a preferred variant of the inventive method for safe remote train control, in a first part in the automatic train;

Fig. 8 schematically illustrates the variant of Fig. 7, in a second part in the control center; Fig. 9 schematically illustrates a packet data transmission in a preferred variant for the invention;

Figure 10 schematically illustrates a preferred variant of packet transmission delay checking for the invention.

Overview

The invention relates to apparatus and methods for safe remote train control, particularly based on time-synchronized video transmission. More specifically, the invention relates to apparatus and methods for safe remote train control of a semi-automated or fully automated train that copes with the limited bandwidth of a mobile communication network.

Should automated trains need to be temporarily monitored and controlled in specific situations, the automated train will allow recovery situations to be handled from a remote control station. The inventive remote control system will adapt to public mobile networks, even when some temporary bandwidth limitations apply, and still operate safely and with high availability. Furthermore, remote train driving is seen as the introduction of GoA4's automated train capabilities.

The invention includes, in particular, a method for a remote control device for an automatic train having the following characteristics:

- Video images are transmitted from the train server via the mobile data link, so that the video I-frames are time-tagged and geo-referenced in the additional header bytes of the transmission protocol.

- Dynamic response of mobile bandwidth limitation so that on at least two channels connecting the automatic train with the remote unit, and one channel is using reduced spatial resolution and compression, and the other channel is using vectorization and symbolization of objects, areas and points of interest, and both channels are joined by a perfect superposition so that a realistic impression is given to the operator.

- A secure and robust time synchronization procedure between the remote device and the automated train by measuring message sending and confirmation times, so that any information freezes, including but not limited to video transmission or control packets, are reported to the operator within a time limit.

Safety-relevant remote connection monitoring is provided, which monitors relevant air-gap parameters and ensures integrity through online testing.

The invention in particular provides a secure and easy-to-use remote control and display unit.

Intended use and context

The object of the invention is part of the railway traffic operation infrastructure that allows automatic train driving up to automation level 4, which is defined as unattended train operation that includes all automated train operation functions.

The invention primarily focuses on remote control and driving functionality. In essence, the remote train control capability is intended to be used by an agent of the rail traffic operation and management center, for example, in the event that the automated train generates an alert, e.g., if a situation is recognized that the train cannot handle in automatic mode. Furthermore, control in the event of emergencies or degraded functional train systems is also subject to human intervention by the remote driving capability. Other examples are a routine automatic train supervision check or a mission readiness start check. The remote train control capability is intended to be used to safely bring a train to the end of the journey, rather than a driver boarding the train and manually continuing the journey. A further example: The intended use is that a train can be remotely started for a dedicated operating sequence, such as starting the mission after passengers have boarded the train until the predefined operating sequence is performed, such as stopping at the next station.

A typical automatic train drive control system, including the necessary infrastructure, is summarized as follows in Fig. 1: The ATO's trackside rail traffic operation and management center includes a remote train control console. This console allows a train to be controlled remotely via a mobile communication network. On the train, as an inherent function of the automated driving system, remote control capability is implemented. The remote train control/function is divided into two components, one in the train's on-board unit and the other in the ATO's trackside rail traffic operation and management center.

A summary of the high-level functions for remote train control is provided below:

a) Login and authentication for remote control of the pickup train or remote control of the release train; b) Remote access to the train driving functionality;

c) Monitor the actual status of the train as well as the diagnostic interaction capability of the train;

d) Secure, reliable and real-time communication, radio links and network management between the train and the remote control console.

Onboard remote train control

The train remote control function is divided into two components: one in the train's onboard unit and the other in the ATO's ground-based rail traffic management and operation center. Figure 2 shows the two main components, with typical functional entities.

Typically, the on-board automated driving system includes the following components:

O-A) On-board assignment and automatic train driving controller

This onboard function receives the trip assignment from the Ground Control Unit (ATO), triggers execution, and ensures that the operations defined in the assignment are scheduled and performed correctly. The component also sends task execution reports to ground control. In remote control mode, the automated driving system is disabled and remote control access is enabled.

O-B) On-board maneuvering controller:

The component has the following subfunctions: Time Table Speed Optimizer, which establishes the optimal speed to reach the stop or platform crossing points on time, subject to optimal energy usage. The Safe Profile Speed Optimizer establishes the maximum speed the train can travel without interfering with ETCS speed limits. The Automatic Stop Point Controller establishes the speed profile to accurately stop the train at the stop points. The final speed curve is calculated by considering timing points, signaling, maximum speed, obstacles, and environmental conditions.

O-C) On-board motion protection controller

This component is responsible for protecting the train from unauthorized movement, danger points, and overspeeding. This component also ensures safe opening and closing of the doors and safe coupling and uncoupling of the train.

O-D) Onboard Incident Prevention Controller: This subfunction monitors and assesses travel situations according to established railway rules, specifically for signaling and signing off on actions. The subfunction identifies the overall status and then initiates the required alert measures.

O-E) The perception controller accumulates the identification and properties of side signal monitoring, obstacle monitoring, collision monitoring, platform monitoring, and docking monitoring.

O-F) Onboard Train Interface Controller: Generates ATO output commands including ATO traction/brake control and various action commands including, but not limited to, emergency brake, door control, catenary control, head/tail light switch, coupling control, passenger car control. In addition, the subfunction reads train status and train messages. The function also provides interface to legacy train control systems such as NTC and ETCS.

O-G) Onboard Train Maintenance Controller: Diagnostic interaction, collects detailed status information for all subcomponents, provides legal records, and monitors the actual status of the train. Upon request, the various information records are delivered to the onboard reference data server and management system.

O-H) The integrated reference data server and remote controller processes and manages the data flow between onboard and ground, as well as for onboard functionalities. The function also includes configuration data and map data management. In addition, communication login, configuration, and authentication, as well as encryption for the onboard side, are supported. Remote command handling and data exchange are also possible.

O-I) On-board positioning and speed/distance measurement. This subfunction provides the train's position, direction, and speed, including safe confidence intervals.

Remote control of the train on the ground

The remote console allows remote access to the train.

T-A) The Remote Mission Manager provides input controls for the remote train driving capability, including selecting, starting, or aborting a train trip, or executing a semi-automated driving sequence. Additionally, the component translates input actions into commands. The functionality oversees mission execution. It also provides login and authentication for remote train pickup control or remote train release control. The Mission Manager allows various configuration parameters to be entered and managed so that the applicable subset is synchronized between ground-based and on-train objects, including, but not limited to, the track map database.

T-B) The remote train control manager provides low-level access to train driving control without going through the automated driving system. Braking force and tractive force input are directly accessible via controls such as a throttle lever. In addition, commands are provided for initial speed ramp to final speed, cruise speed, minimum coasting entry speed, and maximum coasting speed. Control buttons for engine start/stop, horn, wipers, lights, air conditioning, catenary control, front/tail light switch, door switch, or emergency stop, etc., are available as soft controls or toggle switches. In addition, the component translates input actions into commands; depending on the train model, this functionality can be configured to fit the TCS interface.

T-C) The Remote Diagnostic Manager requests the actual train status and diagnostic information, stores a history, and displays the information in appropriate menus. It records and monitors operational command actions and train status processes. It integrates and analyzes information from other onboard sources such as obstacle detection subsystems, side signal recognition, and passenger information subsystems.

T-D) The remote train location map manager displays train position information with full visual information of ground objects within the context of a track map situation.

T-E) The remote transmission control manager provides the functionality of starting, synchronizing and joining images for the different transmission channels.

T-F) Communication Client and Exchange Protocol Manager

The subfunction includes protocol protection of a secure protocol sent over 2 channels simultaneously and received with valid cryptographic protection such as, but not limited to, message digest hash (MDx) protection and/or cyclic redundancy check protection and replay request/handshake and sequence numbering and confirmed sequence number, timestamp and confirmed timestamp. The message validity check shall include

- Cryptographic protection security code check

- Message authentication check

- Checking the message type

- Checking the sequence number

- Punctuality check

Redundant train computer units (e.g., three units) will exchange control messages so that each computer has a tuple of at least 6 messages in the case of 3 units and 2 channels. A voting algorithm in each computer will compare the messages and, in case of dissimilarity, vote on which message failed, if any. The result of the failure is also exchanged among the redundant train computers, and any failed messages are ejected.

Data channel and exchange protocol between ATO-On-board and ATO-On-Ground

For example, it is a risk if a remote driver is operating control elements with an outdated video image of the driver's cab that does not represent the current frontal view of the train driver. The same applies to outdated states of the display elements, which can cause the train driver to react incorrectly. Therefore, secure transmission is mandatory for remote train control applications.

A key feature of the ATO Ground Remote Console is the transmission-based monitoring protocol data exchange. Figure 3 summarizes how transmission data is exchanged from various optical systems monitoring signaling, obstacles, collision events, the environment, the apron, onboard and passenger situations, and rolling stock and docking maneuvers.

Secure operation can only be achieved through time synchronization, low latency, and reliable transmission of information over communication networks. Redundancy across multiple physical channels allows information from one channel to be compared with that of another. However, sending the same information across dual channels, as well as switching channels from hot to standby in networks such as GSM-R, 3G, and 5G public mobile networks, are well-known techniques.

Because streaming video information from multiple sensors requires high bandwidth, which may not fit into a single channel, redundant sensor information is mapped to channels so that one sensor is associated with one set of channels and the other redundant sensor is associated with the other set of channels. Streaming refers to the purpose of timely information delivery rather than storing the information. Remote control includes a communication protocol that provides confidentiality, message authentication, and replay protection, as well as real-time capabilities for video streaming and control messages. Because the remote train driver console (ATO) needs to communicate with various types of automated trains, a standardized protocol, generally based on SRTP, will be used. Communication is assumed to be packet-based.

A multimedia session is defined as a collection of sessions, which may include a video session, a data session, or a command-control session. The payload data stream of any multimedia session is packaged and encrypted. The process of encrypting a packet consists of generating the keystream segment corresponding to the packet and then applying a bitwise exclusive-ORing operation of that keystream segment to the packet's data payload to produce the encrypted portion of the packet. Replay protection is achieved by the receiver maintaining a playlist, which contains the indices of all packets that have been received and authenticated.

The procedure includes a video image being sent in one or more packets. Packets exchanged between the Onboard ATO and the Ground ATO include a header containing the following information, in addition to typical data such as the packet number; the packet length; and a timestamp identifying when the packet was sent. In addition to the time the packet was sent, integrity can be improved by end-to-end time synchronization with time delay control of the video transmission between the Onboard ATO camera and the Ground ATO remote control console display. Video images are transmitted such that video I-frames are time-tagged and georeferenced in the additional header bytes of the transmission protocol. Additional frame synchronization in the form of a one-byte header extension is appended to the first packet, allowing for up to 15 additional bytes. The header extension byte containing an ID and a length is used. These following bytes are used to encode the UTC time when measuring the data. The UTC time is encoded as a counter in milliseconds. The static I-frame identifier adds to the robustness of video decoding because the I-frame is a key frame to which all other P- and B-type frames refer. These bytes are also used to georeference the video image. For example, the 7-byte UTC time and the 8-byte georeferencing, including the x and y coordinates of the pixel (0,0) in the top left corner. A scale factor is used for georeferencing in WGS84 latitude and longitude coordinates: Lat<Ascale> = f<S>*Lat<Packet>. The video camera properties relevant for image processing are fixed configuration parameters, known to ATO-On-Ground as well as ATO-On-Board, including but not limited to pixel scale factors for width and height, ATO-On-Board camera mounting position, rotation, and focal length.

Secure data connection with dual streams for low-speed control data using a secure protocol. For video data compression, the number of key frames (I-frames) varies depending on the train speed. Each video frame is enhanced with a pattern that correlates the time encoding of a global time source and the frame number. On the remote display, each video frame number is decoded and compared with the global time source so that the transmission delay time difference relative to the video captures the time and the time of the ATO system on the ground. With this procedure, video delay tolerance can be monitored.

The time encoding by the header extension, as well as the packet transmission time, can be compared to the real-time clock of the ATO-Ground computer, where the real-time clock is also synchronized to UTC. The time difference between the video packet header timestamp and the real-time clock is compared to a threshold, as are the timestamps from the video image capture. If the threshold is exceeded, an alert indication is generated when both the timestamp and the camera timestamp are sent. This real-time double-check at the packet level is much more efficient and secure than indicating an empty transmission buffer.

Physical Layer of Remote Train Control

Various sensors are connected to the on-board ADS computer, including cameras, positioning sensors, etc. The physical implementation of communication between the ATO-Onboard and ATO-Ground remote train control network is based on mobile communication routers. The router forwards data over shared or public networks via any type of network channel or any combination of channels such as, but not limited to, GSM-R, Mobile Telecom 5G or 4G, LTE, or WLAN.

Bandwidth and round-trip delay are measured online on all channels, and routing selection is dynamically adjusted based on bandwidth. In particular, video information consumes the majority of bandwidth. Therefore, a balance is required between optimal channel selection and bandwidth reduction for data transfer between the on-train ATO and the ground-based ATO remote control console, ensuring that neither integrity nor operation are compromised.

Taking or releasing remote control of the train

As stated in EP 3220613 A1, if an onboard ATO ADS initializes communication on the packet-based communication network, the ADS will publish its IP address, the symbolic mission name (the train running number may be a part of this mission name), and the ATO Onboard hardware ID of the train. The same procedure is used by the ATO Groundside remote console, which publishes the fixed IP for each remote driver location, including a destination transport address pair. It is assumed that the IP communication setup can be established via known IP addresses or, given that a DNS service is available, also via name resolution. Acquisition by the remote controller will generally include the selection of a specific train ID or mission name, as well as the input of various authorization data into the remote train controller console of the ATO's rail traffic operation and management center, including, but not limited to, user credentials, mission authorization, and train authorization.

For the packet-based protocol, a preferred embodiment is to extend a public standardized protocol such as the Real-Time Streaming Protocol (RTSP) to improve the integrity of the data transfer. However, the procedure can also be applied to a proprietary protocol. For ATO-Onboard and ATO-Ground session setup, it is necessary to exchange keys. The protocol typically includes a master key, from which the session keys are identified. Further authentication must be successful before data encryption takes place on the sender side and vice versa on the receiver side. RTSP exchanges are encrypted to protect the key exchange itself. If the media server detects that encryption is enabled (based on RTSP parameters in the URL), it sends the key management information and then the encrypted stream. Fig. 4 illustrates an example of stream authentication.

Air-gapped mobile router and switch

Remote airspace information will be available to diagnose the connection between the On-Board ATO and the Ground ATO, such as SSID modes, IP connections, addresses, operator status, connection type (LTE, 5G), channels, International Mobile Equipment Identity, International Mobile Subscriber Identity, Cell ID, active protocols and services, authentication status, etc. An event log will allow checking for relevant connection or configuration events.

Time synchronization between the Ground-based ATO remote control console and the On-board ATO train is a key constraint for a reliable and safe control loop. Time delays introduced into the control loop by insufficient transfer bandwidth or insufficient time synchronization between the remote station and the train can cause control instabilities or safety risks. A secure time base is established in the On-board ATO with at least two independent computers having a stable UTC clock; as a preferred embodiment, this could be GNSS receivers or computers connected to GNSS receivers that have time synchronization access better than 100 microseconds. In data exchange between the On-board ATO and the Ground-based ATO remote control console, preferably for control messages, where an order acknowledgment protocol is mandatory, the round-trip time is measured to ensure that the round-trip delay is less than a certain limit that depends on the control loop filter time constant. If the round-trip time limit or clock synchronization limit is exceeded, an alert is issued, as security may be compromised.

The packet round-trip delay is continuously determined by two independent primary stratum UTC-synchronized train computers communicating via at least two mobile communication network data channels and received from at least two independent remote control console computers. For each time epoch, at least 2A4 round-trip delays can be determined.

The on-wire protocol header addition uses four timestamps that are embedded in the overall protocol flow of command/request sent by one unit and response/acknowledgement by the other unit, the round-trip time being determined by both the on-train ATO component and the ground-based ATO remote control console. Each message contains both the timestamp and the acknowledged timestamp in the message header. The timestamp Ts is a four-byte value, containing the sender’s message timestamp. The acknowledged timestamp T<c> is a four-byte value, reflecting the partner’s timestamp of its last (youngest) valid message.

T<i s>is the clock time when the packet was sent, T<i c>is the clock time acknowledged when the request/order packet was received, T<2S>is the clock time when the acknowledgment was sent, and T<2C>is the clock time when the ACK was received and acknowledged.

The four clock time measurements are used to delay the round trip

AT Roundtrip = TiC-Tl S+T 2C-T 2S

The systematic part of the round-trip delay is eliminated using a moving average of the window. The threshold can be determined using confidence estimation. An alert is generated if the round-trip threshold is exceeded:

ATldayReturn-ATldayReturn_PROM. >ThresholdRT Kconf ■ Sigma(ATldayLap)

The difference between the Ground and On-Board clocks is determined as:

AT syncclock = 0.5*(TlC-TlS+T2S-T2C)

An alert is generated if the synchronization threshold is exceeded:

AT syncclock-AT syncclock_PROM.>T CD Threshold

In addition to round-trip delay, other statistical properties of the data channels will be monitored, such as, but not limited to, bandwidth utilization (RX rate, TX rate), bit errors, received reference signal energy, received signal quality (reception strength/reference signal), and signal-to-interference-plus-noise ratio. Monitoring of these indicators is performed on a statistical basis, with a sliding window limiting the sample set length, which is reset upon each new mobile cell takeover. Central moments such as mean, variance, and skewness are tracked, and test statistics, as for hypothesis testing, can identify any failures or degradation of the remote wireless connection. Integrity is represented by a confidence interval of a Gaussian probability density distribution, which is based on the statistical properties of the variance at a mean time, as well as a confidence inflation coefficient KFF, is linked to the Gaussian distribution and is calculated using the Quantil QN function of the associated integrity risk.

Since the remote air-gap channel's characteristics are within tolerance, the channel is established as healthy and can be used. A counter checks if any channel is empty for a given time and generates an alert if the time limit is exceeded.

Video image processing

The Data Reference Server and Remote Controller function processes and manages the flow of data between onboard and ground, as well as for onboard functionalities. Data flow includes general data management, including the transmission of video, audio, status, and command data. This function is responsible for distributing the designated information flow to ground entities, including the train's remote control console.

Secure video processing and compression is a key function of the data reference server and remote controller. The precursor to multimedia data transmission is data compression by reducing resolution and color depth. The camera image pixel stream is then compressed using state-of-the-art image-level redundancy removal procedures, such as run-length coding or discrete cosine transforms. In the next stage, the image is compressed by a codec-encoder that covers the relevant redundancy between images. Long-term prediction encoded in P and B frames compresses changes with reference to intra frames (I frames).

For further bandwidth reduction, the inventive method is proposed: Mobile bandwidth limitation response such that on at least two channels connecting the automatic train with the remote control console can cope with the bandwidth limitations such that one channel (called channel 1 here) is using reduced resolution and the other channel (called channel 2 here) is using a vectorization and symbolization of objects, areas and points of interest. Both channels carrying different information are joined by a perfect superposition such that a realistic impression is given to the operator (see Fig. 5).

The train's onboard map, as well as the remote control console map, inherit identical layers, where operationally relevant points of interest (signals, signs, overhead pylons, etc.) are geocoded. In addition, the procedure includes a map database section, where, in addition to the object vector characteristics, a typical object appearance image is stored. On channel 2, only the object ID and the relative object image embedding properties, such as frame number, embedding vector point, georeferencing, and scale/rotation factors, need to be transferred.

After receiving the object ID at the ATO-Remote Control console, the typical object appearance image is correlated with the expected insertion area from the video image, where geocoding and the object insertion vector guide the expected insertion location. This high-resolution object appearance image from the map database is overlaid on the low-resolution video image in such a way that the disadvantage of reduced video image quality is compensated by overlaying the object's typical appearance pixel areas. This is important, for example, for detecting signal light status, where high resolution is necessary. For the vegetation image, a low resolution is still acceptable without compromising safety or operational decisions. In addition, the vectorized symbol from the map database is overlaid on top to highlight relevant elements.

In the ATO Ground remote control component, the overlay may be processed by resampling the low-resolution video image to a high-resolution image and then stitching the overlapping object appearance images together. The image stitching operation includes options such as multiplying the overlapping object appearance image intensity values by the video intensity values, adding both intensity values, averaging the image values, or maximum contrast stitching. In one embodiment, for dark areas of the video image, the pixel choice is based on the minimum of the video image and the object appearance image, and for bright areas, the maximum intensity of the object appearance image is used for the overlay.

In order to identify known objects, video segmentation preprocessing on the train requires video images to be prepared. Classifying the different objects in the image is one of the stages in the processing chain. The characteristics of the image objects are calculated and then compared using feature detectors/classifiers. Feature extraction reduces the information to be transferred by extracting regions, edges, and corners. Successful object identifications are marked with a bounding frame and an object ID. In the example shown in Fig. 5 and Fig. 6, from the front view of the remote camera, the objects on the track overlap, a train is identified on the opposite track, and a person is identified in front of the front track.

To ensure the integrity of the graphic elements, a validity code (cyclic redundancy check or comparable) is added to the generation source. This code includes the time information and, in the case of video images, the frame number and the identification number, which correspond to the common map database between the Ground-based ATO and the Airborne ATO, the insertion vector, and the scale/rotation factor. On the remote display, the pattern is decoded in such a way that the integrity is validated, and the time difference is compared with the real-time clock.

The ATO On-Board and ATO remote control map database has at least two main parts: vector data and reference symbol icons. In the vector part: The vector map part includes railway infrastructure areas such as restriction areas, stop areas, level crossings, tunnels, bridges, platforms, vegetation, etc. The database information for those infrastructure areas includes at least an ID, a bounding frame (width, height, length), track under infrastructure, line name, associated properties, and a link to a pixel image, representing a typical area or the specific area including the object area reference point (CoG center of gravity), object pixel height/width. The geometry of the area is represented by a vector element such as a point, a plot line, or a polygon area. The geometry type and the related number of geometry coordinates (e.g., latitude, longitude, altitude or segment, abscissa, offset) are stored in the database. A polygon is defined as a connected and closed sequence of four or more points, while a plot line is defined as a sequence of lines with two or more points. In addition, the database map stores track assets such as, but not limited to, signs, signals, dirt markings, pylons, and markers. For markers, in addition to the properties mentioned above, extended link information is stored, including the vector between the marker origin and the track segment, as well as the BG identifier (Nid_BG); the marker's position within the marker group; and the arc length from the segment's start.

Additionally, databases store track geometry as a series of points, or higher geometric abstractions such as, but not limited to, polylines, curves, clothoids, splines, etc. Due to the fact that track location needs higher geometric precision, properties may include in the preferred embodiment, for example, point ID, associated segment ID, line ID, point heading, point grade, point superelevation. Depending on the chosen geometry type representation, properties may include center point and radius, or start point end point, etc. Track topology is represented by edges and edge connections that represent track elements such as switches, buffers, or junctions.

The second part of the map database contains dual-layered landmark icons: one as a vector layer and one as a pixel layer, both representing the same object. These landmark icons can be stored specifically for a country, region, or trip. The database also contains other configuration information such as, but not limited to, lever arms, system properties, IP addresses, and so on.

Plausibility checks are part of the verification process, including whether the object type and location in the map database match the identified object type and location recorded in the video image. Furthermore, the correlation of the object's reference image and the low-resolution pixel image is a means of verifying whether the object identification was correct. Completeness is achieved by calculating the correlation between the icon's pixel image and the low-resolution video image on the remote side, across the entire pixel width and height. Since the correlation peak is within the object's bounding frame, the process was a match; otherwise, an alert is issued.

Furthermore, the reference image can be reduced to the object's shape using image processing techniques (e.g., the Sobel operator) and adapted to the object's vector shape. All of these checks are intended to detect errors. Given that all three data sets (low-resolution, database reference image, and vector image) match, or at least two out of three match, the transmission content is validated.

Remote identification of train positioning status

When using the remote train control function, the operator must know where the train is actually located on the track network, as well as where the train identified its position. In the event of a system failure, these two positions do not always coincide. The remote control functions provide a visual indication of the front view of the train, as well as a moving map view. The remote operator can identify if discrepancies appear for significant landmarks. The train's position is graphically represented on the moving map as a symbol. Further along the moving map are relevant objects with associated labels, such as markers, track segment edges, signals, signs, and landmarks. In addition, the segment ID and the distance traveled from the segment ID to the calculated train position are also displayed to allow the remote operator to cross-check whether this is the location where the train is expected to be based on the travel profile.

ATO train positioning is generally referenced to the origins of the tracking segments instead of the beacon group according to ETCS. Therefore, the distance traveled by the train is provided with respect to the segment profile identifier, the segment profile version, and the segment profile travel direction. In one embodiment, each track segment identity is defined by the SP Country or Region Identity (NID_C(k)), as well as the Requested Segment Profile Identity (NID_SP(k)), together with a status ("Valid": Requested SP, "Invalid": SP not found in the ATO-Ground database) and version number. Each segment is characterized by a Length of the rail segment covered by the segment profile L_SP(k), as well as the altitude as reference to the start of the SP, and a gradient G_New_Gradient(k), defined as the value of the new gradient at the start of the Segment Profile. Other embodiments may use only a subset of the properties or additional properties. In order to verify this information to achieve a certain level of security, the location of the segment profile must also be defined by a geographical position, for example, in latitude, longitude, and altitude.

Positions within the ATO-On-Board are typically defined relative to the start of the Segment Profiles. The journey profile lists the sequence of track segments the train is supposed to travel for a given route. When using ETCS, the train position refers to the signal groups and the distance traveled. Therefore, to convert the train position from the signal group reference to the track segment reference, the segment profile must incorporate signal information, including, but not limited to, the BG identifier NID_BG; the signal position within the signal group; and the signal location as an arc length from the start of the segment. To verify this information, the signal location must be defined by a geographic position, e.g., latitude, longitude, and altitude, in addition to the reference to the start of the track segment. The problem is that the segment profile generally does not achieve the level of safety necessary for the required operation, unless extreme effort is invested in the map generation process. To facilitate this restriction, comparing the distance of the marker location relative to the track segment origin against the difference in geodetic coordinates between the track origin and the marker location allows for the detection of errors and inconsistencies in the map. For example, if the marker position on the track does not match its geodetic position within a certain error threshold, these errors can accumulate, and error propagation is difficult to detect when it falls below a certain threshold.

The stored beacon position can be compared with the measured beacon position.

Since the segment is already present in the ATO-On-Board map database with the correct version, the information will be taken from this on-board map. Otherwise, the track segment is requested using the segment profile request packet. The consistency check for segment and trip profiles includes consistency checks such as waypoint versus trip/segment profile, ascending order of waypoints, consecutiveness, and completeness of segment profiles.

Remote identification of train control and management status

The status and output results of the automated driving controller will be available on-screen in a human user interface, along with output data and error indications. The remote control console issues a self-diagnosis request, and the integrated component will report a self-diagnosis result. The output list for identifying and controlling the applicability of rules for the various subsystems, including, but not limited to, task assignment, trip and schedule management, and assignment execution monitoring, will be displayed in the self-diagnosis function. In addition, the train diagnostic capability allows the on-board configuration to be displayed. The version and status of key components and database configuration items can be verified.

The self-diagnostic status and output results of the onboard automated driving system, collected by the train maintenance controller, will be available on-screen in a human user interface with output data as well as error indications. Monitoring of the journey assignment execution will report an assignment forecast to the remote console assignment scheduler. A listing report of the executed Waypoint/Timepoint will be available for diagnostics. In particular, relevant data such as the journey profile indicating the protocol status, with ATO packet errors, timestamps and header data, as well as the assigned journey with past and upcoming time points (arrival time, departure time, dwell time, station, door open side), and temporal restrictions (speed restrictions, low adhesion) will be available. In read access mode, the remote train console can check the current parameters and history of functional inputs, such as

- ATO-TMS telegrams including time, travel profile data, exit authorization;

- ETCS telegrams, including executed and authorized movement authorities, speed profiles, position reports, etc.;

- Train movement and position outputs, including the timetable speed profile, the monitored speed envelope profile, and the automatic train stop profile.

In addition, the remote driver will have control over mission data, for example, by rescheduling or rerouting the train. Updating the assignment task and forecasting the assignment execution will be part of the remote mission manager's role.

The self-diagnostic status and output results of the maneuver controller will be available on-screen in a human user interface, along with output data and error indications. The maneuver summary list, as well as the summary output list of TCMS commands, will be available for viewing.

The self-diagnostic status and output results of the Motion Protection Controller, which validates incoming train infrastructure data before providing it to the Speed Profile Optimizer, will be available for display in a human user interface with output data and error indications. A summary of track data is provided, including profiles of actual and upcoming track segments, as well as relevant rail infrastructure data, including gradients, curves, de-energized sections, static speed restrictions, and marker IDs. The display option allows navigation through the track profile, including object properties. Furthermore, it is also capable of reporting map object properties; for example, marker properties, consisting of marker ID, location, number of markers in a group, and so on. Relevant specific information is found in data elements for segment profile details, such as platforms, tunnels, level crossings, and markers. Thus, the remote driver can identify issues related to the track input data, where the system may be in a fault condition and the remote driver needs to resolve it, before returning to automatic mode. The remote driver must have a means to clear errors or alerts, as well as to manually override the output data from the motion protection controller.

The self-diagnostic status and output results of the perception controller will be available on-screen in a human user interface with output data as well as error indications. In particular, the results of identified railway infrastructure objects, such as, but not limited to, switches, stations, tunnels, bridges, catenary pylons, signals, and signs. The results of this perception controller, based on obstacle detection and classification, environmental monitoring, and side signal detection, will be included in the self-diagnostics. The probability of detections, as well as classification attributes, will be available in menus so that the remote driver can identify misclassifications or failures. In read access mode, the train's remote console can check the current parameters and history of all functional inputs and action outputs of the automated driving functionality, such as

- Pattern recognition outputs including signal, sign, obstacle detection, platform and environment;

- On-board situational awareness sorties.

Additionally, the rail load meter clearance and any violations of unknown objects in this area will be available on the remote GUI screen by ID, position, and attributes. The remote driver will also have the ability to override the results or clear errors or alerts.

Train diagnostic interaction capability

The train's remote console enables train diagnostics, providing the relevant status of train components. It also provides access to the train status and diagnostic database covering train component faults (e.g., the traction unit, doors, and brake system); the remote console will access a diagnostic function that detects faults associated with the automatic driving function. The diagnostic display message will include at least the following information:

- ID: Diagnostic ID, which is unique within a maintenance interval;

- Timestamp of occurrence;

- Type classification: Information, Warning and Fault;

- LRU: Component/function to which the diagnosis is associated;

- Diagnostic content, which may include a description of the fault or an error string.

Although the train status and diagnostic database is manufacturer dependent, it typically contains items such as, but not limited to, status data for:

- Catenary status;

- Catenary voltage;

- On-board power;

- Energy and traction status;

- Brake pressure;

- Brake reservoir pressure;

- Brake status of various brake systems;

- Oil pressure;

- Engine RPM;

- Wheel speed;

- Engine temperature, gearbox, cooling system;

- Condition of the doors;

- Climate comfort;

- Coupling state;

- Condition of the rolling stock.

Remote train driving capability

The remote train control device includes a computer graphical user interface (GUI) within the ATO's rail traffic operation and management center, capable of displaying information and receiving human input through, for example, a pointing device, a keyboard, or a touchscreen device. In addition, critical control devices are separately equipped, such as, but not limited to, the traction/brake lever, the alert button, or the operator monitoring button. The remote train control device is also equipped with audible devices, for example, a headset and microphone, to enable communication with the train or passengers.

Safety is achieved by providing all train statuses online and in real time, including standard driver DMI statuses, additional statuses a driver must collect through maintenance investigation, and track-related information. Comprehensive operational information, tailored to the information a driver experiences in the locomotive cab, is vital to safe operations. The remote train's cab-like appearance enables a smooth and intuitive workflow.

The graphical user interface presents all necessary information to the remote train driver. A preferred embodiment of the human-machine interface functionality may include menus or display elements/modes as follows:

a) Dialog elements for user authentication and login/logout;

b) Dialog elements to start/stop a train journey or execute sections of a train journey;

c) Display elements for monitoring the status of the train with status warning in case of a critical situation;

d) Display elements of the train subsystem states, such as, but not limited to, train speed, brake pressure, catenary voltage, motion control status, etc.;

e) Display element for wireless remote connection properties;

f) Display elements for train and ATP mode (e.g., full ETCS or PZB monitoring); g) Configurable display elements for specific purposes depending on the train model;

h) Control elements for train control, such as, but not limited to, train on/off, train acceleration/driving lever, braking lever, catenary control, front/rear light switch, door switch, etc.;

i) Control element for emergency stop and horn;

j) Configurable control elements that are represented by a specific device (button, lever, switch);

k) Configurable control elements that are part of a graphical user interface, such as a touch screen;

l) Control elements for specific purposes of ETCS or STM;

m) Elements for checking train protection or for the cabin signalling system (inductive or radio-based);

n) Video display of the driver's cabin view (optional in the selection);

o) Track map with train position and position data;

p) Video screen with view of the train driver's cabin instrument;

q) Menus from other sources such as obstacle detection subsystems, side sign recognition, passenger information subsystems;

r) Control element for the surveillance or alertness of operators.

An exemplary embodiment of the remote driver display shown in Fig. 6 includes the appearance of an extended ETCS display with the first-level groups: control elements, ETCS display elements, train status elements, moving map display, front camera display, and driver cab view. Other display elements from the above list (a to r) can be organized in a second dialog level or displayed upon menu selection. Optical and audible redundancy is provided for critical elements when information requires operator action or is outdated. For example, an alert sound or a standby/idle icon with a grayed-out menu is used if the information freezes or becomes outdated.

Since the information is not up-to-date due to exceeding a time threshold, a waiting pattern is displayed to warn the operator. A preferred embodiment of the icon or waiting pattern can be, for example, a progress/loading bar or a grayed-out image to intuitively display outdated information. This waiting pattern is used in addition to the timestamp on the displays. During initialization, the display elements are run through a visual check, where all states are shown in sequence, or a check pattern is placed on the displays. The train's remote control console provides functionality to execute or manage actions such as emergency stop, power cut, passenger recall, or live incident handling.

Discussion of preferred variants

Fig. 7 illustrates a preferred variant of the inventive method for safe remote control of the train, with respect to the part that takes place in an ATR automatic train.

A CAM camera system installed at the front of the ATR automated train continuously monitors the section of track ahead of the train (or its leading locomotive/car) and generates the corresponding HRI image frames. These HRI image frames have a high resolution to resolve all the details necessary for safe remote control of the train. The high-resolution HRI image frames are then processed in three processing lines: PL2, PL2, and PL3.

In the first processing line PL1, the high-resolution HRI image frames are converted into low-resolution LRI image frames. Typically, the number of pixels is reduced by a factor of 4 or more in each image direction, thus reducing the number of pixels by a factor of 16 or more. Additional compression and coding techniques are applied. The low-resolution LRI image frames are transmitted via a packet-based mobile communications network (MCN) via a first data channel CH1 to a ground-based rail traffic management and operation center for Automatic Train Operation (ATO), also simply called a control center (see Fig. 8). Due to the low resolution, only a small bandwidth is required for image transmission in the mobile communications network (MCN).

In the second PL2 processing line, the high-resolution HRI image frames undergo pattern recognition. If the patterns/structures recognized in a high-resolution HRI image frame sufficiently match the reference objects stored in an ATR automatic train's MDB map database, the objects are identified. For each identified object, the object ID of the corresponding reference object in the map database is determined, as well as the relative object insertion properties. In the example shown, a railway signal, a flashing train control, and a section of track have been identified as objects. Note that the reference objects represent typical ground-based objects, which can be found in large numbers along tracks. For each reference object, (high-resolution) appearance information is stored to enable comparison with the structures found in the high-resolution image frames. This appearance information is typically 3D vector information. The relative object insertion properties indicate the position of the identified object in the HRI high-resolution image frame, as well as its size and rotation.

In the variant shown here, the MDB map database also contains a register of known objects, where each known object is defined by an object ID of a corresponding reference object (indicating the type of known object) and a georeference where the known object can be found (i.e., indicating the geographic location of the known object), as well as a unique name for the known object. In the example, the railway signal identified as an object also matches a known object from the register with the unique name "Signal 0158," and the train flashing control identified as an object also matches a known object from the register with the unique name "PZB 0739." Note that the track identified as an object does not match a known object here.

All identified objects are placed in an LIO list, which contains the object IDs and relative object insertion properties, as well as the frame number of the (high-resolution) image frame where the objects were found. In the example shown, the LIO list also contains information about assignment to known objects, where applicable. The LIO list is sent via the mobile communication network (MCN) through a second data channel (CH2) to the control center (see Fig. 8). This requires only a small amount of bandwidth compared to transmitting image frames.

In the third PL3 processing line, the high-resolution HRI image frames are subjected to a vectorization algorithm. The striking structures in the high-resolution image frames are translated into vector elements such as polylines and polygons, and their vector element properties, also simply called vector properties, are determined. The appearance and position of the assigned vector elements are defined by means of the vector properties.

All assigned vector elements are placed in a LAV list, which contains the vector properties of the assigned vector elements and the frame number of the (high-resolution) image frame where the vector element was found. The LAV list is sent via the mobile communication network (MCN) through a third data channel (CH3) to the control center (see Fig. 8). This requires only a small amount of bandwidth compared to transmitting image frames.

Fig. 8 further illustrates the variant of the inventive method for the safe remote control of the train of Fig. 7, with respect to the part that takes place in the control center COC.

Via the first data channel, CH1, the low-resolution LRI image frames are received by the ATO rail traffic management and operation center (COC), also referred to simply as the COC control center. The COC control center is stationary, e.g., integrated into a building, and comprises a remote train control console (not shown here) from which a remote driver can control the automated train.

The list of identified objects (LIO) is received via the second data channel (CH2). Based on a map database (MDB) from the control center (COC), in particular the reference object appearance information contained therein for each reference object identified by its object ID, the information in the LIO list is used to reconstruct the (high-resolution) HROA object appearances of the identified objects. For this purpose, for each identified object listed, its object ID is used to read the corresponding reference object appearance information, and relative object image embedding properties are applied to properly scale, rotate, and position the object.

The list of assigned vector elements (LAV) is received via the third data channel, CH3. The information in the LAV list is used to reconstruct the (high-resolution) vector element appearances of the assigned vector elements.

The high-resolution object appearances (HROAs) originating from the second data channel CH2 are then superimposed on the low-resolution LRI image frames, resulting in OLI overlay image frames. The OLI overlay image frames are then displayed on a display in the remote train control console of the control center (COC) to the remote driver. The high-resolution object appearances HROA inserted into the OLI overlay image frames have sufficient resolution to enable the remote driver to recognize all information relevant to safe train control, such as the switching status of the railway signal "Signal 0158" here. In addition, the remote train driver can obtain an overall impression of the landscape in front of the train by means of the underlying low-resolution LRI image frames of the OLI overlay image frames.

Additionally, VEA (Vector Element Appearances) can also be overlaid on the OLI overlay image frames to enhance the detail of the OLI overlay image frames (not shown here). Alternatively, and in the example shown, VEA (Vector Element Appearances) are used to check the plausibility of HROA high-resolution object appearances before overlaying with the low LRI image frames. In the example, all identified objects from data channel CH2 are matched to vector elements from data channel CH3. Therefore, the OIF overlay image frame is displayed normally. If not enough matches are found, a warning message is displayed indicating that the overlay image frames may be corrupted.

Note that in another variant, both LIO and LAV lists can be transmitted via the mobile communication network MCN via a common data channel (i.e. the second channel CH2, whereby the third data channel CH3 is carried out without continuation), if desired (not shown in more detail).

Figs. 9 and 10 illustrate methods that may be applied to verify timely provision of image data in the course of the present invention. In particular, the method may be applied to transmit data packets pertaining to image data via channels CH1, CH2 and/or CH3 in Figs.

7 and 8.

Fig. 9 illustrates packet data transmission or a corresponding protocol flow for the present invention.

The ATR train wants to send a DP data packet to the control center. For this purpose, an ATR-MCR mobile communications router of the ATR train sends the DP data packet to the mobile communications network (MCN). When the DP data packet is sent, the time T<1>S is read from a UTCC of the ATR train's UTC clock and noted; the T<1>S is included in a header of the DP data packet (first/top image).

The COC control center receives the DP data packet via its mobile router COC-MCR. At this point, the UTCC time T<1C> is read and recorded from the COC control center's UTC clock (second image).

The COC-MCR mobile router of the COC control center then sends an acknowledgment ACK, for which the sending time T<2S> is read from the UTCC of the COC control center's UTC clock and noted. The time T<2S> is included in a header of the acknowledgment ACK (third image).

Finally, the ACK acknowledgement is received by the ATR-MCR mobile communications router of the ATR train. The time T<2C> is then read from the UTCC of the ATR clock and recorded (fourth image/bottom image).

Knowing T<1 S>, T<1 C>, T<2 S> and T<2 C>, a round trip delay AT<IdayBack>=T<1 C>-T<1 S>+T<2 C>-T<2 S> can be calculated. From other/past data packet sending events, an average round trip delay AT<IdayBack_AVG> has also been determined. Furthermore, a clock offset AT<clocksync>=0.5*(T1<c>-T1<s>+T2<s>-T2<c>) can be calculated. From other/past data packet sending events, an average clock offset AT<clocksync_AVG> has also been determined.

As illustrated in Fig. 10, for a particular piece of image data sent or received, compare the data packet DP of Fig. 9 above, the round trip delay AT<RoundTripDelay> minus the average round trip delay AT<RoundTripDelay_AVG> is compared to a threshold value T<RTThreshold> for the round trip delay. Preferably, the threshold value T<RTThreshold> for the round trip delay is chosen based on a standard deviation of past round trip delays, typically by applying a safety factor (also called a confidence establishment value).

If AT<IdayVuelta>-AT<IdayVuelta_AVG.>>T<UmbralRT>, then a warning message is displayed to the remote driver, indicating that the displayed image frame based (at least partially) on the respective piece of image data is not (or at least may be) up to date.

Furthermore, in the illustrated example, the clock difference AT<clocksync>minus the average clock difference AT<clocksync>_<AVG>is compared to a threshold value T<CDThreshold>for the clock difference.

If AT<syncclock> - AT<AVG_syncclock> T<CDThreshold>, then the UTC clocks of the automatic train and the control center are not correctly synchronized, which may distort the true round-trip delays. Consequently, the respective image data cannot be guaranteed to be up-to-date, and a corresponding warning message is displayed to the remote driver.

If the checks do not reveal a risk of delay in the delivery of the image data, the image frame based (at least partially) on the respective piece of image data is regularly displayed to the remote driver, without a warning message.

In summary, the present invention proposes a safe remote train control of an automated train, wherein the track section in front of the automated train is observed with a video camera system that produces high-resolution image frames. High-resolution image frames are converted into low-resolution image frames and transmitted via a mobile communication network to a remote train control console at a ground control center via a first data channel. Furthermore, the high-resolution image frames are subjected to pattern recognition, and for identified objects corresponding to reference objects in a map database, reconstruction information is transmitted via the mobile communication network to the remote train control console via a second data channel. At the remote train control console, the high-resolution object appearances for the identified objects are reconstructed using the reconstruction information and a corresponding map database. High-resolution object aspects are superimposed on low-resolution image frames received via the first data channel and displayed on the train's remote control console. Thus, both safety-relevant details and an overall impression of the track section are made available to a remote train driver with low bandwidth consumption of the mobile communications network.

List of reference signs

ACK acknowledgment

ATR automatic train

ATR-MCR mobile communications router (on automatic train)

CAM camera system

CH1 first data channel

CH2 second data channel

CH3 third data channel

COC automatic train operation ground rail traffic operation and management center/control center

COC-MCR mobile communications router (in control center)

DP data packet

HRI high resolution image frame

HROA high resolution object appearance

LIO list of identified objects

LAV list of assigned vector elements

MCN mobile communication network

MDB map database

LRI low resolution image frame

OLI overlay image frame

PL1 first processing line

PL2 second processing line

PL3 third processing line

UTCC UTC clock

SEE appearance of vector element

Claims (8)

1. A procedure for the safe remote control of the train, where an automatic train (ATR.) comprises - an on-board automatic driving system unit, also called ADS unit, including a camera system (CAM) monitoring a section of track ahead of the automatic train (ATR), and - an automatic train mobile communications router (ATR-MCR) for forwarding and receiving data via a mobile communication network (MCN), where a railway traffic operation and management centre (COC) for automatic train operation, also called ATO, comprises - a remote train control console for remotely controlling the automated train, including a display showing an image of the track section in front of the train (ATR) to a remote driver on the remote train control console, and in particular including a moving map display for indicating a train position of the automated train (ATR), and - a management center mobile router (COC-MCR) for forwarding and receiving data via the mobile communication network (MCN), wherein the ATO rail traffic management and operation center (COC) and the automatic train system (ATR) each comprise a map database (MDB) containing common reference objects, characterized by that said reference objects each represent typical types of objects on Earth, wherein for each reference object at least one object ID and high-resolution object appearance information representative of the corresponding typical type of object on Earth are stored, wherein the high-resolution object appearance information is vectorized, that the camera system (CAM) generates high-resolution video image frames (HRI), which are assigned image frame numbers, and which are processed in at least two processing lines (PL1, PL2, PL3), where in a first processing line (PL1), - High Resolution Image (HRI) frames are downsampled, compressed and encoded in the integrated ADS unit into Low Resolution Image (LRI) frames, and - the low resolution image (LRI) frames and their image frame numbers are transmitted via a first data channel (CH1) from the on-board ADS unit to the train remote control console via the mobile communication network (MCN), where in a second processing line (PL2), - the high-resolution image (HRI) frames are subjected to a pattern recognition algorithm, which identifies objects in the high-resolution image (HRI) frames, maps them to reference objects stored in the map database (MDB) of the integrated ADS unit, and determines the corresponding object ID and relative object image inset properties of each identified object, including at least an image frame number, an insertion vector point, and scale/rotation parameters, and - the determined object IDs and image insertion properties are transmitted via a second data channel (CH2) from the on-board ADS unit to the train's remote control console via the mobile communication network (MCN), that at the ATO rail traffic management and operation centre (COC), object IDs and image embedding properties from the second data channel (CH2) are translated into high resolution object appearances (HROAs) according to corresponding stored reference objects in the map database (MDB) of the ATO rail traffic management and operation centre (COC), and that these high resolution object appearances (HROAs) are seamlessly overlaid on the low resolution (LRI) image frames with matching image frame number received via the first data channel (CH1), resulting in the image of the track section ahead of the train (ATR) displayed to the remote driver at the ATO rail traffic management and operation centre (COC). 2. Method according to claim 1, characterized by that in a third processing line (PL3), - High Resolution Image (HRI) frames are subjected to a vectorization algorithm, which assigns vector elements, such as points, polylines, or polygons, to the High Resolution Image (HRI) frames, and determines the corresponding vector properties of each assigned vector element, including an image frame number, and - the vector properties determined for each assigned vector element are transmitted via a third data channel (CH3) from the on-board ADS unit to the train's remote control console via the mobile communication network (MCN), that in the ATO rail traffic management and operation centre (COC), the vector properties of the assigned vector elements of the third data channel (CH3) are translated into vector element appearances (VEAs), and that the vector element appearances (VEAs) originating from the third data channel (CH3) are used for plausibility checks and/or included in the overlay of the high resolution object appearances (HROAs) originating from the second data channel (CH2) and the low resolution image frames (LRIs) received via the first data channel (CH1) with matching image frame number. 3. Method according to claim 1, characterized by that in a third processing line (PL3), - High Resolution Image (HRI) frames are subjected to a vectorization algorithm, which assigns vector elements, such as points, polylines, or polygons, to the High Resolution Image (HRI) frames, and determines the corresponding vector properties of each assigned vector element, including an image frame number, and - the vector properties determined for each assigned vector element are also transmitted via the second data channel (CH2) from the on-board ADS unit to the train's remote control console via the mobile communication network (MCN), that in the ATO rail traffic operation and management centre (COC), the vector properties of the assigned vector elements of the second data channel (CH2) of the third processing line (PL3) are translated into vector element appearances (VEA), and that the vector element appearances (VEA) originating from the second data channel (CH2) of the third processing line (PL3) are used for plausibility checks and/or are included in the overlay of the high resolution object appearances (HROA) originating from the second data channel (CH2) of the second processing line (PL2) and the low resolution image frames (LRI) received via the first data channel (CH1) with a matching image frame number. 4. Method according to any one of the preceding claims, characterized by that the relative object image embedding properties of each identified object also include a georeference, that the map database (MDB) of the railway traffic management and operation centre (COC) of the ATO and/or the automatic train (ATR) also contains a register of known objects defined by a georeference and a reference object assigned in each case, and that the ATO's railway traffic management and operation centre (COC) and/or the on-board ADS unit verify the plausibility of the assignment of the identified objects to the reference objects based on a comparison of the georeferences and assigned reference objects of the identified objects, on the one hand, and the georeferences and attributed reference objects of the known objects, on the other hand. 5. Method according to one of the preceding claims, characterized in that the railway traffic management and operation center (COC) of ATO checks the plausibility of the assignment of the identified objects to the reference objects based on a correlation of the high-resolution object appearances (HROA) and the corresponding object appearances in the low-resolution image frame (LRI) for matching frame numbers. 6. Method according to any one of the preceding claims, characterized by that the on-board ADS unit and the ATO's rail traffic operation and management centre (COC) each comprise at least one UTC clock (UTCC), preferably two UTC clocks, to provide UTC time, that the communication between the automatic train mobile communications router (ATR-MCR) and the management center mobile communications router (COC-MCR) via the mobile communication network (MCN) is packet-based, where each data packet (DP) is provided with a timestamp in a packet (DP) header section, indicating the UTC time of the packet (DP) being sent, that a Handshake protocol flow notes UTC times as follows: T<1>S UTC time when the packet was sent (DP), T<1>C UTC time acknowledged when the packet was received (DP), T<2S>UTC time when the acknowledgment (ACK) was sent and T<2C>UTC time when the acknowledgment (ACK) was received, which is determined by a round trip delay AT<IdayVuelta>, with AT<TodayReturn>=T<1C>-T<1S>+T<2C>-T<2 S>, that an average round trip delay AT<IdayVuelta_APR.> is determined over a plurality of passed packets (DP), and a warning message is sent if AT<IdayVuelta>- AT<IdayVuelta_AVROM>>T<UmbralRT>, with T<UmbralR T>: Threshold value for the round-trip delay, in particular with T<UmbralRT>=K<conf>*o(AT<IdayVuelta>), with K<conf>: confidence estimate value; o: standard variance. 7. Method according to claim 6, characterized by that a clock difference AT<clocksync> is determined, with AT<clocksync>= 0.5*(T<1C>-T<1S>+T<2S>-T<2c>), that an average clock difference AT<APR_sync_clock> is determined over a plurality of passed packets (DP), and that the warning message is also delivered if AT<clocksync>- AT<clocksync_AVG>>T<CDThreshold>, with T<CLOCKDThreshold>: Threshold value for the clock difference. 8. A system for the safe remote control of the train, comprising a) for arrangement in an automated train (ATR): - an on-board automatic driving system unit, also called ADS unit, including a camera system (CAM) for monitoring a section of track ahead of the automatic train (ATR), and - an automatic train mobile communications router (ATR-MCR) for forwarding and receiving data via a mobile communication network (MCN), and b) for arrangement in an automatic train operation, also called ATO, centre of operation and management of railway traffic on land (COC): - a remote train control console for remote control of the automated train (ATR), including a display for displaying an image of the track section in front of the train (ATR) to a remote driver on the remote train control console, and in particular including a moving map display for indicating a train position of the automated train (ATR), and - a management center mobile communication router (COC-MCR) for forwarding and receiving data via the mobile communication network (MCN), characterized in that the system is adapted to perform a method according to one of the preceding claims. characterized by that the system is adapted to perform a method according to one of the preceding claims.