Classifications

• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B66—HOISTING; LIFTING; HAULING
  • B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
  • B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
  • B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
  • B66F9/063—Automatically guided
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
  • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
  • B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
  • B65G1/02—Storage devices
  • B65G1/04—Storage devices mechanical
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
  • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
  • B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
  • B65G1/02—Storage devices
  • B65G1/04—Storage devices mechanical
  • B65G1/0492—Storage devices mechanical with cars adapted to travel in storage aisles
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
  • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
  • B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
  • B65G1/02—Storage devices
  • B65G1/04—Storage devices mechanical
  • B65G1/06—Storage devices mechanical with means for presenting articles for removal at predetermined position or level
  • B65G1/065—Storage devices mechanical with means for presenting articles for removal at predetermined position or level with self propelled cars
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
  • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
  • B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
  • B65G1/02—Storage devices
  • B65G1/04—Storage devices mechanical
  • B65G1/137—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed
  • B65G1/1373—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed for fulfilling orders in warehouses
  • B65G1/1375—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed for fulfilling orders in warehouses the orders being assembled on a commissioning stacker-crane or truck
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B66—HOISTING; LIFTING; HAULING
  • B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
  • B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
  • B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
  • B66F9/075—Constructional features or details
  • B66F9/0755—Position control; Position detectors
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B66—HOISTING; LIFTING; HAULING
  • B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
  • B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
  • B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
  • B66F9/075—Constructional features or details
  • B66F9/07568—Steering arrangements
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B66—HOISTING; LIFTING; HAULING
  • B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
  • B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
  • B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
  • B66F9/075—Constructional features or details
  • B66F9/20—Means for actuating or controlling masts, platforms, or forks
  • B66F9/24—Electrical devices or systems
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
  • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
  • G05D1/02—Control of position or course in two dimensions
  • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
  • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
  • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
  • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
  • G05D1/02—Control of position or course in two dimensions
  • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
  • G05D1/0259—Control of position or course in two dimensions specially adapted to land vehicles using magnetic or electromagnetic means
  • G05D1/0261—Control of position or course in two dimensions specially adapted to land vehicles using magnetic or electromagnetic means using magnetic plots
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
  • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
  • G05D1/20—Control system inputs
  • G05D1/24—Arrangements for determining position or orientation
  • G05D1/243—Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals

Definitions

• FIGs. 3A and 3B are exemplary perspective illustrations of portions of the autonomous guided vehicle of Fig. 2 in accordance with aspects of the disclosed embodiment
• Fig. 27 is an exemplary flow diagram in accordance with aspects of the disclosed embodiment.
• the supplemental sensor system also effects augmented reality operator inspection of the storage and retrieval system environment as well as remote control of the autonomous transport vehicle 110 as will be described herein.
• the power distribution unit 444 may also be configured to control a charge mode of a power supply 481 of the autonomous transport vehicle so as to maximize a number of charge cycles of the power supply 481.
• the power distribution unit 444 monitors current draw for components (e.g., motors, sensors, controllers, etc. that are communicably coupled to the power source 481 on "branch circuits") of the autonomous transport vehicle 110 and manages (e.g., switches on and off) the power supply to each of the components to conserve the charge (e.g., energy usage) of the power supply 481.
• the automated storage and retrieval system 100 in Figs. 1A and IB may be disposed in a retail distribution center or warehouse, for example, to fulfill orders received from retail stores for replenishment goods shipped in cases, packages, and or parcels.
• case, package and parcel are used interchangeably herein and as noted before may be any container that may be used for shipping and may be filled with case or more product units by the producer.
• Case or cases as used herein means case, package or parcel units not stored in trays, on totes, etc. (e.g., uncontained).
• the automated storage and retrieval system 100 may be generally described as a storage and retrieval engine 190 coupled to a palletizer 162.
• the storage and retrieval system 100 may be configured for installation in, for example, existing warehouse structures or adapted to new warehouse structures.
• the automated storage and retrieval system 100 shown in Figs. 1A and IB is representative and may include for example, in-feed and out- feed conveyors terminating on respective transfer stations 170, 160, lift module(s) 150A, 150B, a storage structure 130, and a number of autonomous transport vehicles 110 (also referred to herein as "bots").
• the transfer decks have multiple lanes between which the autonomous transport vehicles 110 freely transition for accessing the picking aisles 130A and/or lift modules 150A, 150B.
• open and undeterministic denotes the travel surface of the picking aisle and/or the transfer deck has no mechanical restraints (such as guide rails) that delimit the travel of the autonomous transport vehicle 110 to any given path along the travel surface. It is noted that while the aspects of the disclosed embodiment are described with respect to a multilevel storage array, the aspects of the disclosed embodiment may be equally applied to a single level storage array that is disposed on a facility floor or elevated above the facility floor.
• the in-feed transfer stations 170 and out-feed transfer stations 160 may operate together with their respective lift module(s) 150A, 150B for bi-directionally transferring case units CU to and from one or more storage structure levels 130L of the storage structure 130. It is noted that while the lift modules 150A, 150B may be described as being dedicated inbound lift modules 150A and outbound lift modules 150B, in alternate aspects each of the lift modules 150A, 150B may be used for both inbound and outbound transfer of case units from the storage and retrieval system 100.
• the frame 200 includes one or more idler wheels or casters 250 disposed adjacent the front end 200E1.
• the frame 200 also includes one or more drive wheels 260 disposed adjacent the back end 200E2.
• the position of the casters 250 and drive wheels 260 may be reversed (e.g., the drive wheels 260 are disposed at the front end 200E1 and the casters 250 are disposed at the back end 200E2).
• the autonomous transport vehicle 110 is configured to travel with the front end 200E1 leading the direction of travel or with the back end 200E2 leading the direction of travel.
• the drive wheels 260 have a fully independent suspension 280 coupling each drive wheel 260A, 260B of the at least pair of drive wheels 260 to the frame 200, with at least one intervening pivot link (described herein) between at least one drive wheel 260A, 260B and the frame 200 configured to maintain a substantially steady state traction contact patch between the at least one drive wheel 260A, 260B and rolling/travel surface 284 (also referred to as autonomous vehicle travel surface 284- see, e.g., Figs.
• the frame 200 includes one or more casters 250 disposed adjacent the front end 200E1.
• the front face case center point FFCP may be determined along the vertical axis VER (e.g. in the Z direction) relative to a case unit support plane PSP of the payload bed 210B (Fig. 4B - formed by one or more of the tines 210AT of the transfer arm 210A and the payload bed floor 473).
• the front face case center point FFCP may be determined along the lateral axis LAT (e.g. in the X direction) relative to a justification plane surface JPP of the pushers 470 (Fig. 4A). Determination of the front face case center point FFCP of the case units CU located on a storage shelf 555 (see Fig.
• case unit holding location provides, as non-limiting examples, for localization of the autonomous transport vehicle 110 relative to case units CU to be picked, mapping locations of case units within the storage structure (e.g., such as in a manner similar to that described in United States patent number 9,242,800 issued on January 26, 2016 titled “Storage and retrieval system case unit detection", the disclosure of which is incorporated herein by reference in its entirety), and/or pick and place accuracy relative to other case units on the storage shelf 555 (e.g., so as to maintain predetermined gap sizes between case units.
• mapping locations of case units within the storage structure e.g., such as in a manner similar to that described in United States patent number 9,242,800 issued on January 26, 2016 titled “Storage and retrieval system case unit detection", the disclosure of which is incorporated herein by reference in its entirety
• pick and place accuracy relative to other case units on the storage shelf 555 e.g., so as to maintain predetermined gap sizes between case units.
• the determination of the front face case center point FFCP also effects a comparison of the "real world" environment in which the autonomous transport vehicle 110 is operating with the virtual model 400VM so that controller 122 of the autonomous transport vehicle 110 compares what is "sees” with the vision system 400 substantially directly with what the autonomous transport vehicle 110 expects to "see” based on the simulation of the storage and retrieval system structure.
• the object (case unit) and characteristics determined by the vision system controller 122VC are coapted (combined, overlayed) to the virtual model 400VM enhancing resolution, in up to six degrees of freedom resolution, of the object pose with respect to a facility reference frame.
• the determination of the front face surface and case center point FFCP also effects a comparison of the "real world" environment in which the autonomous transport vehicle 110 is operating with the virtual model 400VM so that controller 122 of the autonomous transport vehicle 110 compares what is "sees” with the vision system 400 substantially directly with what the autonomous transport vehicle 110 expects to "see” based on the simulation of the storage and retrieval system structure.
• the image data obtained from the one or more three- dimensional imaging system 440A, 440B may supplement the image data from the cameras 410A, 410B in the event data from the cameras 410A, 410B is incomplete or missing.
• the forward navigation cameras 420A, 420B are any suitable cameras configured to provide object detection and ranging.
• the forward navigation cameras 420A, 420B may be placed on opposite sides of the longitudinal centerline LAXCL of the autonomous transport vehicle 110 and spaced apart by any suitable distance so that the forward facing fields of view 420AF, 420BF (see also Fig. provide the autonomous transport vehicle 110 with stereo vision.
• the one or more out of plane (e.g., upward or downward facing) localization cameras 477A, 477B are disposed on the frame 200 of the autonomous transport vehicle 110 so as to sense/detect location fiducials (e.g., location marks 971, lines 900, etc.) disposed on a ceiling 991 of the storage and retrieval system or on the rolling surface 284 of the storage and retrieval system.
• location fiducials e.g., location marks 971, lines 900, etc.
• the location fiducials have known locations within the storage and retrieval system and may provide unigue identification marks/patterns that are recognized by the vision system controller 122VC (e.g., processing data obtained from the localization cameras 477A, 477B).
• known objects such as case units CUI, CU2, CU3 (or storage system structure) (e.g., having a known physical characteristics such as shape, size, etc.) may be placed within the field of view of a camera (or the vehicle 110 may be positioned so that the known objects are within the field of view of the camera) of the supplemental navigation sensor system 288.
• These known objects may be imaged by the camera from several angles/view points to calibrate each camera so that the vision system controller 122VC is configured to detect the known objects based on sensor signals from the calibrated camera.
• Figs. 5A- 5C are exemplary images captured from one of case unit monitoring cameras 410A, 410B from, for exemplary purposes, three different view points.
• physical characteristics/parameters e.g., shape, length, width, height, etc.
• the vision system controller 122VC e.g., the physical characteristics of the different case units CUI, CU2, CU3 are stored in a memory of or accessible to the vision system controller 122VC.
• the vehicle 110 is moved so that any suitable number of view points of the case units CUI, CU2, CU3 are obtained/imaged by the case unit monitoring camera 410A, 410B to effect a convergence of the case unit characteristics/parameters (e.g., estimated by the vison system controller 122VC) for each of the known case units CUI, CU2, CU3.
• the case unit monitoring camera 410A, 410B is calibrated. The calibration process is repeated for the other case unit monitoring camera 410A, 410B.
• the vision system controller 122VC is configured with three-dimensional rays for each pixel in each of the case unit monitoring cameras 410A, 410B as well as an estimate of the three-dimensional baseline line segment separating the cameras and the relative pose of the case unit monitoring cameras 410A, 410B relative to each other.
• Suitable markers may also be placed on the case units/structure to facilitate feature extraction from the images obtained by the case unit monitoring cameras 410A, 410B and effect calibration of the case unit monitoring cameras 410A, 410B.
• Calibration of the other cameras e.g., the forward and rearward navigation cameras 420A, 420B, 430A, 430B, the traffic monitoring camera(s) 460A, 460B, and the out of plane localization camera(s) 477A, 477B, etc.
• the supplemental navigation sensor system 288 may be effected in a manner similar to that described above.
• the sensor groups 1111, 1114 may be employed by the vision system controller 122VC (and controller 122) for vehicle operations where the rear end 200E2 of the vehicle 110 leads a direction of vehicle travel (e.g., backward travel along a picking aisle 130A).
• the sensor group 1114 may be employed by the vision system controller 122VC (and controller 122) for vehicle operations where the transfer arm 210A loads or unloads a case unit CU to or from the payload bed 210B (e.g., pick place operations).
• the vision system 400 includes the at least one camera 292. It is noted that although the aspects of the present disclosure are described with respect to a forward facing camera (i.e., a camera that faces in the direction of travel with the end 200E1 of the autonomous transport vehicle 110 leading), the camera(s) may be positioned to face in any direction (rearward, sideways, up, down, etc.) for up to 360° monitoring about the autonomous transport vehicle 110.
• a forward facing camera i.e., a camera that faces in the direction of travel with the end 200E1 of the autonomous transport vehicle 110 leading
• the camera(s) may be positioned to face in any direction (rearward, sideways, up, down, etc.) for up to 360° monitoring about the autonomous transport vehicle 110.
• the at least one camera 292 may be placed on the longitudinal centerline LAXCL, on either side of the longitudinal centerline LAXCL, more than one camera 292 may be placed on opposite sides of the longitudinal centerline LAXCL of the autonomous transport vehicle 110 so that the field of view 292F provides the autonomous transport vehicle 110 with stereo vision (e.g., such as cameras 420A, 420B), or any other suitable configuration.
• the at least one camera 292, is any suitable camera configured to provide object or spatial feature 299 detection.
• the at least one camera 292 is any suitable high resolution or low resolution video cameras, a 3D imaging system, time-of-flight camera, laser ranging camera, or any other suitable camera configured to provide detection of the object or spatial feature 299 within at least a portion of the facility 100 viewed by the at least one camera 292 with the autonomous transport vehicle 110 in the different positions in the facility 100 while executing autonomous navigation and transfer tasks.
• the at least one camera 292 provides for imaging and detection (with either end 200E1, 200E2 of the autonomous transport vehicle 110 leading a direction of travel or trailing the direction of travel).
• the object or spatial feature 299 detection may be compared to reference floor maps and structure information (e.g., stored in a memory of or accessible by) of the vision system controller 122VC.
• the at least one camera 292 may also send signals to the controller 122 (inclusive of or through the vision system controller 122VC) so that as the autonomous transport vehicle 110 approaches the object or spatial feature 299, the autonomous transport vehicle 110 initiates an autonomous stop (i.e., in an autonomous operation state) or may enter a collaborative operation state so as to be stopped by an operator or maneuvered e.g., on the undeterministic rolling surface of the transfer deck 130B or within the picking aisle 130A (which may have a deterministic or undeterministic rolling surface) by the operator in order to identify the object or spatial feature 299 (e.g., another malfunctioning autonomous transport vehicle, dropped case unit, debris, spill, or other transient object within the storage and retrieval system 100).
• the object or spatial feature 299 e.g., another malfunctioning autonomous transport vehicle, dropped case unit, debris, spill, or other transient object within the storage and retrieval system 100.
• the camera(s) 292 of the supplemental hazard sensor system 290 may be calibrated in any suitable manner (such as by, e.g., an intrinsic and extrinsic camera calibration) to effect sensing/detection of the objects or spatial features 299 in the storage and retrieval system 100. Referring to Figs.
• the vision system controller 122VC is provided with intrinsic and extrinsic camera and case unit parameters that effect calibration of the camera(s) 292.
• the vision system controller 122VC is configured to employ the supplemental navigation sensor system 288 and/or the supplemental hazard sensor system 290 (i.e., imaging information obtained from the cameras of one or more of the supplemental sensor systems) to determine whether the objects are "unknown” (i.e., whether the objects or spatial features 299 are not expected to be within an area or space along the autonomous travel path of the autonomous transport vehicle 110).
• the supplemental navigation sensor system 288 and/or the supplemental hazard sensor system 290 i.e., imaging information obtained from the cameras of one or more of the supplemental sensor systems
• the virtual model 400VM (and the reference representation 400VMR of predetermined features thereof) of the operating environment 401 is stored in any suitable memory of the autonomous transport vehicle (such as a memory of the vision system controller 122VC) or in a memory accessible to the vision system controller 122VC.
• the virtual model 400VM provides the autonomous transport vehicle 110 with the dimensions, locations, etc. of at least the fixed (e.g., permanent) structural components in the operating environment 401.
• the operating environment 401 and the virtual model 400VM thereof includes at least fixed/permanent structure (e.g., transfer deck 130B, picking aisles 130A, storage spaces 130S, case unit transfer areas, case unit buffer locations, vehicle charging locations, support columns, etc.) of one more storage structure level 130L; in one or more aspects, the operating environment 401 and the virtual model 400VM include the fixed structure of the one or more storage structure level 130L and at least some transitory structure (e.g., case units CU stored or otherwise located at case unit holding locations of the storage and retrieval system 100, etc.) of and located within the storage level 130L on which the autonomous transport vehicle 110 operates; in one or more other aspects the operating environment 401 and the virtual model 400VM includes at least the fixed structure and at least some transitory structure (e.g., case units)) of one or more levels 130L of the storage structure 130 on which the autonomous transport vehicle 110 could operate; and in still other aspects, the operating environment 401 and virtual model 400VM includes the entirety of the storage structure and at least some of the transitor
• the sensor data that embodies the virtual representation VR images, is processed with any suitable image processing methods to detect a region of interest and/or edge features of objects in the image.
• the vision system controller 122VC predicts, within the model 400VM, a field of view of the sensor(s) providing the image data and determines, within the predicted field of view, regions of interest and edges of objects.
• the regions of interest and edges of the virtual model 400VM are compared with the regions of interest and edges of the virtual representation VR pose and location determination of one or more of the autonomous transport vehicle 110 and case units (payloads) as described herein.
• the controller 122 is configured to employ the supplemental (e.g., pixel level) position information obtained from the vision system controller 122VC of the supplemental navigation sensor system 288 to what may be referred to as "fine tune" the vehicle pose and location relative to the pick/place location so that positioning of the vehicle 110 and case units CU placed to storage locations 130S by the vehicle 110 may be held to smaller tolerances (i.e., increased position accuracy) compared to positioning of the vehicle 110 or case units CU with the physical characteristic sensor system 270 alone.
• supplemental e.g., pixel level
• the switching from the physical characteristic sensor system pose and location information to the virtual representation VR pose and location information may be effected by the vision system controller 122VC (or controller 122), by de-selecting the pose and location information, generated from the physical characteristic sensor system 270, and selecting/entering pose and location information from the virtual representation VR in a kinematic/dynamic algorithm (such as described in United States patent application number 16/144,668 titled “Storage and Retrieval System” and filed on September 27, 2018, the disclosure of which is incorporated herein by reference in its entirety).
• the vision system controller 122VC effects the above-noted switching the vision system controller 122VC is configured to continue autonomous transport vehicle 110 navigation to any suitable destination (such as a payload place destination, charging destination, etc.); while in other aspects the vision system controller 122VC is configured to select an autonomous transport vehicle 110 safe path and trajectory bringing the autonomous transport vehicle 110 from a position at switching to a safe location 157 (the safe location being a dedicated induction/extraction area of a transfer deck, a lift transfer area, or other area of the transfer deck 130B or picking aisle 130A at which the autonomous transport vehicle 110 may be accessed by an operator without obstructing operation of other autonomous transport vehicles 110 operating in the storage and retrieval system 100) for shut down of the autonomous transport vehicle 110; while in still other aspects, the vision system controller 122VC is configured to initiate communication to an operator of the storage and retrieval system 100 identifying autonomous transport vehicle 110 kinematic data and identify a destination of the autonomous transport vehicle 110 for operator selection (e.g., presented on user interface UI).
• the operator may select or switch control of the autonomous guided vehicle (e.g., through the user interface UI) from automatic operation to either quasi automatic operation (e.g., the autonomous transport vehicle 110 operates autonomously with limited manual input) or manual operation (e.g., the operator remotely controls operation of the autonomous transport vehicle 110 through the user interface UI).
• the user interface UI may include a capacitive touch pad/screen, joystick, haptic screen, or other input device that conveys kinematic directional commands (e.g., turn, acceleration, deceleration, etc.) and/or pick place commands from the user interface UI to the autonomous guided vehicle 110 to effect operator control inputs in the quasi automatic operational and manual operational modes of the autonomous transport vehicle 110.
• the vision system controller 122VC may be configured to apply the variance as a offset that is automatically applied to the data from the physical characteristic sensor system 270 to grossly position the autonomous transport vehicle 110 based on the data from the physical characteristic sensor system 270 as modified by the offset, where comparison with the virtual representation VR and the reference representation RPP verifies the validity of the offset and adjusts the offset (and autonomous transport vehicle 110 pose and location) according to any variance. Where the variance reaches a predetermined threshold the vision system controller 122VC may alert a user of the storage and retrieval system 100 that the autonomous guided vehicle 110 may be due for servicing.
• the vision system controller 122VC is configured to effect a similar pose and location error identification for the case units CU, such as held in storage locations 130S or other holding areas of the storage and retrieval system.
• the vision system controller 122VC is configured to confirm payload pose and location information registered by the vision system controller 122VC from the physical characteristic sensor system 270 based on the comparison between the virtual representation VR and the reference representation RPP of the virtual model 400VM.
• the vision system controller 122VC is configured to identify a variance in the payload (case unit) pose and location based on the comparison between the virtual representation VR and the reference representation RPP, and update (e.g., modify the pose and/or location information from the physical characteristic sensor system 270) or complete (if the pose and/or location information from the physical characteristic system 270 is missing) payload pose or location information from the physical characteristic sensor system based on the variance.
• the vision system controller 122VC is configured to determine a pose error in the information from the physical characteristic sensor system 270 and fidelity of the payload pose and location information from the physical characteristic sensor system 270 based on at least one of the identified variance and an image analysis of the at least one image from the vision system 400 of the supplemental navigation sensor system 288.
• the vision system controller 122VC assigns a confidence value according to at least one of the payload pose error and the fidelity. With the confidence value below a predetermined threshold, the vision system controller 122VC switches autonomous transport vehicle 110 payload handling based on pose and location information generated from the virtual representation VR in place of pose and location information from the physical characteristic sensor system 270.
• the vision system controller 122VC is configured to, in some aspects, continue autonomous guided vehicle handling to a predetermined destination (such as a payload placement location or an area of the storage and retrieval system where the payload may be inspected by an operator); in other aspects the vision system controller 122VC is configured to initiate communication to an operator identifying payload data along with an operator selection of autonomous guided vehicle control from automatic payload handling operation to quasi automatic payload handling operation (where the operator provides limited input to transfer arm 210A and traverse movements of the autonomous guided vehicle) or manual payload handling operation (where the operator manually controls movement of the transfer arm 210A and traverse movements of the autonomous guided vehicle) via the user interface device UI.
• a predetermined destination such as a payload placement location or an area of the storage and retrieval system where the payload may be inspected by an operator
• the vision system controller 122VC is configured to initiate communication to an operator identifying payload data along with an operator selection of autonomous guided vehicle control from automatic payload handling operation to quasi automatic payload handling operation (where the operator provides limited input to transfer
• the vision system controller 122VC is configured to transmit, via a wireless communication system (such as network 180) communicably coupling the vision system controller 122VC and an operator interface UI, a simulation image combining the virtual representation VR of the one or more imaged predetermined features and one or more corresponding reference predetermined features of a reference presentation RPP presenting the operator with an augmented reality image in real time (see Fig. 10A, where reference predetermined features include the shelves 555 and the virtual representations include those of the case units CU1-CU3).
• a wireless communication system such as network 180
• the vision system 400 of the supplemental navigation sensor system 288 provides a "dashboard camera” (or dash-camera) that transmits video and/or still images from the vehicle 110 to an operator to allow remote operation or monitoring of the vehicle 110. It is noted that the vision system 400 may also operate as a data recorder that periodically sends still images obtained from the vision system cameras to a memory of the user interface UI, where the still images are stored/cached for operator review (e.g., in addition to providing a real-time video stream the vision system 400 provides for non-real time review of the still images).
• a data recorder that periodically sends still images obtained from the vision system cameras to a memory of the user interface UI, where the still images are stored/cached for operator review (e.g., in addition to providing a real-time video stream the vision system 400 provides for non-real time review of the still images).
• the still images may be captured and transmitted to the user interface for storage at any suitable interval such as, for example, every second, every ten seconds, every thirty seconds, every minute, or at any other suitable time intervals (exclusive of real time video stream recording), where the periodicity of the still image capture/recording maintains suitable communication bandwidth between, for example, the control server 120 and the bots 110 (noting that in accordance with aspects of the disclosed embodiment, the number of bots 110 operating/transferring case units in the storage and retrieval system 100 may be on the order of hundreds to thousands of bots 110).
• the user interface UI with the record of stored still images provides for an interactive presentation/data interface where a user reviews the still images to determine how or why an event (e.g., such as a case miss-pick, bot breakage, product spill, debris presence on the transfer deck, etc.) occurred and what transpired prior to and/or after the event.
• an event e.g., such as a case miss-pick, bot breakage, product spill, debris presence on the transfer deck, etc.
• the vision system controller 122VC is configured to receive real time operator commands (e.g., from the user interface UI) to the traversing autonomous guided vehicle 110, which commands are responsive to the real time augmented reality image (see Figs. 9A and 10A), and changes in the real time augmented reality image transmitted to the operator by the vision system controller 122VC.
• the video or still images may be stored (and time stamped) in a memory onboard the vehicle 110 and sent to control server 120 and/or an operator on request; in other aspects the video and/or still images may be broadcast or otherwise transmitted in real time for viewing on a user interface UI (as described herein) accessible to the operator.
• the vision system controller 122VC is also configured to register image data captured by the supplemental hazard sensor system 290 and generate, from the captured image data, at least one image (e.g., still image and/or video image) of one or more object or spatial feature 299 showing the predetermined physical characteristic.
• the at least one image may be formatted as a virtual representation VR of the one or more object or spatial feature 299 (see Figs. 4D and 15) so as to provide a comparison (in at least one but up to the six degrees of freedom X, Y, Z, Rx, Ry, Rz (see Fig.
• the controller 122VC is configured to verify (via the comparison) the existence of presence of the predetermined physical characteristic of the object or spatial feature 299 based on the comparison between the virtual representation and the reference representation (i.e., compare to determine whether the object is "known" or "unknown”).
• the controller 122VC determines a dimension of the predetermined physical characteristic and commands (e.g., through the controller 122) the autonomous transport vehicle 110 to stop in a predetermined location relative to the object 299 (i.e., a trajectory is determined to autonomously place the bot in a predetermined position relative to the object or spatial feature 299) based on a position of the object or spatial features 299 determined from the comparison (as may be realized, the command stop interrupts the automatic routine of the vehicle previous autonomous commands, in effect diverting the bot from automatic tasking).
• the controller 122 selectably reconfigures the autonomous transport vehicle 110 from an autonomous state to a collaborative vehicle state so as to finalize discrimination of the object or spatial feature 299 as a hazard and identify a mitigation action of the vehicle with respect to the hazard (i.e., selectably switches the autonomous transport vehicle 110 from an autonomous operation state to a collaborative operation state and identifies whether the vehicle can mitigate the hazard, e.g., remove a disabled vehicle or act as a signal/beacon to warn other vehicles performing autonomous tasks).
• the autonomous transport vehicle 110 is disposed to receive operator commands for the autonomous transport vehicle 110 to continue effecting vehicle operation for discriminating and mitigation of the object or spatial feature 299.
• the autonomous transport vehicle 110 may not include the reference map (e.g., virtual model 400VM).
• the controller 122VC determines a position of the object within a reference frame of the at least one camera 292, which is calibrated and has a predetermined relationship to the autonomous transport vehicle 110. From the object pose in camera reference frame, the controller 122VC determines presence of the predetermined physical characteristic of object 299 (i.e., whether the object 299 is extended across bot path, blocks the bot, or is proximate, within a predetermined distance, to the bot path to be deemed an obstacle or hazard).
• the controller 122VC Upon determination of presence of an object and switch from the autonomous state to the collaborative vehicle state, the controller 122VC is configured to initiate transmission communicating image/video the of presence of the predetermined physical characteristic to an operator (user) interface UI for collaborative operator operation of the autonomous transport vehicle 110 as will be further described below (Here the vehicle 110 is configured as an observation platform and pointer for a user in collaborative mode. The vehicle 110 in this mode is also a pointer for other bots executing in autonomous operation, that identify the pointer bot (e.g., via control system 120, or beacon) and reroute automatically to avoid the area until further command and if avoidance is not available to stop ahead of encountering the object/hazard).
• the vision system controller 122VC is configured (as described herein with at least part of the virtual model 400VM and with suitable imaging processing non-transitory computer program code) so that the virtual representation VR, of the imaged object or spatial feature 299 is effected resident on the autonomous transport vehicle 110, and comparison between the virtual representation VR of the one or more imaged object or spatial feature 299 and the one or more corresponding reference predetermined features RPF (e.g., presented in a reference presentation RPP of the virtual model 400VM) is effected resident on the autonomous transport vehicle 110 (see Fig. 15).
• the comparison between the virtual representation VR and the reference representation RPP by the vision system controller 122VC verifies whether the object or spatial feature 299 is "unknown".
• the vision system controller 122VC is configured to determine a dimension of the object or spatial feature 299 based on image analysis of at least one image (from the vision system 400 of the supplemental hazard sensor system 290). Where the dimensions are unidentifiable, the vision system controller 122VC is configured to switch the autonomous transport vehicle 110 into the collaborative operation state for collaborative discrimination of the object 299 with the operator. The switching from the autonomous to the collaborative state may be effected by the vision system controller 122VC (or controller 122), by selectably reconfiguring the autonomous transport vehicle 110 from an autonomous vehicle to a collaborative vehicle (i.e., selectably switches the autonomous transport vehicle 110 from an autonomous operation state to a collaborative operation state).
• the controller 122 is configured to continue autonomous transport vehicle 110 navigation to any suitable destination relative to the detected object, applying a trajectory to the autonomous transport vehicle 110 that brings the autonomous transport vehicle 110 to a zero velocity within a predetermined time period where motion of the autonomous transport vehicle 110 along the trajectory is coordinated with "known" and "unknown” objects located relative to the autonomous transport vehicle 110.
• the vision system controller 122VC initiates communication to the operator of the storage and retrieval system 100 displaying the object or spatial feature 299 on the user interface UI for the operator to discriminate the object 299 and determine a mitigation action such as maintenance (e.g., clean-up of a spill, removal of a malfunctioning bot, etc.) and a location of the autonomous transport vehicle 110 (e.g., presented on user interface UI).
• maintenance e.g., clean-up of a spill, removal of a malfunctioning bot, etc.
• a location of the autonomous transport vehicle 110 e.g., presented on user interface UI.
• the controller 122 may initiate a signal/beacon to at least another bot(s) so as to alert the other bot(s) of a traffic obstacle and to avoid the obstacle or indicate a detour area (thus, in effect, the supplemental hazard sensor system 290 provides for a hazard pointer/indicator mode of one bot to others on the same level).
• the signal/beacon is sent via a local communication transmission to a system area bot task manager, managing tasks of nearby bots, or bots within a predetermined distance of the pointer bot.
• the controller 122 is configured, based on object information from the vision system 400 and vision system controller 122VC, to select an autonomous transport vehicle 110 safe path and trajectory bringing the autonomous transport vehicle 110 from a position at switching to a location 157 where the operator may view the object 299 without further obstructing operation of other autonomous transport vehicles 110 operating in the storage and retrieval system 100.
• the vision system controller is configured, based on object information from the vision system 400 and vision system controller 122VC, to select an autonomous transport vehicle 110 safe path and trajectory bringing the autonomous transport vehicle 110 from a position at switching to a location 157 where the operator may view the object 299 without further obstructing operation of other autonomous transport vehicles 110 operating in the storage and retrieval system 100.
• the operator may select or switch control of the autonomous guided vehicle (e.g., through the user interface UI) from automatic operation to collaborative operation (e.g., the operator remotely controls operation of the autonomous transport vehicle 110 through the user interface UI).
• the user interface UI may include a capacitive touch pad/screen, joystick, haptic screen, or other input device that conveys kinematic directional commands (e.g., turn, acceleration, deceleration, etc.) from the user interface UI to the autonomous transport vehicle 110 to effect operator control inputs in the collaborative operational mode of the autonomous transport vehicle 110.
• the vision system 400 of the supplemental hazard sensor system 290 provides a "dashboard camera" (or dash-camera) that transmits video and/or still images from the autonomous transport vehicle 110 to an operator (through user interface UI) to allow remote operation or monitoring of the area relative to the autonomous transport vehicle 110 in a manner similar to that described herein with respect to supplemental navigation sensor system 288.
• a "dashboard camera” or dash-camera
• the vision system controller 122VC (and/or controller 122) is in one or more aspects configured to provide remote viewing with the vision system 400, where such remote viewing may be presented to an operator in augmented reality or in any other suitable manner (such as unaugmented).
• the autonomous transport vehicle 110 is communicably connected to the warehouse management system 2500 (e.g., via the control server 120) over the network 180 (or any other suitable wireless network).
• the warehouse management system 2500 includes one or more warehouse control center user interfaces UI.
• the warehouse control center user interface US may be any suitable interfaces such as desktop computers, laptop computers, tablets, smart phones, virtual reality headsets, or any other suitable user interface configured to present visual and/or aural data obtained from the autonomous transport vehicle 110.
• the vehicle 110 may include one or more microphones MCP (Fig. 2) where the one or more microphones and/or remote viewing may assist in preventative maintenance/troubleshooting diagnostics for storage and retrieval system components such as the vehicle 110, other vehicles, lifts, storage shelves, etc.
• the warehouse control center user interfaces UI are configured so that warehouse control center users reguest or are otherwise supplied (such as upon detection of an unidentifiable object 299) with images from the autonomous transport vehicle 110 and so that the requested/supplied images are viewed on the warehouse control center user interfaces UI.
• the images supplied and/or requested may be live video streams, pre-recorded (and saved in any suitable memory of the autonomous transport vehicle 110 or warehouse management system 2500) images, or images (e.g., one or more static images and/or dynamic video images that correspond to a specified (either user selectable or preset) time interval or number of images taken on demand substantially in real time with a respective image request.
• live video stream and/or image capture provided by the vision system 400 and vision system controller 122VC may provide for real-time remote controlled operation (e.g., teleoperation) of the autonomous transport vehicle 110 by a warehouse control center user through the warehouse control center user interface UI.
• the live video is streamed from the vision system 400 of the supplemental navigation sensor system 288 and/or the supplemental hazard sensor system 290 to the user interface UI as a conventional video stream (e.g., the image is presented on the user interface without augmentation, what the camera "sees” is what is presented) as illustrated in Figs. 9A and 15.
• Fig. 9A illustrates a live video that streamed without augmentation from both the forward navigation cameras 420A, 420B (a similar video stream may be provided by the rearward navigation cameras 430A, 430B but in the opposite direction); while Fig.
• FIG. 15 illustrates a live video that streamed without augmentation from the forward camera 292/477A (a similar video stream may be provided by the rearward camera 292/477B but in the opposite direction). Similar video may be streamed from any of the cameras of the supplemental navigation sensor system 288 and/or supplemental hazard sensor system 290 described herein. While Fig. 9A illustrates a side by side presentation of the forward navigation cameras 420A, 420B, the video stream, where requested by the user, may be for but one of the forward navigation cameras 420A, 420B.
• images from the right side forward navigation camera 420A may be presented in a viewfinder of the virtual reality headset corresponding to the user's right eye and images from the left side forward navigation camera 420B may be presented in a viewfinder of the virtual reality headset corresponding to the user's left eye.
• the live video is streamed from the vision system 400 of the supplemental navigation sensor system 288 to the user interface UI as an augmented reality video stream (e.g., a combination of live video and virtual objects are presented in the streamed video) as illustrated in Fig. 10A.
• Fig. 10A illustrates a live video that is streamed with augmentation from one of the case unit monitoring cameras 410A, 410B (a similar video stream may be provided by the other of the case unit monitoring cameras 430A, 430B but offset by the separation distance between the cameras 430A, 430B).
• Similar augmented video may be streamed from any of the cameras of the supplemental navigation sensor system 288 described herein.
• Fig. 10A illustrates a live video that is streamed with augmentation from one of the case unit monitoring cameras 410A, 410B (a similar video stream may be provided by the other of the case unit monitoring cameras 430A, 430B but offset by the separation distance between the cameras 430A, 430B).
• the case units CUI, CU2, CU3 are presented to the user through the user interface UI in the live video stream as the case units are captured by the one of the case unit monitoring cameras 410A, 410B.
• Virtual representations of the shelf 555 and slats 520L on which the case units CUI, CU2, CU3 are seated may be inserted into the live video stream by the vision system controller 122VC or other suitable controller (such as control server 120) to augment the live video stream.
• the virtual representations of the shelf 555 and slats 520L may be virtually inserted into the live video stream such as where portions of the structure are not within the field of view 410AF, 410BF of the case unit monitoring cameras 410A, 410B (or a field of view of whichever camera of the supplemental navigation sensor system 288 is capturing the video).
• the virtual representations of the storage and retrieval structure may be virtually inserted into the live video streams to supplement/augment the live video stream with information that may be useful to the user (e.g., to provide a completed "picture" of what is being "observed” by the autonomous transport vehicle) where such information is not captured by cameras or not clearly discernable in the camera image data.
• the virtual representations of the storage and retrieval structure that are virtually inserted into the live video stream are obtained by the vision system controller 122VC (or control server 120) from the virtual model 400VM.
• the video streams may be augmented to provide the operator with a transport path VTP and/or destination location indicator DL that provide the operator with guidance as to a destination location of the autonomous transport vehicle 110.
• the transport path VTP and destination location indicator DL may also be presented in the video streams with the autonomous transport vehicle operating in the automatic/autonomous and quasi automatic operation modes to provide an operator with an indication of the planned route and destination.
• the method includes providing the autonomous transport vehicle 110 (Fig. 12, Block 1200) as described herein.
• Sensor data is generated (Fig. 12, Block 1205) with the physical characteristic sensor system 270 where, as described herein, the sensor data embodies at least one of a vehicle navigation pose or location information and payload pose or location information.
• Image data is captured (Fig. 12, Block 1210) with the supplemental navigation sensor system 288 where, as described herein, the image data informs the at least one of a vehicle navigation pose or location and payload pose or location supplement to the information of the physical characteristic sensor system 270.
• the method may also include determining, with the vision system controller 122VC, from the information of the physical characteristic sensor system 270 vehicle pose and location (Fig. 12, Block 1220) effecting independent guidance of the autonomous transport vehicle 110 traversing the storage and retrieval system 100 facility.
• the vision system controller 122VC may also determine from the information of the physical characteristic sensor system 270 payload (e.g., case unit CU) pose and location (Fig. 12, Block 1225) effecting independent underpick and place of the payload to and from the storage location and independent underpick and place of the payload in the payload bed 210B as described herein.
• the vision system controller 122VC may also register the captured image data and generating therefrom at least one image of one or more features of the predetermined features (Fig. 12, Block 1215) where, as described herein, the at least one image is formatted as a virtual representation VR of the one or more predetermined features so as to provide comparison to one or more corresponding reference e.g., a corresponding feature of the virtual model 400VM that serves as a reference for identifying the form and/or location of the imaged predetermined feature) of the predetermined features of the reference representation 400VMR.
• a corresponding reference e.g., a corresponding feature of the virtual model 400VM that serves as a reference for identifying the form and/or location of the imaged predetermined feature
• the vision system controller 122VC is configured so that the virtual representation VR, of the imaged one or more features of the predetermined features, is effected resident on the autonomous transport vehicle 110, and the comparison between the virtual representation VR of the one or more imaged predetermined features and the one or more corresponding reference predetermined features (of the reference representation 400VMR) is effected resident on the autonomous transport vehicle 110.
• the vision system controller 122 may confirm autonomous guided vehicle pose and location information or payload pose and location information (Fig. 12, Block 1230) registered by the vision system controller 122VC from the physical characteristic sensor system 270 based on the comparison between the virtual representation VR and the reference representation 400VMR.
• the vision system controller 122VC may identify a variance in the autonomous transport vehicle 110 pose and location or a variance in the payload pose and location (Fig. 12, Block 1235) based on the comparison between the virtual representation VR and the reference representation 400VMR, and update or complete autonomous transport vehicle 110 pose or location information or update and complete the payload pose and location information from the physical characteristic sensor system 270 based on the variance. In the method, the vision system controller 122VC may determine a pose error (for the autonomous guided vehicle and/or the payload) (Fig.
• Block 1240 in the information from the physical characteristic sensor system 270 and fidelity of the pose and location information (for the autonomous guided vehicle and/or the payload) from the physical characteristic sensor system 270 based on at least one of the identified variance and image analysis of the at least one image (e.g., from the vision system 400), and assign a confidence value according to at least one of the pose error and the fidelity.
• the vision system controller 122VC switches payload handling based on pose and location information generated from the virtual representation VR in place of pose and location information from the physical characteristic sensor system 270; and/or with the confidence value below a predetermined threshold, the vision system controller 122VC switches autonomous guided vehicle 110 navigation based on pose and location information generated from the virtual representation VR in place of pose and location information from the physical characteristic sensor system 270.
• the controller is configured to: continue autonomous guided vehicle navigation to destination or select an autonomous guided vehicle safe path and trajectory bringing the autonomous guided vehicle from a position at switching to a safe location for shut down, or initiate communication to an operator identifying autonomous guided vehicle kinematic data and a destination for operator selection of autonomous guided vehicle control from automatic operation to quasi automatic operation or manual operation via a user interface device; and/or continue autonomous guided vehicle handling to destination, or initiate communication to an operator identifying payload data along with an operator selection of autonomous guided vehicle control from automatic payload handling operation to quasi automatic payload handling operation or manual payload handling operation via a user interface device.
• the controller transmits, via a wireless communication system (such as network 180) communicably coupling the vision system controller 122VC and the operator/user interface UI, a simulation image (see Figs. 9A, 10A, 10B) (Fig. 12, Block 1245) combining the virtual representation VR of the one or more imaged predetermined features and one or more corresponding reference predetermined features RPF of a reference presentation RPR presenting the operator with an augmented reality image in real time.
• the vision system controller 122VC receives real time operator commands to the traversing autonomous guided vehicle 110, which commands are responsive to the real time augmented reality image (see Figs. 9A, 10A, 10B), and changes in the real time augmented reality image transmitted to the operator by the vision system controller 122VC.
• Figs. 1A, IB, 2, 4A, 4B, and 14 an example of an autonomous transport vehicle 110 case unit (s) transfer transaction including a case unit (s) multi-pick and place operation with on the fly sortation of the case units for creating a mixed pallet load MPL (e.g., a pallet load having mixed cases or cases having different stock keeping units as shown in Fig. IB) according to a predetermined order out sequence will be described in accordance with an aspects of the disclosed embodiment.
• a mixed pallet load MPL e.g., a pallet load having mixed cases or cases having different stock keeping units as shown in Fig. IB
• the autonomous transport vehicle 110 picks at least a first case unit CUA from a first shelf of a first storage location 130S1 of picking aisle 130A1 (Fig. 14, Block 1400). As described above, localization of the autonomous transport vehicle 110 relative to the case unit CUA in storage location 130S1 is effected with the physical characteristic sensor system 270 and/or the supplemental navigation sensor system 288 in the manner described herein.
• the autonomous transport vehicle 110 traverses the picking aisle 130A1 and buffers the at least first case unit CUA within the payload bed 210B (Fig. 14, Block 1410).
• the autonomous transport vehicle 110 traverses the picking aisle 130A1 to a second storage location 130S2 and picks at least a second case unit CUB that is different than the at least first case unit CUA (Fig. 14, Block 1420). While the at least second case unit CUB is described as being in the same picking aisle 130A1 as the at least first case unit CUA, in other aspects the at least second case unit CUB may be in a different aisle or any other suitable holding location (e.g., transfer station, buffer, inbound lift, etc.) of the storage and retrieval system.
• suitable holding location e.g., transfer station, buffer, inbound lift, etc.
• the at least first case unit CUA and the at least second case unit CUB may comprising more than one case in ordered seguence corresponding to a predetermined case out order sequence of mixed cases.
• the autonomous guided vehicle 110 traverses the picking aisle 130A1 and/or transfer deck 130B, with both the at least first case unit CUA and the at least second case unit CUB held within the payload bed 210B, to a predetermined destination (such as outbound lift 150B1).
• the positions of the at least first case unit CUA and the at least second case unit CUB within the payload bed 210B may be monitored by at least one or more of the case unit monitoring cameras 410A, 410B, one or more three-dimensional imaging system 440A, 440B, and one or more case edge detection sensors 450A, 450B and arranged relative to one another (e.g., the supplemental navigation sensor system 288 at least in part effects on-the-fly justification and/or sortation of case units onboard the vehicle 110 in a manner substantially similar to that described in United States patent number 10,850,921, the disclosure of which was previously incorporated herein by reference in its entirety) within the payload bed 210B (e.g., with the justification blades 471, pushers 470, and/or pullers 472) based on data obtained from the at least one or more of the case unit monitoring cameras 410A, 410B, one or more three-dimensional imaging system 440A, 440B, and one or more case edge detection sensors 450A, 450
• the autonomous transport vehicle 110 is localized (e.g., positioned) relative to the destination location with the physical characteristic sensor system 270 and/or the supplemental navigation sensor system 288 in the manner described herein.
• the autonomous transport vehicle 110 places the at least first case unit CUA and/or the at least second case unit CUB (Fig. 14, Block 1430) where the transfer arm 210A is moved based on data obtained by one or more of the physical characteristic sensor system 270 and/or the supplemental navigation sensor system 288.
• the method includes providing the autonomous transport vehicle 110 (Fig. 16, Block 1700) as described herein.
• the autonomous transport vehicle 110 is configured to autonomously navigate to different positions with the navigation system and operates to effect predetermined transfer tasks at the different positions (Fig. 16, Block 1705) while incidentally capturing image data (Fig. 16, Block 1710) with the supplemental hazard sensor system 290.
• the image data informs objects and/or spatial features 299 (having intrinsic physical characteristics) within at least a portion of the facility 100 viewed by the at least one camera 292 of the supplemental hazard sensor system 290 with the autonomous transport vehicle 110 in the different positions in the facility 100.
• the method may also include determining, with the vision system controller 122VC, from the information of the supplemental hazard sensor system 290 presence of a predetermined physical characteristic of at least one object or spatial feature (Fig. 16, Block 1715), and in response thereto, selectably reconfiguring the vehicle from an autonomous state to a collaborative vehicle state (Fig. 16, Block 1720) for collaboration with an operator, the vehicle in the collaborative state is disposed to receive operator commands for the vehicle to continue effecting vehicle operation so as to finalize discrimination of the object or spatial feature 299 as a hazard (Fig. 16, Block 1725) and identify a mitigation action of the vehicle with respect to the hazard (Fig. 16, Block 1730) as described herein.
• the vision system controller 122VC may also register the captured image data and generating therefrom at least one image of the presence of a predetermined physical characteristic of the at least one object or spatial feature 299 (Fig. 16, Block 1735) where, as described herein, the at least one image is formatted as a virtual representation VR of the predetermined physical characteristic of the at least one object or spatial feature 299 so as to provide comparison to one or more corresponding reference (e.g., a corresponding feature of the virtual model 400VM that serves as a reference for identifying the form and/or location of the imaged object or spatial feature 299) of the predetermined features of the reference representation 400VMR.
• a corresponding reference e.g., a corresponding feature of the virtual model 400VM that serves as a reference for identifying the form and/or location of the imaged object or spatial feature 299
• the vision system controller 122VC is configured so that the virtual representation VR, of the imaged object or spatial feature 299, is effected resident on (e.g., onboard) the autonomous transport vehicle 110, and the comparison between the virtual representation VR of the object or spatial feature 299 and the one or more corresponding reference predetermined features (of the reference representation 400VMR) is effected resident on the autonomous transport vehicle 110.
• the vision system controller 122VC may determine presence of an unknown physical characteristic of the at least one object or spatial feature and switch the autonomous transport vehicle 110 from an autonomous operation state to a collaborative operation state.
• the controller 122 is configured to: stop the autonomous transport vehicle 110 relative to the object or spatial feature 299 or select an autonomous guided vehicle path and trajectory bringing the autonomous transport vehicle 110 from a position at switching to a location 157 to initiate communication to an operator for identifying the object or spatial feature 299 via a user interface device UI.
• the controller 122VC transmits, via a wireless communication system (such as network 180) communicably coupling the vision system controller 122VC and the operator/user interface UI, an image (see Fig. 15) (Fig. 16, Block 1740) combining the virtual representation VR of the one or more imaged object or spatial feature 299 and one or more corresponding reference predetermined features RPF of a reference presentation RPR presenting the operator with an augmented (or un-augmented) reality image in real time.
• the controller 122 receives real time operator commands to the autonomous transport vehicle 110, which commands are responsive to the real time augmented reality or unaugmented image (see Fig. 15), and changes in the real time augmented reality or un-augmented image transmitted to the operator by the vision system controller 122VC.
• the autonomous transport vehicle 110 includes the controller 122 that is coupled respectively to the drive section 261D, the case handling assembly 210, the peripheral electronics section 778, and other components/features of the autonomous transport vehicle 110 that are described herein so as to form a control system 122CS (see Figs. 25A-25C).
• the control system 122CS effects each autonomous operation of the autonomous transport vehicle 110 described herein.
• the controller system 122CS may be configured to provide communications, supervisory control, vehicle localization, vehicle navigation and motion control, payload sensing, payload transfer, and vehicle power management as described herein. In this and other aspects, the control system may also be configured to provide any suitable services to the vehicle 110.
• the control system 122CS includes any suitable non- transitory program code and/or firmware that configure the vehicle 110 to perform the vehicle operations described herein.
• the control system 122CS may be configured for (but is not limited to) one or more of remote updating of control system firmware/software, remote debugging of the vehicle 110, remote operation of the vehicle 110, tracking a position of the vehicle 110, tracking operational status of the vehicle 110, and tracking any other suitable information pertaining to the vehicle 110.
• the control system 122CS is a distributed control system that includes, as described, herein, the controller 122, the vision system controller 122VC, and the power management section 444 (which includes the switching device 449 and the monitoring and control device 447).
• the vision system controller 122VC and the power management section 444 are at least partially integral to the controller 122; while in other aspects one or more of the system controller 122VC and the power management section 444 are separate from but communicably coupled to the controller 122.
• Components of the control system may be distributed throughout the autonomous transport vehicle 110 and communicably coupled to the controller 122 in any suitable manner (such as described in Figs. 25A-25C).
• the controller 122 includes at least one of an autonomous navigation control section 122N and an autonomous payload handling control section 122H.
• the autonomous navigation control section 122N is configured to register and hold in a volatile memory (such as memory 446 of a comprehensive power management section 444 of the controller 122) autonomous guided vehicle state and pose navigation information that is deterministic (and provided in real time) of and describes current and predicted state, pose, and location of the autonomous transport vehicle 110.
• the autonomous transport vehicle state and pose navigation information includes both historic and current autonomous guided vehicle state and pose navigation information.
• the state, pose, and location information is deterministic (and provided in real time) and describes the current and predicted state, pose, and location in up to six degrees of freedom X, Y, Z, Rx, Ry, Rz so that the historic, current and predicted state, pose, and location is described in full.
• the autonomous payload handling control section 122H is configured to register and hold in the volatile memory (such as memory 446) current payload identity, state, and pose information (e.g., both historic and current).
• the payload identity, state, and pose information describes historic and current payload identity, payload pose and state location relative to a frame of reference of the autonomous transport vehicle (e.g., such as the X, Y, Z coordinate axes and suitable datum surfaces within the payload bed 210B), and pick/place locations of current and historic payloads.
• a frame of reference of the autonomous transport vehicle e.g., such as the X, Y, Z coordinate axes and suitable datum surfaces within the payload bed 210B
• the controller 122 comprises a comprehensive power management section 444 (also referred to as a power distribution unit - see also Fig. 26) that is separate and distinct from each other section (such as the vision system controller 122VC) of the controller 122.
• the power distribution unit 444 is communicably connected to the power supply 481 so as to monitor a charge level (e.g., voltage level or current level) of the power supply 481.
• the power distribution unit 444 is connected to each respective branch circuit 482 (also referred to herein as a branch power circuit - see Fig.
• the power distribution unit 444 is configured to comprehensively manage power consumption to each respective branch circuit 482 based on demand level of each branch circuit 482 relative to the charge level available from the power supply 481.
• the power distribution unit 444 includes a monitoring and control device 447 (referred to herein as monitoring device 447), a switching device 449 (having switches 449S), a memory 446, a wireless communications module 445, and an analog to digital converter 448 (referred to herein as AD converter 448).
• the monitoring device 447 is any suitable processing device configured to monitor at least the current usage and fuse status of the branch power circuits 482 and control shutdown of one or more selected branch power circuits 482 as described herein.
• the monitoring device 447 is one or more of a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), a system on chip integrated circuit (SOC), and a central processing unit (CPU).
• FPGA field-programmable gate array
• CPLD complex programmable logic device
• SOC system on chip integrated circuit
• CPU central processing unit
• the monitoring device 447 operates independent of the controller 122 and vision system controller 122VC, and the monitoring device 447 is programmed with non-transitory code to manage (e.g., at least power distribution to) one or more low level systems of the autonomous transport vehicle 110.
• the power distribution unit 444 is configured to communicate with and control at least one branch device 483.
• the power distribution unit 444 is communicably coupled to one or more of the analog sensors 483C (e.g., case edge detection sensors, line following sensors 275, and other analog sensors as described herein), the digital sensors 483B (e.g., cameras 410, 440, 450 of the vision system 400 and other digital sensors described herein), lights 483A, casters 250, drive/traction wheels 260, transfer arm 210A extension motor 667A-667C, transfer arm lift motors 669, payload justification motors 668A-668F of the payload bed 210B/transfer arm 210A, suspension lock motors, and any other suitable features of the autonomous transport vehicle 110 (see Figs.
• the analog sensors 483C e.g., case edge detection sensors, line following sensors 275, and other analog sensors as described herein
• the digital sensors 483B e.g., cameras 410, 440, 450 of the vision system 400 and other digital sensors described herein
• lights 483A e.g., casters
• the power distribution unit 444 may receive commands from the controller 122 to actuate one or more of the analog sensors 483C and the digital sensors 483B so that the one or more of the analog sensors 483C and the digital sensors 483B obtain one or more of pose and location information of the autonomous transport vehicle within the storage and retrieval system 100 storage structure 130 in a manner substantially similar to that described herein, and in United States patent numbers 8,425,173 titled “Autonomous Transport for Storage and Retrieval Systems” issued on April 23, 2013; 9,008,884 titled “Bot Position Sensing” issued on April 14, 2015; and 9,946,265 titled Bot Having High Speed Stability” issued on April 17, 2018, the disclosures of which are incorporated herein by reference in their entireties.
• the power distribution unit 444 is configured to process and filter (in any suitable manner) the sensor data obtained by the one or more of the analog sensors 483C and the digital sensors 483B.
• the power distribution unit 444 may also be configured to process and filter (in any suitable manner) control signals sent by the controller 122 (or vision system controller 122VC) to the one or more of the analog sensors 483C and the digital sensors 483B.
• the power distribution unit 444 includes the AD converter 448 to effect conversion of the analog sensor data to digital sensor data for filtering and processing by the power distribution unit 444.
• the autonomous transport vehicle may include lights 483A (Fig. 20, see also lighting/LED in Figs. 25A-25C) that are coupled to the frame 200 (or any other location of the autonomous transport vehicle 110) and that illuminate portions of the storage structure 130 adjacent the autonomous transport vehicle 110.
• the power distribution unit 444 is configured to control operation of the lights 483A.
• the power distribution unit 444 is configured to provide a pulse width modulation control signal to the lights 483A to actuate the lights 483A in a manner that minimizes power consumption.
• the pulse width modulation control signal is configured to minimize an amount of power drawn from the power supply 481 for illuminating the lights 483A for a given autonomous transport vehicle task (e.g., reading a barcode with the vision system 400, detecting case unit features with the vision system, illumination of a portion of the storage and retrieval system 100 for remote operator viewing effected by the vision system as described herein.
• the lights 483A may be any suitable lights including but not limited to light emitting diodes (LED).
• the power distribution unit 444 is configured to manage power needs of the autonomous transport vehicle 110 so as to preserve higher level functions/operations of the autonomous transport vehicle 110.
• the power distribution unit 444 is configured so as to comprehensively manage a demand charge level of each respective branch power circuit 482 (on which respective branch devices 483A-483F ... 483n (collectively referred to as branch devices 483, where n denotes an integer representing a maximum number of branch devices) are disposed - see Figs.
• the predetermined pattern (e.g., for switching off the branch power circuits 482) is arranged to switch off branch power circuits 482 with a decrease in the available charge level from the power supply 481, so as to maximize available charge level from the power supply 481 directed to the controller 122.
• the predetermined pattern is arranged to switch off the branch power circuits 482 with the decrease in the available charge level from the power supply 481 so that the available charge level directed to the controller 122 is equal to or exceeds the demand charge level of the controller 122 for a maximum time based on the available charge level of the power supply 481 (e.g., to preserve operation of the controller 122).
• the monitoring device 447 of the power distribution unit 444 is configured to monitor the voltage of the power supply 481 (Fig. 23, Block 23800) as described herein and shut down components/systems (e.g., analog sensors, digital sensors drive systems, communications systems, etc.) of the autonomous transport vehicle 110 in a sequenced shutdown order where the each shutdown operation in the sequenced shutdown order depends on a respective threshold voltage of the power supply.
• the power supply 481 power supply has a fully charged voltage of VI. With the power distribution unit 444 detecting the voltage VI the components/systems of the autonomous transport vehicle 110 are substantially fully operational to effect transport of case units throughout the storage structure 130.
• the voltage of the power supply 481 may drop (and the power distribution unit 444 detects such voltage drop) to a first predetermined threshold voltage V2 (where V2 is less than VI).
• the power distribution unit 444 monitoring the power supply 481 voltage detects that power supply voltage drops to a voltage equal to about the first predetermined threshold voltage V2 (Fig.
• the power distribution unit 444 may operate the switches 449S to remove power from (e.g., shut down) branch power circuits 482 corresponding to case unit handling components/systems (e.g., arm extension drives 667, payload justification drives 668, arm lift drives 669, case unit sensors, arm/case unit justification position sensors, suspension locks, etc.) of the autonomous transport vehicle 110 (Fig. 23, Block 23820) so that remaining power of the power supply 481 may be employed to effect traverse of the autonomous transport vehicle to a charging station/location or other predetermined location within the storage structure 130.
• case unit handling components/systems e.g., arm extension drives 667, payload justification drives 668, arm lift drives 669, case unit sensors, arm/case unit justification position sensors, suspension locks, etc.
• the controller 122 may effect traverse of the autonomous transport vehicle to a safe location as described herein (e.g., a predetermined location of the storage and retrieval system where the autonomous vehicle may be accessed by an operator for maintenance or removal from the storage structure 130).
• the power distribution unit 444 continues to monitor the voltage of the power supply 481 for a drop in the power supply voltage to a subsequent (e.g., next) lower threshold voltage (Fig. 23, Block 23830).
• the power distribution unit 444 operates the switches 449S to remove power from (e.g., shut down) branch power circuits 482 (such as circuits 483D, 483F) corresponding to drives/systems that effect vehicle traverse (e.g., the right and left drive/traction wheels 260A, 260B (Figs. 2 and 21), caster wheel steering drives 600M (Fig. 2), traction control system 666 (Fig. 21), sensors and sensor controllers effecting vehicle navigation (e.g., vision system, line following sensors, etc. such as provided with sensor system 270) (Fig.
• branch power circuits 482 such as circuits 483D, 483F
• Block 23840 so that remaining power of the power supply 481 may be employed to effect operation of the controller 122 of the autonomous transport vehicle 110.
• primary communications between the autonomous transport vehicle 110 and the control server 120 and/or an operator may also be shut down to preserve power for the controller 122.
• the communications module 445 of the power distribution unit 444 operates to maintain a secondary communications channel between the controller 122 and the control server 120 and/or an operator (e.g., via the laptop, smart phone/tablet, etc.).
• the power distribution unit 444 continues to monitor the voltage of the power supply 481 for the next subsequent lower threshold voltage (Fig. 23, Block 23850). For example, where a threshold voltage V4 (where V4 is less than V3) of the power supply 481 is detected by the power distribution unit 444, the power distribution unit 444 is configured to initiate shutting down of the controller 122 (Fig. 23, Block 23860) so that the controller 122 (and its software) is not adversely affected by a loss of power or an under-voltage/under-current failure.
• the controller 122 is configured so that upon indication from (e.g., a prediction by) the power distribution unit 444 of imminent decrease in available charge level, directed from the power supply 481 to the controller 122, to less than a demand level of the controller 122, the controller 122 enters suspension of operation and hibernation. With the controller 122 in suspension and hibernating (e.g., shut down) the power distribution unit 444 may also shut itself down so that substantially all operations of the autonomous transport vehicle 110 are suspended.
• the threshold voltage V4 is described above as the "lowest threshold voltage” such that detection of the threshold voltage V4 initiates shutdown of the controller 122.
• the above shut down sequence effected by the power distribution unit 444 is exemplary only and in other aspects there may be any suitable number of threshold voltages at which any suitable number of corresponding vehicle components/systems are shut down to preserve power of the power supply 281.
• Blocks 23830 and 23840 of Fig. 23 may be repeated in a loop until the next to lowest threshold voltage is reached.
• each threshold voltage in the descending values of threshold voltages is known to the power distribution unit 444 (such as stored in memory 446 and accessible by the monitoring device 447) such that the loop ends when the next to lowest threshold voltage is reached.
• the autonomous transport vehicle 110 has a power supply 481 with a fully charged voltage of about 46V (in other aspects the fully charged voltage may be more or less than about 46V).
• the power distribution unit 444 monitors the voltage output by the power supply 481 during autonomous transport vehicle 110 operation in a manner similar to that described above with respect to Fig. 23.
• the power distribution unit 444 operates the switches 449S to disable the traction motors 261M and other features (e.g., sensors associated with navigation/traverse of the autonomous transport vehicle) of the autonomous transport vehicle so that driving of the autonomous transport vehicle is disabled.
• the power distribution unit 444 continues to monitor the output voltage of the power supply 481 for the next lowest threshold voltage of about 20V (in other aspects the output voltage may be more or less than about 20V). Upon detection of the threshold voltage of about 20V, the power distribution unit 444 effects, through the controller 122, positioning of any case units CU carried by the autonomous transport vehicle 110 to a known safe state (e.g., retracted into the payload bed 210B in a predetermined justified location) within the payload bed 210B.
• a known safe state e.g., retracted into the payload bed 210B in a predetermined justified location
• the controller 122 may effect extension of the transfer arm 210A to place the case unit(s) CU at the destination location rather than retract the case unit (s) CU into the payload bed 210B (noting that after placement of the case unit (s) CU the transfer arm 210A is retracted within the payload bed 21B to a safe/home position).
• the power distribution unit 444 is configured to operate the switches 499S, upon detection of the next lowest threshold voltage of about 18V of the power supply 481 (in other aspects the output voltage may be more or less than about 18V), so as to shut down the vision system 400 and other 24V peripheral power supplies (e.g., including but not limited to case detection sensors, vehicle localization sensors, hot swap circuitry, etc.).
• the power distribution unit 444 Upon detection of the next lowest power supply 481 output threshold voltage of about 14V (in other aspects the output voltage may be more or less than about 14V) the power distribution unit 444 is configured to operate the switches 499S to disable onboard and off-board communications (e.g., wireless communications module 445 and onboard Ethernet communications) of the autonomous transport vehicle 110.
• onboard and off-board communications e.g., wireless communications module 445 and onboard Ethernet communications
• the power distribution unit 444 continues to monitor the power supply 481 output voltage for the next lowest threshold voltage of about 12V (in other aspects the output voltage may be more or less than about 12V), and upon detection of the about 12V output voltage the power distribution unit 444 turns off lighting (e.g., LEDs) of the autonomous transport vehicle 110 and provides command signals to the controller 122 so that the controller 122 is placed into hibernation/sleep as described above.
• lighting e.g., LEDs
• the power distribution unit 444 Upon detection of the lowest power supply 481 output threshold voltage of about 10V (in other aspects the output voltage may be more or less than about 10V) by the power distribution unit 444, the power distribution unit 444 effects a complete shutdown of the autonomous transport vehicle 444 such that the controller 122, the vision system controller 122VC, and other suitable programmable devices (e.g., FPGAs, CPLDs, SOCs, CPUs, etc.) of the autonomous transport vehicle 110 are turned off/shut down.
• suitable programmable devices e.g., FPGAs, CPLDs, SOCs, CPUs, etc.
• the monitoring device 447 is configured to substantially continuously (e.g., with the autonomous transport vehicle 110 in operation) monitor power supply 481 operation and status.
• the monitoring device 447 is configured to substantially continuously (or at any suitable predetermined time intervals) monitor a voltage of the power supply 481 (e.g., with any suitable voltage sensors) and communicate a low voltage condition (e.g., the voltage has dropped below a predetermined voltage level) to the controller 122 so that the controller 122 may effect a safe state of the autonomous transport vehicle 110.
• the controller 122 is configured (e.g., via the monitoring device 447) so that upon indication from the power distribution unit 444 of imminent decrease in available charge level of the power supply 481, directed from the power supply 481 to the branch power circuit of the drive section 261D (see Fig. 21), the controller 122 is configured to command the drive section 261D so as to navigate the autonomous transport vehicle 110 along a predetermined auxiliary path AUXP and auxiliary trajectory AUXT (known as safe, nonconflicting with other vehicles 110, not impedimental nor blocking other vehicle paths, pass through nor destination location - see Fig. IB) to a predetermined bot auxiliary stop location 157 in the storage and retrieval facility (e.g., structure) 130.
• a predetermined auxiliary path AUXP and auxiliary trajectory AUXT known as safe, nonconflicting with other vehicles 110, not impedimental nor blocking other vehicle paths, pass through nor destination location - see Fig. IB
• the predetermined auxiliary stop location 157 is a safe, uncongested area of a transport deck 130B or picking aisle 130A or a human access zone (such as described in United States patent number 10,088,840 titled “Automated Storage and Retrieval System with Integral Secured Personnel Access Zones and Remote Rover Shutdown” issued on October 2, 2018, the disclosure of which is incorporated herein by reference in its entirety).
• the controller 122 is configured so that upon indication from the power distribution unit 444 of imminent decrease in available charge level of the power supply 481, directed from the power supply 481 to the branch circuit of the payload handling section 210 (see Fig. 21) the controller 122 is configured to command the payload handling section 210 to move the payload handling actuator or transfer arm 210A (e.g., via one or more of arm extension drives 667 and arm lift drives 669), and any payload thereon (e.g., via payload justification drives 668), to a predetermined safe payload position in the payload bed 210B.
• the safe payload position may be such that the payload does not overhang outside of the payload bed and is securely held within the payload bed 210B.
• the controller 122 may also be configured to actively monitor a health status of the autonomous transport vehicle 110 and effect onboard diagnostics of vehicle systems.
• vehicle system health is monitored in any suitable manner such as by monitoring current used and fuse status of the vehicle systems (and the branch power circuits 482 of which the branch devices 483 are a part).
• the controller 122 includes at least one of a vehicle health status monitor 447V, a drive section health status monitor 447D, a payload handling section health monitor 447H, and a peripheral electronics section health monitor 447P.
• the vehicle health status monitor 447V, the drive section health status monitor 447D, the payload handling section health monitor 447H, and the peripheral electronics section health monitor 447P may be sections of the monitoring device 447.
• the controller also includes a health status register section 447M, which may be a section of the memory 446 (or memory 122M or any other suitable memory accessible by the controller 122).
• the vehicle health status monitor 447V may monitor dynamic responses of the frame 200 and wheel suspension, such as with any suitable vehicle health sensors (such as accelerometers) coupled to the frame (e.g., such as described in United States provisional patent application number 63/213,589 titled "Autonomous Transport Vehicle with Synergistic Vehicle Dynamic Response” and filed on June 22, 2021, the disclosure of which is incorporated herein by reference in its entirety). Where a dynamic response is outside of a predetermined range the vehicle health status monitor 447V may effect (through controller 122) a maintenance request (e.g., presented on user interface UI) to an operator of the storage and retrieval system 100. In other aspects, any suitable characteristics of the vehicle may be monitored by the vehicle health status monitor 447V.
• any suitable vehicle may be monitored by the vehicle health status monitor 447V.
• the drive section health status monitor 447D may monitor power drawn by the motors 261M of the drive section 261D, drive section sensor (e.g., wheel encoders, etc.) status, and a status of the traction control system 666. Where the power usage of the motors 261M, drive section sensor responsiveness, and/or a traction control system response is outside of predetermined operating characteristics the drive section health status monitor 447D may effect (through controller 122) a maintenance request (e.g., presented on user interface UI) to an operator of the storage and retrieval system 100.
• a maintenance request e.g., presented on user interface UI
• the payload handling section health monitor 447H may monitor power drawn by the motors (e.g., extension lift, justification, etc.) of the case handling assembly 210 and a status of the case handling assembly sensors. Where the power usage of the case handling assembly motors and/or a case handling assembly sensor response is outside of predetermined operating characteristics the payload handling section health monitor 447H may effect (through controller 122) a maintenance request (e.g., presented on user interface UI) to an operator of the storage and retrieval system 100.
• a maintenance request e.g., presented on user interface UI
• the peripheral electronics section health monitor 447P may monitor the sensor system 270 and the at least one peripheral motor 777. Where the power usage of at least one peripheral motor 777 and/or a sensor (of the sensor system 270) response is outside of predetermined operating characteristics the peripheral electronics section health monitor 447P may effect (through controller 122) a maintenance request (e.g., presented on user interface UI) to an operator of the storage and retrieval system 100.
• a maintenance request e.g., presented on user interface UI
• the power distribution unit 444 is configured to monitor current in the branch power circuits 482 (in any suitable manner, such as directly with ammeters or indirectly by monitoring voltage and/or resistance of the respective branch power circuits 482) and a status of the respective fuses 484 of the branch power circuits 482.
• Real-time feedback e.g., input data relating to current and fuse status is processed by the monitoring device 447 within milliseconds so that the processed data it is available substantially immediately as feedback
• the controller 122 and control server 120 is provided to effect autonomous transport vehicle 110 operator and/or service/maintenance requests.
• the real time feedback effected by the monitoring device 447 monitoring at least the branch power circuit 482 current and fuse 484 status provides for onboard diagnostics and health monitoring of the autonomous transport vehicle systems.
• the power distribution unit 444 is configured to detect the fuse 484 status (e.g., inoperable or operable) based on, for example current of the respective branch power circuit 482. Where there is an absence of current detected in the respective branch power circuit 482 the monitoring device 447 determines that the fuse 484 is inoperable and in need of service, otherwise where current is detected the fuse 484 is operable (i.e., a fault state (see, e.g., Fig. 5) is detected).
• the monitoring device 447 provides the fuse status (e.g., fault state) as feedback to, for example, the control server 120 and/or an operator through the communications module 445 so that servicing of the autonomous transport vehicle 110 can be scheduled.
• the power distribution unit 444 is configured to monitor each branch power circuit 482 separately from each other power branch power circuit 482 so that where a fuse is determined to be inoperable the monitoring device 447 also identifies the branch power circuit 482 of which the fuse is a part so as to reduce downtime and troubleshooting of the autonomous transport vehicle 110 for fuse 484 replacement.
• An increased current within a branch power circuit may be indicative of an impending drive motor fault, an impending bearing fault, or other impending electrical/mechanical fault.
• each branch power circuit is monitored separately so that where an increased current is detected the corresponding branch power circuit 482 is also identified.
• the monitoring device 447 provides the increased current value (e.g., fault state) and identifies the branch power circuit 482 with the overcurrent therein to, for example, the control server 120 and/or an operator through the communications module 445 so that servicing of the autonomous transport vehicle 110 can be scheduled.
• the power distribution unit 444 is configured to monitor voltage regulators 490, branch device central processing units (CPUs) 491, and/or position sensors 492 of peripheral devices (e.g., such as transfer arm 210A, payload justification pushers/pullers, wheel encoders, navigation sensor systems (as described herein), payload positioning sensor systems (as described herein) (it is noted that suitable examples of payload justification pushers/pullers are described in, for example United States provisional patent application number 63/236,591 having attorney docket number 1127P015753-US (-#3) filed on August 24, 2021 and titled “Autonomous Transport Vehicle” as well as United States pre-grant publication number 2012/0189416 published on July 26, 2012 (United States patent application number 13/326, 52 filed on December 15, 2011) and titled "Automated Bot with Transfer Arm”; United States patent number 7591630 issued on September 22, 2009 titled “Materials-Handling System Using Autonomous Transfer and Transport Vehicles”
• the monitoring device 447 is configured to monitor communications between the position sensors 492 and the controller 122, communications between the branch device controller(s) 491 and the controller 122, and the voltage from the voltage regulators 490. Where communication is expected from a sensor 492 and/or branch device controller 491 the monitoring device 447 may register a fault (e.g., time stamped) in the memory 446 and communicate such fault state (e.g., with the communications module 445 to the control server 120 and/or operator effecting a maintenance reguest.
• a fault e.g., time stamped
• the monitoring unit 447 may continue to monitor and register faults from the branch device 483/branch power circuit 482 and send a service reguested message to the control server 120 or operator depending on a freguency of the faults or any other suitable criteria.
• the monitoring device 447 is configured to monitor a voltage of a voltage regulator 490 for one or more power branch circuits 482 in any suitable manner (such as feedback from the voltage regulator or voltmeter). Where there is an over-voltage or under-voltage detected by the monitoring device 447 the monitoring device 447 may register a fault (e.g., time stamped) in the memory 446 and communicate such fault state (e.g., with the communications module 445 to the control server 120 and/or operator effecting a maintenance reguest.
• a fault e.g., time stamped
• the monitoring unit 447 may continue to monitor and register faults from the voltage regulator 490 and send a service reguested message to the control server 120 or operator depending on a freguency of the faults or any other suitable criteria (such as a magnitude of the over-voltage or under-voltage).
• the power distribution device 444 of the controller 122 is configured as a boot device so that at autonomous transport vehicle 110 cold startup (initialization) the monitoring device 447 is brought online before other sections of the controller 122 and vision system controller 122VC so as to set initial (safe) states of the autonomous transport vehicle 110 prior to boot-up of the controller 122 and vision system controller 122VC.
• the controller 122 is configured so that upon indication from the power distribution unit 444 of imminent decrease in available power supply charge level, directed from the power supply 481 to the controller 1222, to less than a demand level of the controller 122, the controller 122 configures at least one of the autonomous guided vehicle state and pose navigation information and the payload identity, state, and pose information, held in respective registry and memory (e.g., such as memory 446 or other memories 122M of corresponding ones of the autonomous navigation control section 122N, the autonomous payload handling control section 122H, and the vision system control section (e.g., vision system controller 122VC)), into an initialization file 122F (Fig. 2) available on reboot of the controller 122.
• registry and memory e.g., such as memory 446 or other memories 122M of corresponding ones of the autonomous navigation control section 122N, the autonomous payload handling control section 122H, and the vision system control section (e.g., vision system controller 122VC)
• the controller 122 may also be configured so that upon indication from the power distribution unit 444 of imminent decrease in available power supply charge level, directed from the power supply 481 to the controller 122, to less than a demand level of the controller 122, to configure stored health status information from the at least one of the vehicle health status monitor 447V, the drive section health status monitor 447D, the payload handling section health monitor 447H, and the peripheral electronics section health monitor 447P in the health status register section 447M (such as in memory 122M or memory 446) into an initialization file 122F available on reboot of the controller 122.
• the monitoring device 447 of the power distribution unit 444 is configured to control power up sequencing of the controller 122 sections (e.g., the autonomous navigation control section 122N, the autonomous payload handling control section 122H, and vision system controller 122VC), and branch devices 483 (e.g., sensors, drive motors, caster motors, transfer arm motors, justification device motors, payload bed 210B motors, etc.).
• the sequencing may be that the vision system controller 122VC is powered up before the autonomous navigation control section 122N and the branch devices are powered up last; however, in other aspects any suitable power sequence may be employed such that control devices are powered up before the devices they control.
• a exemplary autonomous transport vehicle 110 power up or cold startup process will be described with the power distribution device 444 as a boot device.
• power to the autonomous transport vehicle 110 is turned on (Fig. 27, Block 2200) and the power distribution device 444 monitors the output voltage of the power supply 481 and determines if the output voltage is greater than a startup threshold voltage Via (Fig. 27, Block 2205) of about 16V (in other aspects the startup threshold voltage Via may be more or less than about 16V).
• the power distribution unit 444 operates switches 499S so that power is provided to, for example, the controller 122, the vision system controller 122VS, the wireless communications module 445, and the other suitable programmable devices (e.g., FPGAs, CPLDs, SOCs, CPUs, etc.) of the autonomous transport vehicle 110 (Fig. 27, Block 2210).
• the initialization file 122F (described above) may be employed on startup of the controller 122, the vision system controller 122VS, the wireless communications module 445, and the other suitable programmable devices (e.g., FPGAs, CPLDs, SOCs, CPUs, etc.) (Fig.
• the power distribution unit 444 continues to monitor the voltage output by the power supply 481 and where the output voltage is detected as being above a next higher startup threshold voltage V2a (Fig. 27, Block 2220) of about 18V (in other aspects the startup threshold voltage V2a may be more or less than about 18V), the power distribution unit 444 operates switches 449S to turn on the lighting (e.g., LEDs - see Figs. 10A-10C) of the autonomous transport vehicle 110 (Fig. 27, Block 2225). Where the next higher startup threshold voltage V2a has not been reached the power distribution unit 444 continues to monitor the power supply 481 output voltage until the next higher startup threshold voltage V2a is reached (such as with the autonomous transport vehicle 110 being charged), or until a shutdown sequence is initiated (see Fig. 8 described herein).
• the power distribution unit 444 With the power distribution unit 444 continuing to monitor the voltage output of the power supply 481, and with a next higher startup threshold voltage V3a detected by the power distribution unit (Fig. 27, Block 2230), the power distribution unit 444 operates the switches 449S so as to power up/turn on the case handling drives of, for example, the front and rear justification module 210ARJ, 210AFJ, payload bed 210B, and transfer arm 210A (Fig. 27, Block 2235) as well as 24V peripherals and instruments (see Figs. 25A-25C) of the autonomous transport vehicle 110.
• the threshold voltage V3a may be about 24V but in other aspects the threshold voltage V3a may be more or less than about 24V.
• the power distribution unit 444 continues to monitor the power supply 481 output voltage until the next higher startup threshold voltage V3a is reached (such as with the autonomous transport vehicle 110 being charged), or until a shutdown sequence is initiated (see Fig. 23 described herein).
• the power distribution unit 444 With the power distribution unit 444 monitoring the voltage output of the power supply 481, and with detection of a next higher startup threshold voltage V4a (Fig. 27, Block 2240), the power distribution unit 444 operates the switches 449S so as to power up/turn on the traction drive motors 261M (Fig. 27, Block 2245).
• the threshold voltage V4a may be about 28V but in other aspects the threshold voltage V4a may be more or less than about 28V.
• the power distribution unit 444 continues to monitor the power supply 481 output voltage until the next higher startup threshold voltage V4a is reached (such as with the autonomous transport vehicle 110 being charged), or until a shutdown sequence is initiated (see Fig. 23 described herein).
• the power distribution unit 444 is configured (e.g., with any suitable non-transitory computer program code) to power up the components of the autonomous transport vehicle 110 in the manner/sequence described above with respect to Fig. 27.
• the power distribution unit 444 is configured so that control devices are powered up before the devices they control.
• the controller 122 may be configured to effect one or more of onboard power supply charge mode, active control of inrush current to branch devices 483 (e.g., lower level system of the autonomous transport vehicle), and regenerative power supply 481 charging.
• branch devices 483 e.g., lower level system of the autonomous transport vehicle
• power distribution unit 444 With the autonomous transport vehicle 110 at a charging station (Fig. 28, Block 1300) power distribution unit 444 detects the presence of the traverse surface charging pad(s) (see Fig. 21 and Fig. 28, Block 1310).
• the power distribution unit 444 as described herein, is configure to monitor the output voltage of the power supply 481 and effect control tasks based on the output voltage level.
• control of power supply 481 charging is based on the output voltage of the power supply 481 detected by the power distribution unit 444.
• the monitoring device 447 of the power distribution unit 444 is configured to control a low level charging logic of the autonomous transport vehicle 110.
• An exemplary charging logic block diagram for the power distribution unit 444 is illustrated in Fig. 21. As can be seen in Fig.
• the autonomous transport vehicle 110 is configured with vehicle mounted charging contacts that receive charging current from a charging pad located on a traverse surface of the transfer deck 130B, picking aisle 130A, and/or any other suitable traverse surface of the storage and retrieval system on which the autonomous transport vehicle 110 travels.
• the traverse surface mounted charging pad and the vehicle mounted charging contacts are substantially similar to that described in United States patent number 9,469,208 titled “Rover Charging System” and issued on October 18, 2016; United States patent number 11,001,444 titled “Storage and Retrieval System Rover Interface” and issued on May 11, 2021; and United States patent application number 14/209,086 titled “Rover Charging System” and filed on March 13, 2014).
• the autonomous transport vehicle 110 may also be configured with remote charging ports mounted to the front end 200E1 or rear end 200E2 of the frame 200 that engage (e.g., plug into) corresponding charge ports mounted to the storage structure 130 or a hand-held plug which an operator plugs into the remote charging ports of the autonomous transport vehicle 110.
• remote charging ports mounted to the front end 200E1 or rear end 200E2 of the frame 200 that engage (e.g., plug into) corresponding charge ports mounted to the storage structure 130 or a hand-held plug which an operator plugs into the remote charging ports of the autonomous transport vehicle 110.
• the monitoring device 447 controls a charge mode/rate of the power supply 481 so as to maximize a number of charge cycles of the power supply 481.
• the monitoring device 447 is configured to effect one or more of a trickle charge mode (e.g., having a charge rate below a set threshold voltage), a slow charge mode, and an ultra-high-speed (e.g., high current) charge mode, where the charging current is limited by the monitoring device 447 to a set maximum charge voltage threshold to substantially prevent adverse effects on the power supply 481 from charging.
• a trickle charge mode e.g., having a charge rate below a set threshold voltage
• a slow charge mode e.g., a slow charge mode
• an ultra-high-speed (e.g., high current) charge mode e.g., high current) charge mode
• the charging current and voltage may be dependent on a capacity of and type of the power supply 481.
• the power supply 481 may have any suitable voltage and charge capacity and may be an ultra-capacitor or any other suitable power source (e.g., lithium ion battery pack, lead acid battery pack, etc.). As can also be seen in Fig. 6, the autonomous transport vehicle 110 includes suitable active reverse voltage protection for the power supply 481.
• the power distribution unit 444 detects that the output voltage from the power supply 481 is below a threshold charging voltage Vic (Fig. 28, Block 1320) of about 23V (in other aspects the threshold charging voltage Vic may be more or less than 23V), the monitoring device 477 of the power distribution unit 444 effects a limited current charging of the power supply 1330.
• the limited charging current may be the slow charging mode described above.
• the slow charge charging mode described above may have a charge current higher than that of the trickle charging mode but lower than a full charge current.
• the power distribution unit 444 continues to monitor the output voltage of the power supply 481 during charging and with the detection of the output voltage of the power supply 481 being at or equal to the threshold charging voltage Vic (Fig. 28, Block 1320), the monitoring device 477 of the power distribution unit 444 effects another charging mode, such as the full charge current mode (Fig. 28, Block 1350).
• the power distribution unit 444 monitors the output voltage of the power supply 481 during charging at full charge current and where the output voltage is at or greater than a next higher threshold charging voltage V2c (Fig. 13, Block 1340) of about 44V (in other aspects the output voltage may be more or less than about 44V), the monitoring device 477 of the power distribution unit 444 terminates charging.
• the monitoring device 477 may effect the trickle charge mode so as to maintain the power supply 481 at peak/maximum charge with the vehicle charge contacts of autonomous transport vehicle 110 engaged/coupled with the traverse surface charging pad(s) (see Fig. 21).
• the autonomous transport vehicle 110 includes one or more of current inrush protection, over voltage/current protection, and under voltage/current protection.
• the autonomous transport vehicle 110 may include hot swap circuitry (substantially similar to that described in United States patent number 9,469,208 titled “Rover Charging System” and issued on October 18, 2016; United States patent number 11,001,444 titled “Storage and Retrieval System Rover Interface” and issued on May 11, 2021; and United States patent application number 14/209,086 titled “Rover Charging System” and filed on March 13, 2014) that is configured to effect autonomous transport vehicle 110 roll-on and roll-off of the traverse surface mounted charging pads regardless of an energization status of the traverse surface mounted charging pads.
• the power distribution unit 444 is configured to actively control inrush current to the branch devices 483A-483F ... 483n (collectively referred to as branch devices 483, where n denotes an integer representing a maximum number of branch devices) of the respective branch power circuits 482, where the power distribution unit 444 receives from the controller 122 (and the controller 122 is configured to generate) a pulse width modulation signal that effects active control of the switches 449S to limit the inrush current (such as from charging or power surges) to the branch devices 483.
• the power distribution unit 444 may operate one or more of the switches 449S so as to open the one or more switches to prevent inrush current from flowing to the branch devices 483.
• one or more of the branch power circuits includes an electrical protection circuit 700 configured to protect the branch device 483 (a sensor is illustrated in Fig. 22 for exemplary purposes but in other aspects any suitable branch device, such as those described herein, may be provided).
• the electrical protection circuit 700 is configured to substantially protect the branch device 483 (and any controls/measurement instruments devices associated therewith) from, for example, short circuits, over-voltage, and over-current.
• the branch device 483 in this example a sensor
• the electrical protection circuit 700 for exemplary purposes only, includes an adjustable three-terminal positive-voltage regulator 710 and a single resistor 720.
• the voltage regulator 710 is configured to supply more than about 1.5 A over an output-voltage range of about 1.25 V to about 37 V.
• the voltage regulator 710 with the resistor 720 coupled thereto limits the current to about 27 mA by leveraging the internal reference voltage of the voltage regulator 710.
• the insertion of the electrical protection circuit 700 into the branch power circuit 482 substantially does not affect the about 4 mA to about 20 mA signal while providing control/measurement protection to devices disposed both upstream and downstream (with respect to the flow of current) the electrical protection circuit 700.
• the configuration of the electrical protection circuit 700 is exemplary only and that the electrical protection circuit 700 may be configured with any suitable voltage regulator and resistor (having suitable specifications) for providing control/measurement protection for signal that are less than about 4 mA or more than about 20 mA.
• the power distribution unit 444 is configured to effect regenerative charging of the power supply 481.
• the back electromotive force (EMF) voltage produced by the respective motors 261M is fed back into the respective branch power circuit 483E, 483F.
• the monitoring device 447 may operate the switches 449S (such as the Vcap_IN switch - see Fig. 21) so that the back EMF voltage (and current) regeneratively charges the power supply 481. With the motors 261M under power to drive the drive wheels 260A, 260B the monitoring device 447 may close the Vcap_IN switch to prevent power drain from the power supply 481.
• the power distribution unit 444 includes the wireless communication module 445.
• the wireless communication module 445 may be configured for any suitable wireless communication including, but not limited to, Wi-Fi, Bluetooth, cellular, etc.
• the wireless commination module 445 configures the power distribution unit 444 so as to control at least in part, for example, communication between the autonomous transport vehicle 110 and other features of the storage and retrieval system including but not limited to the control server 120 over any suitable network such as network 180.
• the wireless communication module 445 and monitoring device 447 configure the power distribution unit 444 as a secondary processor/controller such as where processing function errors of the controller 122 (e.g., such as safety related functions including remote shutdown, communications or other general component errors) are detected by the monitoring device 447.
• processing function errors of the controller 122 e.g., such as safety related functions including remote shutdown, communications or other general component errors
• the power distribution unit 444 maintains (secondary) communication between the control server 120 (and operators of the storage and retrieval system 100) and the different components of the autonomous transport vehicle 110 (e.g., through the communication module 445) so that the autonomous transport vehicle 110 can be remotely shut down or driven (either autonomously, semi-autonomously, or under manual remote control of an operator in a manner described herein to any suitable destination location.
• the wireless commination module 445 also provides for "over the air” programming of the of the controller 122, vision system controller 122VC and updating firmware/programming of the monitoring device 447 or other suitable programmable devices (e.g., FPGAs, CPLDs, SOCs, CPUs, etc.) of the autonomous transport vehicle 110.
• suitable programmable devices e.g., FPGAs, CPLDs, SOCs, CPUs, etc.
• the power distribution unit 444 includes any suitable memory 446 that may buffer the software updates for installation in the monitoring device 447, controller 122, vision system controller 122VC and/or other suitable programmable devices (e.g., FPGAs, CPLDs, SOCs, CPUs, etc.).
• suitable programmable devices e.g., FPGAs, CPLDs, SOCs, CPUs, etc.
• the wireless commination module 445 of the power distribution unit 444 may also be configured as an Ethernet switch or Bridge.
• the wireless communication modules 455 of the autonomous transport vehicles 110 travelling throughout the storage structure 130 may form a mesh network.
• wireless communications from, for example the control server 122 or other suitable device such as a laptop, smart phone/tablet, etc. may be extended to a range the covers substantially an entirety of the storage structure 130 without dedicated Ethernet switches and bridges being disposed throughout (e.g., mounted to) the storage structure 130 in fixed/predetermined locations.
• the method includes providing the autonomous transport 110 as described herein (Fig. 24, Block 24900). Autonomous operation of the autonomous transport vehicle 110 is effected with the controller 122 (Fig. 24, Block 24910) and a charge level of the power supply 481 of the autonomous transport vehicle 110 is monitored by the power distribution unit 444 (Fig. 24, Block 24920) as described herein.
• the method may also include, as described herein, the switching of the branch power circuits 482 on and off in the predetermined pattern (such as described herein) based on the demand charge level of each respective branch power circuit 482 with respect to other branch power circuits 482 and the charge level available from the power supply 481 (Fig. 24, Block 24930).
• the controller 122 Upon indication from the power distribution section 444 of imminent decrease in available power supply charge level, directed from the power supply 481 to the branch circuit 482 of the drive section 261D (see Fig. 21) and/or the case handling assembly 210, the controller 122 commands the drive section 261D to move of the autonomous transport vehicle 110 to a safe location and/or commands the case handling assembly 210 to move the payload to a safe location (Fig. 24, Block 24960) as described herein.
• the controller 122 upon indication from the power distribution unit 444 of imminent decrease in available power supply charge level, directed from the power supply 481 to the controller 122, to less than demand level of the controller 122, the controller 122 creates at least one initialization file (Fig. 24, Block 24940).
• the controller 122 may configure at least one of the autonomous guided vehicle state and pose navigation information and the payload identity, state, and pose information, held in respective registry and memory (e.g., such as memory 446 or other memories 122M of corresponding ones of the autonomous navigation control section 122N, the autonomous payload handling control section 122H, and the vision system control section (e.g., vision system controller 122VC)) of corresponding controller sections, into an initialization file 122F available on reboot of the controller 122.
• registry and memory e.g., such as memory 446 or other memories 122M of corresponding ones of the autonomous navigation control section 122N, the autonomous payload handling control section 122H, and the vision system control section (e.g., vision system controller 122VC)
• the controller 122 may store health status information from the at least one of vehicle health status monitor 447V, the drive section health status monitor 447D, the payload handling section health monitor 447H, and the peripheral electronics section health monitor 447P in the health status register section 477M into the initialization file 122F (or a different initialization file) available on reboot of the controller 122.
• an autonomous guided vehicle comprises:
• a drive section coupled to the frame with drive wheels supporting the autonomous guided vehicle on a traverse surface, the drive wheels effect vehicle traverse on the traverse surface moving the autonomous guided vehicle over the traverse surface in a facility;
• a payload handler coupled to the frame configured to transfer a payload, with a flat undeterministic seating surface seated in the payload hold, to and from the payload hold of the autonomous guided vehicle and a storage location, of the payload, in a storage array;
• a physical characteristic sensor system connected to the frame having electro-magnetic sensors, each responsive to interaction or interface of a sensor emitted or generated electromagnetic beam or field with a physical characteristic, the electromagnetic beam or field being disturbed by interaction or interface with the physical characteristic, and which disturbance is detected by and effects sensing by the electro-magnetic sensor of the physical characteristic, wherein the physical characteristic sensor system is configured to generate sensor data embodying at least one of a vehicle navigation pose or location information and payload pose or location information; and
• a supplemental sensor system connected to the frame, that supplements the physical characteristic sensor system, the supplemental sensor system being, at least in part, a vision system with cameras disposed to capture image data informing the at least one of a vehicle navigation pose or location and payload pose or location supplement to the information of the physical characteristic sensor system.
• the autonomous guided vehicle further comprises a controller connected to the frame, operably connected to the drive section or the payload handler, and communicably connected to the physical characteristic sensor system, wherein the controller is configured to determine from the information of the physical characteristic sensor system vehicle pose and location effecting independent guidance of the autonomous guided vehicle traversing the facility.
• the controller is configured to determine from the information of the physical characteristic sensor system payload pose and location effecting independent underpick and place of the payload to and from the storage location and independent underpick and place of the payload in the payload hold.
• the controller is programmed with a reference representation of predetermined features defining at least part of the facility traversed through by the autonomous guided vehicle.
• the controller is configured to register the captured image data and generate therefrom at least one image of one or more features of the predetermined features, the at least one image being formatted as a virtual representation of the one or more predetermined features so as to provide comparison to one or more corresponding reference of the predetermined features of the reference representation.
• the controller is configured so that the virtual representation, of the imaged one or more features of the predetermined features, is effected resident on the autonomous guided vehicle, and comparison between the virtual representation of the one or more imaged predetermined features and the one or more corresponding reference predetermined features is effected resident on the autonomous guided vehicle.
• the controller is configured to confirm autonomous guided vehicle pose and location information registered by the controller from the physical characteristic sensor system based on the comparison between the virtual representation and the reference representation.
• the controller is configured to identify a variance in the autonomous guided vehicle pose and location based on the comparison between the virtual representation and the reference representation, and update or complete autonomous guided vehicle pose or location information from the physical characteristic sensor system based on the variance.
• the controller is configured to determine a pose error in the information from the physical characteristic sensor system and fidelity of the autonomous guided vehicle pose and location information from the physical characteristic sensor system based on at least one of the identified variance and analysis of the at least one image, and assign a confidence value according to at least one of the pose error and the fidelity.
• the controller is configured so that with the confidence value below a predetermined threshold, the controller switches autonomous guided vehicle navigation based on pose and location information generated from the virtual representation in place of pose and location information from the physical characteristic sensor system.
• the controller is configured to:
• [0215] initiate communication to an operator identifying autonomous guided vehicle kinematic data and a destination for operator selection of autonomous guided vehicle control from automatic operation to quasi automatic operation or manual operation via a user interface device.
• the controller is configured to confirm payload pose and location information registered by the controller from the physical characteristic sensor system based on the comparison between the virtual representation and the reference representation.
• the controller is configured to identify a variance in the payload pose and location based on the comparison between the virtual representation and the reference representation, and update or complete payload pose or location information from the physical characteristic sensor system based on the variance.
• the controller is configured to determine a pose error in the information from the physical characteristic sensor system and fidelity of the payload pose and location information from the physical characteristic sensor system based on at least one of the identified variance and analysis of the at least one image, and assign a confidence value according to at least one of the pose error and the fidelity.
• the controller is configured so that with the confidence value below a predetermined threshold, the controller switches autonomous guided vehicle payload handling based on pose and location information generated from the virtual representation in place of pose and location information from the physical characteristic sensor system.
• the controller is configured to: [0221] continue autonomous guided vehicle handling to destination, or
• [0222] initiate communication to an operator identifying payload data along with an operator selection of autonomous guided vehicle control from automatic payload handling operation to guasi automatic payload handling operation or manual payload handling operation via a user interface device.
• the controller is configured to transmit, via a wireless communication system communicably coupling the controller and an operator interface, a simulation image combining the virtual representation of the one or more imaged predetermined features and one or more corresponding reference predetermined features of a reference presentation presenting the operator with an augmented reality image in real time.
• the controller is configured to receive real time operator commands to the traversing autonomous guided vehicle, which commands are responsive to the real time augmented reality image, and changes in the real time augmented reality image transmitted to the operator by the controller.
• the supplemental sensor system at least in part effects on-the-fly justification and/or sortation of case units onboard the autonomous guided vehicle.
• imaged or viewed objects described by one or more of supplemental information, supplemental vehicle navigation pose or location, and supplemental payload pose or location, from the supplemental sensor system are coapted to a reference model of one or more of surrounding facility features and interfacing facility features so as to enhance, via the one or more of the supplemental information, the supplemental vehicle navigation pose or location, and the supplemental payload pose or location resolution of one or more of the vehicle navigation pose or location information and the payload pose or location information.
• an autonomous guided vehicle comprises:
• a drive section coupled to the frame with drive wheels supporting the vehicle on a traverse surface, the drive wheels effect vehicle traverse on the traverse surface moving the autonomous guided vehicle over the traverse surface in a facility;
• a payload handler coupled to the frame configured to transfer a payload, with a flat undeterministic seating surface seated in the payload hold, to and from the payload hold of the autonomous guided vehicle and a storage location, of the payload, in a storage array;
• a physical characteristic sensor system connected to the frame having electro-magnetic sensors, each responsive to interaction or interface of a sensor emitted or generated electromagnetic beam or field with a physical characteristic, the electromagnetic beam or field being disturbed by interaction or interface with the physical characteristic, and which disturbance is detected by and effects sensing by the electro-magnetic sensor of the physical characteristic, wherein the physical characteristic sensor system is configured to generate sensor data embodying at least one of a vehicle navigation pose or location information and payload pose or location information; and
• an auxiliary sensor system connected to the frame, that is separate and distinct from the physical characteristic sensor system, the auxiliary sensor system being, at least in part, a vision system with cameras disposed to capture image data informing the at least one of a vehicle navigation pose or location and payload pose or location which image data is auxiliary information to the information of the physical characteristic sensor system.
• the autonomous guided vehicle further comprises a controller connected to the frame, operably connected to the drive section or the payload handler, and communicably connected to the physical characteristic sensor system, wherein the controller is configured to determine from the information of the physical characteristic sensor system vehicle pose and location effecting independent guidance of the autonomous guided vehicle traversing the facility.
• the controller is configured to determine from the information of the physical characteristic sensor system payload pose and location effecting independent underpick and place of the payload to and from the storage location and independent underpick and place of the payload in the payload hold.
• the controller is programmed with a reference representation of predetermined features defining at least part of the facility traversed through by the autonomous guided vehicle.
• the controller is configured to register the captured image data and generate therefrom at least one image of one or more features of the predetermined features, the at least one image being formatted as a virtual representation of the one or more predetermined features so as to provide comparison to one or more corresponding reference of the predetermined features of the reference representation.
• the controller is configured so that the virtual representation, of the imaged one or more features of the predetermined features, is effected resident on the autonomous guided vehicle, and comparison between the virtual representation of the one or more imaged predetermined features and the one or more corresponding reference predetermined features is effected resident on the autonomous guided vehicle.
• the controller is configured to confirm autonomous guided vehicle pose and location information registered by the controller from the physical characteristic sensor system based on the comparison between the virtual representation and the reference representation.
• the controller is configured to identify a variance in the autonomous guided vehicle pose and location based on the comparison between the virtual representation and the reference representation, and update or complete autonomous guided vehicle pose or location information from the physical characteristic sensor system based on the variance.
• the controller is configured to determine a pose error in the information from the physical characteristic sensor system and fidelity of the autonomous guided vehicle pose and location information from the physical characteristic sensor system based on at least one of the identified variance and analysis of the at least one image, and assign a confidence value according to at least one of the pose error and the fidelity.
• the controller is configured so that with the confidence value below a predetermined threshold, the controller switches autonomous guided vehicle navigation based on pose and location information generated from the virtual representation in place of pose and location information from the physical characteristic sensor system.
• the controller is configured to:
• [0244] initiate communication to an operator identifying autonomous guided vehicle kinematic data and a destination for operator selection of autonomous guided vehicle control from automatic operation to quasi automatic operation or manual operation via a user interface device.
• the controller is configured to confirm payload pose and location information registered by the controller from the physical characteristic sensor system based on the comparison between the virtual representation and the reference representation.
• the controller is configured to identify a variance in the payload pose and location based on the comparison between the virtual representation and the reference representation, and update or complete payload pose or location information from the physical characteristic sensor system based on the variance.
• the controller is configured to determine a pose error in the information from the physical characteristic sensor system and fidelity of the payload pose and location information from the physical characteristic sensor system based on at least one of the identified variance and analysis of the at least one image, and assign a confidence value according to at least one of the pose error and the fidelity.
• the controller is configured so that with the confidence value below a predetermined threshold, the controller switches autonomous guided vehicle payload handling based on pose and location information generated from the virtual representation in place of pose and location information from the physical characteristic sensor system.
• the controller is configured to:
• the controller is configured to transmit, via a wireless communication system communicably coupling the controller and an operator interface, a simulation image combining the virtual representation of the one or more imaged predetermined features and one or more corresponding reference predetermined features of a reference presentation presenting the operator with an augmented reality image in real time.
• the controller is configured to receive real time operator commands to the traversing autonomous guided vehicle, which commands are responsive to the real time augmented reality image, and changes in the real time augmented reality image transmitted to the operator by the controller.
• the supplemental sensor system at least in part effects on-the-fly justification and/or sortation of case units onboard the autonomous guided vehicle.
• imaged or viewed objects described by one or more of supplemental information, supplemental vehicle navigation pose or location, and supplemental payload pose or location, from the auxiliary sensor system are coapted to a reference model of one or more of surrounding facility features and interfacing facility features so as to enhance, via the one or more of the supplemental information, the supplemental vehicle navigation pose or location, and the supplemental payload pose or location resolution of one or more of the vehicle navigation pose or location information and the payload pose or location information.
• a method comprises:
• a drive section coupled to the frame with drive wheels supporting the autonomous guided vehicle on a traverse surface, the drive wheels effect vehicle traverse on the traverse surface moving the autonomous guided vehicle over the traverse surface in a facility, and
• a payload handler coupled to the frame configured to transfer a payload, with a flat undeterministic seating surface seated in the payload hold, to and from the payload hold of the autonomous guided vehicle and a storage location, of the payload, in a storage array;
• [0261] generating sensor data with physical characteristic sensor system, the sensor data embodying at least one of a vehicle navigation pose or location information and payload pose or location information, wherein the physical characteristic sensor system connected to the frame and has electro-magnetic sensors, each responsive to interaction or interface of a sensor emitted or generated electro-magnetic beam or field with a physical characteristic, the electro-magnetic beam or field being disturbed by interaction or interface with the physical characteristic, and which disturbance is detected by and effects sensing by the electro-magnetic sensor of the physical characteristic; and
• capturing image data with a supplemental sensor system the image data informing the at least one of a vehicle navigation pose or location and payload pose or location supplement to the information of the physical characteristic sensor system, wherein the supplemental sensor system is connected to the frame and supplements the physical characteristic sensor system, the supplemental sensor system being, at least in part, a vision system with cameras disposed to capture the image data.
• the method further comprises determining, with a controller, from the information of the physical characteristic sensor system vehicle pose and location effecting independent guidance of the autonomous guided vehicle traversing the facility, wherein the controller is connected to the frame and operably connected to the drive section or the payload handler, and communicably connected to the physical characteristic sensor system.
• the method further comprises, with the controller, determining from the information of the physical characteristic sensor system payload pose and location effecting independent underpick and place of the payload to and from the storage location and independent underpick and place of the payload in the payload hold.
• the controller is programmed with a reference representation of predetermined features defining at least part of the facility traversed through by the autonomous guided vehicle.
• the method further comprises, with the controller, registering the captured image data and generating therefrom at least one image of one or more features of the predetermined features, the at least one image being formatted as a virtual representation of the one or more predetermined features so as to provide comparison to one or more corresponding reference of the predetermined features of the reference representation.
• the controller is configured so that the virtual representation, of the imaged one or more features of the predetermined features, is effected resident on the autonomous guided vehicle, and comparison between the virtual representation of the one or more imaged predetermined features and the one or more corresponding reference predetermined features is effected resident on the autonomous guided vehicle.
• the method further comprises, with the controller, confirming autonomous guided vehicle pose and location information registered by the controller from the physical characteristic sensor system based on the comparison between the virtual representation and the reference representation.
• the method further comprises, with the controller, identifying a variance in the autonomous guided vehicle pose and location based on the comparison between the virtual representation and the reference representation, and updating or completing autonomous guided vehicle pose or location information from the physical characteristic sensor system based on the variance.
• the controller determines a pose error in the information from the physical characteristic sensor system and fidelity of the autonomous guided vehicle pose and location information from the physical characteristic sensor system based on at least one of the identified variance and analysis of the at least one image, and assign a confidence value according to at least one of the pose error and the fidelity.
• the controller switches autonomous guided vehicle navigation based on pose and location information generated from the virtual representation in place of pose and location information from the physical characteristic sensor system.
• the controller is configured to: [0273] continue autonomous guided vehicle navigation to destination or select an autonomous guided vehicle safe path and trajectory bringing the autonomous guided vehicle from a position at switching to a safe location for shut down, or
• [0274] initiate communication to an operator identifying autonomous guided vehicle kinematic data and a destination for operator selection of autonomous guided vehicle control from automatic operation to guasi automatic operation or manual operation via a user interface device.
• the controller confirms payload pose and location information registered by the controller from the physical characteristic sensor system based on the comparison between the virtual representation and the reference representation.
• the controller identifies a variance in the payload pose and location based on the comparison between the virtual representation and the reference representation, and update or complete payload pose or location information from the physical characteristic sensor system based on the variance.
• the controller determines a pose error in the information from the physical characteristic sensor system and fidelity of the payload pose and location information from the physical characteristic sensor system based on at least one of the identified variance and analysis of the at least one image, and assign a confidence value according to at least one of the pose error and the fidelity.
• the controller switches autonomous guided vehicle payload handling based on pose and location information generated from the virtual representation in place of pose and location information from the physical characteristic sensor system.
• the controller is configured to:
• [0281] initiate communication to an operator identifying payload data along with an operator selection of autonomous guided vehicle control from automatic payload handling operation to quasi automatic payload handling operation or manual payload handling operation via a user interface device.
• the controller transmits, via a wireless communication system communicably coupling the controller and an operator interface, a simulation image combining the virtual representation of the one or more imaged predetermined features and one or more corresponding reference predetermined features of a reference presentation presenting the operator with an augmented reality image in real time.
• the controller receives real time operator commands to the traversing autonomous guided vehicle, which commands are responsive to the real time augmented reality image, and changes in the real time augmented reality image transmitted to the operator by the controller.
• controller effects, with at least the supplemental sensor system, justification and/or sortation of case units onboard the autonomous guided vehicle.
• imaged or viewed objects described by one or more of supplemental information, supplemental vehicle navigation pose or location, and supplemental payload pose or location, from the supplemental sensor system are coapted to a reference model of one or more of surrounding facility features and interfacing facility features so as to enhance, via the one or more of the supplemental information, the supplemental vehicle navigation pose or location, and the supplemental payload pose or location resolution of one or more of the vehicle navigation pose or location information and the payload pose or location information.
• an autonomous guided vehicle comprises:
• a frame with a payload hold [0287] a frame with a payload hold; [0288] a drive section coupled to the frame with drive wheels supporting the vehicle on a traverse surface, the drive wheels effect vehicle traverse on the traverse surface moving the vehicle over the traverse surface in a facility;
• a payload handler coupled to the frame configured to transfer a payload to and from the payload hold of the vehicle and a storage location, of the payload, in a storage array;
• a supplemental sensor system connected to the frame for collaboration of the vehicle and an operator, supplemental sensor system supplements a vehicle autonomous navigation/operation sensor system configured to at least collect sensory data embodying vehicle pose and location information for auto navigation by the vehicle of the facility,
• the supplemental sensor system is, at least in part, a vision system with at least one camera disposed to capture image data informing objects and/or spatial features within at least a portion of the facility viewed by the at least one camera with the vehicle in different positions in the facility; and
• a controller connected to the frame and communicably coupled to the supplemental sensor system so as to register the information from the image data of the at least one camera, and the controller is configured to determine, from the information, presence of a predetermined physical characteristic of at least one object or spatial feature, and in response thereto, selectably reconfigure the vehicle from an autonomous state to a collaborative vehicle state disposed to receive operator commands for the vehicle to continue effecting vehicle operation.
• the predetermined physical characteristic is that the at least one object or spatial feature extends across at least part of, the traverse surface, a vehicle traverse path across the traverse surface or through space of the vehicle or another different vehicle traversing the traverse surface
• the controller is programmed with a reference representation of predetermined features defining at least in part the facility traversed through by the vehicle.
• the controller is configured to register the captured image data and generate therefrom at least one image of the at least one object or spatial feature showing the predetermined physical characteristic.
• the at least one image is formatted as a virtual representation of the at least one object or spatial feature so as to provide comparison to one or more reference features of the predetermined features of the reference representation.
• the controller is configured to identify the presence of the predetermined physical characteristic of the object or spatial feature based on the comparison between the virtual representation and the reference representation, determine a dimension of the predetermined physical characteristic and command the vehicle to stop in a predetermined trajectory based on a position of the object or spatial features determined from the comparison.
• a stop position in the predetermined trajectory maintains object or spatial reference within field of view of at least one camera and continued imaging of the predetermined physical characteristic, initiates a signal to at least another vehicle of one or more of a traffic obstacle, an area to avoid, or a detour area.
• the predetermined physical characteristic is determined by the controller by determining a position of the object within a reference frame of the at least one camera, that is calibrated and has a predetermined relationship to the vehicle, and from the object pose in the reference frame of the at least one camera determine presence of predetermined physical characteristic of object.
• the controller is configured such that, identification of presence and switch from the autonomous state to the collaborative vehicle state, the controller initiates transmission communicating image, identification of presence of predetermined physical characteristic, to operator interface for operator collaboration operation of the vehicle.
• the controller is configured to apply a trajectory to the autonomous guided vehicle that brings the autonomous guided vehicle to a zero velocity within a predetermined time period where motion of the autonomous guided vehicle along the trajectory is coordinated with location of the objects and/or spatial features.
• the capture of image data informing objects and/or spatial features is opportunistic during transfer of a payload to/from the payload hold of the vehicle or a storage location in a storage array.
• the controller is programmed to command the vehicle to the different positions in the facility associated with the vehicle effecting one or more predetermined payload autonomous transfer tasks, wherein each of the one or more predetermined payload autonomous transfer tasks is a separate and distinct task from the capture image data viewed by the at least one camera in the different positions.
• the controller is configured so that determination of presence of the predetermined physical characteristic of the at least one object or spatial feature is, coincident at least in part with, but supplemental and peripheral to vehicle actions effecting each of the one or more predetermined payload auto transfer tasks.
• the controller is configured so that determination of presence of the predetermined physical characteristic of the at least one object or spatial feature is, opportunistic to vehicle actions effecting each of the one or more predetermined payload auto transfer tasks.
• At least one of the one or more predetermined payload auto transfer tasks is effected at at least one of the different positions.
• the collaborative vehicle state is supplemental to the autonomous state of the vehicle effecting each of the one or more predetermined payload auto transfer tasks.
• a method comprises:
• a drive section coupled to the frame with drive wheels supporting the vehicle on a traverse surface, the drive wheels effect vehicle traverse on the traverse surface moving the vehicle over the traverse surface in a facility;
• a payload handler coupled to the frame configured to transfer a payload to and from the payload hold of the vehicle and a storage location, of the payload, in a storage array;
• the predetermined physical characteristic is that the at least one object or spatial feature extends across at least part of, the traverse surface, a vehicle traverse path across the traverse surface or through space of the vehicle or another different vehicle traversing the traverse surface.
• the controller is programmed with a reference representation of predetermined features defining at least in part the facility traversed through by the vehicle.
• the method further comprises generating, from the registered captured image data, at least one image of the at least one object or spatial feature showing the predetermined physical characteristic.
• the at least one image is formatted as a virtual representation of the at least one object or spatial feature, the method further comprising comparing the virtual representation to one or more reference features of the predetermined features of the reference representation.
• the method further comprises identifying, with the controller, the presence of the predetermined physical characteristic of the object or spatial feature based on the comparison between the virtual representation and the reference representation, determining a dimension of the predetermined physical characteristic, and commanding the vehicle to stop in a predetermined trajectory based on a position of the object or spatial features determined from the comparison.
• the method further comprises, with the vehicle in a stop position in the predetermined trajectory, maintaining the object or spatial reference within a field of view of the at least one camera and continued imaging of the predetermined physical characteristic, initiating a signal to at least another vehicle of one or more of a traffic obstacle, an area to avoid, or a detour area.
• the predetermined physical characteristic is determined by the controller by determining a position of the object within a reference frame of the at least one camera, that is calibrated and has a predetermined relationship to the vehicle, and from the object pose in the reference frame of the at least one camera determine presence of predetermined physical characteristic of the object.
• the controller is configured such that, identification of presence of the predetermined physical characteristic of the at least one object or spatial feature and switch from the autonomous state to the collaborative vehicle state, initiates transmission communicating image, identification of presence of predetermined physical characteristic, to an operator interface for operator collaboration operation of the vehicle.
• the method further comprises applying, with the controller, a trajectory to the autonomous guided vehicle bringing the autonomous guided vehicle to a zero velocity within a predetermined time period, where motion of the autonomous guided vehicle along the trajectory is coordinated with a location of the objects and/or spatial features.
• the capture of image data informing objects and/or spatial features is opportunistic during transfer of a payload to/from the payload hold of the vehicle or a storage location in a storage array.
• an autonomous guided vehicle comprises:
• a payload handling section with at least one payload handling actuator configured so that actuation of the at least one payload handling actuator effects transfer of a payload to and from a payload bed, of the vehicle chassis, and a storage in the facility;
• a peripheral electronics section having at least one of an autonomous pose and navigation sensor, at least one of a payload handling sensor, and at least one peripheral motor, the at least one peripheral motor being separate and distinct from each of the motors of the drive section and each actuator of the payload handling section;
• a controller communicably coupled respectively to the drive section, the payload handling section, and peripheral section so at to effect each autonomous operation of the autonomous guided vehicle, wherein the controller comprises a comprehensive power management section communicably connected to the power supply so as to monitor a charge level of the power supply, and
• the comprehensive power management section is connected to each respective branch circuit of the drive section, the payload handling section, and the peripheral electronics section respectively powering the drive section, the payload handling section, and the peripheral electronics section from the power supply, the comprehensive power management section being configured to manage power consumption of the branch circuits based on a demand level of each branch circuit relative to the charge level available from the power supply.
• the comprehensive power management section is configured so as to manage a demand charge level of each respective branch circuit switching each respective branch circuit on or off in a predetermined pattern based on the demand charge level of each respective branch circuit with respect to other branch circuits and the charge level available from the power supply.
• the predetermined pattern is arranged to switch off branch circuits with a decrease in the available charge level from the power supply, so as to maximize available charge level from the power supply directed to the controller.
• the predetermined pattern is arranged to switch off branch circuits with a decrease in the available charge level from the power supply so that the available charge level directed to the controller is equal to or exceeds the demand charge level of the controller for a maximum time based on the available charge level of the power supply.
• the controller has at least one of:
• an autonomous navigation control section configured to register and hold in volatile memory autonomous guided vehicle state and pose navigation information, historic and current, that is deterministic of and describing current and predicted state, pose, and location of the autonomous guided vehicle; and [0338] an autonomous payload handling control section configured to register and hold in volatile memory current payload identity, state, and pose information, historic and current;
• controller is configured so that upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the controller, to less than demand level of the controller, the controller configures at least one of the autonomous guided vehicle state and pose navigation information and the payload identity, state, and pose information, held in respective registry and memory of corresponding controller sections, into an initialization file available on reboot of the controller.
• the controller is configured so that upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the controller, to less than demand level of the controller, the controller enters suspension of operation and hibernation.
• the controller is configured so that upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the branch circuit of the drive section, the controller is configured to command the drive section so as to navigate the autonomous guided vehicle along a predetermined auxiliary path and auxiliary trajectory (to a predetermined autonomous guided vehicle auxiliary stop location in the facility.
• the controller is configured so that upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the branch circuit of the payload handling section the controller is configured to command the payload handling section to move the payload handling actuator, and any payload thereon, to a predetermined safe payload position in the payload bed.
• the controller includes at least one of:
• controller is configured so that upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the controller, to less than demand level of the controller, to configure stored health status information from the at least one of the vehicle health status monitor, the drive section health status monitor, the payload handling section health monitor, and the peripheral electronics section health monitor in the health status register section into an initialization file available on reboot of the controller.
• the power supply is an ultra-capacitor, or the charge level is voltage level.
• the method comprises:
• an autonomous guided vehicle with a vehicle chassis with a power supply mounted thereon and powered sections connected to the chassis and each powered by the power supply, the powered sections including:
• a payload handling section with at least one payload handling actuator configured so that actuation of the at least one payload handling actuator effects transfer of a payload to and from a payload bed, of the vehicle chassis, and a storage in the facility;
• a peripheral electronics section having at least one of an autonomous pose and navigation sensor, at least one of a payload handling sensor, and at least one peripheral motor, the at least one peripheral motor being separate and distinct from each of the motors of the drive section and each actuator of the payload handling section;
• the comprehensive power management section manages a demand charge level of each respective branch circuit switching each respective branch circuit on or off in a predetermined pattern based on the demand charge level of each respective branch circuit with respect to other branch circuits and the charge level available from the power supply.
• the predetermined pattern is arranged to switch off branch circuits with a decrease in the available charge level from the power supply, so as to maximize available charge level from the power supply directed to the controller.
• the predetermined pattern is arranged to switch off branch circuits with a decrease in the available charge level from the power supply so that the available charge level directed to the controller is equal to or exceeds the demand charge level of the controller for a maximum time based on the available charge level of the power supply.
• the method further comprises at least one of:
• the controller upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the controller, to less than demand level of the controller, the controller configures at least one of the autonomous guided vehicle state and pose navigation information and the payload identity, state, and pose information, held in respective registry and memory of corresponding controller sections, into an initialization file available on reboot of the controller.
• the controller upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the branch circuit of the drive section, commands the drive section so to navigate the autonomous guided vehicle along a predetermined auxiliary path and auxiliary trajectory to a predetermined autonomous guided vehicle auxiliary stop location in the facility.
• the controller upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the branch circuit of the payload handling section the controller commands the payload handling section to move the payload handling actuator, and any payload thereon, to a predetermined safe payload position in the payload bed.
• the controller upon indication from the comprehensive power management section of imminent decrease in available charge level, directed from the power supply to the controller, to less than demand level of the controller, the controller configures stored health status information from the at least one of the vehicle health status monitor, the drive section health status monitor, the payload handling section health monitor, and the peripheral electronics section health monitor in the health status register section into an initialization file available on reboot of the controller.
• the power supply is an ultra-capacitor, or the charge level is voltage level.

Landscapes

• Engineering & Computer Science (AREA)
• Transportation (AREA)
• Structural Engineering (AREA)
• Mechanical Engineering (AREA)
• Geology (AREA)
• Life Sciences & Earth Sciences (AREA)
• Civil Engineering (AREA)
• Physics & Mathematics (AREA)
• Automation & Control Theory (AREA)
• General Physics & Mathematics (AREA)
• Remote Sensing (AREA)
• Radar, Positioning & Navigation (AREA)
• Aviation & Aerospace Engineering (AREA)
• Chemical & Material Sciences (AREA)
• Combustion & Propulsion (AREA)
• Electromagnetism (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Multimedia (AREA)
• Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
• Warehouses Or Storage Devices (AREA)

Applications Claiming Priority (6)

Publications (2)

Family

ID=85200328

Family Applications (1)

Country Status (5)

Cited By (1)

Families Citing this family (3)

Family Cites Families (5)

• 2022
  • 2022-05-26 JP JP2023573211A patent/JP2024532647A/ja active Pending
  • 2022-05-26 EP EP22856718.6A patent/EP4384470A4/de active Pending
  • 2022-05-26 KR KR1020237044782A patent/KR20240046119A/ko active Pending
  • 2022-05-26 CA CA3220378A patent/CA3220378A1/en active Pending
  • 2022-05-26 WO PCT/US2022/072592 patent/WO2023019038A1/en not_active Ceased

Also Published As

Similar Documents

Legal Events