購買設計請充值后下載，，資源目錄下的文件所見即所得，都可以點開預覽，，資料完整，充值下載可得到資源目錄里的所有文件。。?！咀ⅰ浚篸wg后綴為CAD圖紙，doc,docx為WORD文檔，原稿無水印，可編輯。。。具體請見文件預覽，有不明白之處，可咨詢QQ:12401814

1496 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 42, NO. 6, NOVEMBER 2012 Using Machine Vision and Hand-Motion Control to Improve Crane Operator Performance Kelvin Chen Chih Peng, William Singhose, and Purnajyoti Bhaumik Abstract—The payload oscillation inherent to all cranes makes it challenging for human operators to manipulate payloads quickly, accurately, and safely. Manipulation difficulty is also increased by nonintuitive crane-control interfaces. This paper describes a new interface that allows operators to drive a crane by moving a hand-held device (wand or glove) freely in space. A crane-mounted camera tracks the movement of the hand-held device, the position of which is used to drive the crane. Two control architectures were investigated. The first uses a simple feedback controller, and the second uses feedback and an input shaper. Two operator studies demonstrate that hand-motion crane control is faster and safer than using a standard push-button pendent control. Index Terms—Control interface, cranes, input shaping, machine vision, oscillation. I. INTRODUCTION RANES PLAY a key role in maintaining the economic Fig. 1. Standard push-button pendent crane control. In addition to facing the challenges of controlling large- amplitude lightly-damped payload swing, operators must also C vitality of modern-day industry. Their importance can be master nonintuitive control interfaces. Fig. 1 shows the pendent seen at shipyards, construction sites, and warehouses and in a wide variety of material-handling applications. The effec- tiveness of crane manipulation is an important contributor to industrial productivity, low production costs, and worker safety. One inherent property of cranes that is detrimental to efficient operation is the natural tendency for the payload to oscillate like a pendulum, a double pendulum [1], or with even more complex oscillatory dynamics [2]. Significant effort has been made to develop control schemes to reduce the oscillatory response from both issued commands and external disturbances [3]-[9]. There has also been research in controlling cranes that contain rotational joints, which adds an extra level of complexity due to their nonlinear dynamics [10]-[13]. Operators who manip- ulate a crane using traditional interfaces such as push-button pendents benefit from oscillation-suppression technology. They generate safer (less collisions with obstacles) and more efficient crane motions (faster task completion times and less operator button pushes) than operators without such compensation [10], [14]-[16]. Manuscript received September 26, 2010; revised April 7, 2011, June 10, 2011, and February 9, 2012; accepted April 6, 2012. Date of publication June 8, 2012; date of current version October 12, 2012. This work was supported in part by Siemens Industrial Automation, by the Manufacturing Research Center, Georgia Institute of Technology, and by Boeing Research and Technology. This paper was recommended by Associate Editor E. J. Bass. The authors are with the Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA (e-mail: kccpeng@ gatech.edu; Singhose@gatech.edu; pjbhaumik@live.com). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSMCA.2012.2199301 control of a typical overhead crane. The operator must be adept in the cognitive process of transferring the desired manipulation path into a sequence of button presses that will produce the desired crane motion. For example, if the operator wants to drive the crane through a cluttered workspace, then the desired path must be mapped into a sequence of events where the “forward (F),” “backward (B),” “l(fā)eft (L),” and “right (R)” buttons are pushed at the correct time and in the correct se- quence. Furthermore, as operators move through the workspace to drive the crane and monitor its progress, they may rotate their bodies and change the direction they are facing. In such cases, the “forward” button causes motion to the left, right, or even backward. As an additional challenge, the operator can only directly drive the overhead trolley, not the payload. Therefore, the operator must account for the time lag between the commanded motion of the trolley, which can be many meters overhead, and the delayed oscillatory response of the payload. While significant strides have been made to improve the operational efficiency of cranes by controlling the dynamic response to issued commands, relatively little consideration has been given to the way in which operators issue those commands [17]. It has been proven that interfaces that are tailored to the cognitive processes associated with specific control systems have beneficial effects [18]-[20]. For example, in the field of laparoscopic surgery, medical robots such as the da Vinci improve on the traditional procedure by allowing surgeons to operate in a more ergonomic manner and with less cognitive load [21], [22]. The controls move in the same direction as the end effectors for da Vinci, unlike traditional laparoscopic 1083-4427/$31.00 ? 2012 IEEE PENG et al.: USING MACHINE VISION AND MOTION CONTROL TO IMPROVE CRANE OPERATOR PERFORMANCE 1497 procedures where surgeons have to reverse map the controls due to the instruments’ pivot point at the point of insertion. This paper presents a novel control interface that allows an operator to drive a crane by moving a hand-held device in space. Machine vision is used to track the position of the device (a wand or a glove), which is then used to generate the command signal to drive the crane. The hand-motion control interface is well tailored to the task of driving a crane through a cluttered workspace because it eliminates the cognitive map- ping process that is necessary with traditional control interfaces. As a result, operators no longer need to account for the direction in which they are facing. The manual dexterity required for safe and efficient operation is also reduced. Additionally, the control algorithm minimizes payload swing without signifi- cantly slowing the system response. Therefore, the burden of manually reducing payload oscillation is removed. This allows the operator to concentrate solely on the path planning and final positioning of the payload. Hand-motion control offers other cognitive advantages over traditional interfaces. There are two primary divisions of cogni- tive control: analytic problem solving and perceptual process- ing [23]. Perceptual processing tends to be faster and can be performed in parallel, while analytic processing takes longer and typically progresses serially. Analytic problem solving also tends to be more prone to error [23], [24]. The results of many studies also suggest that people prefer, and adopt, perceptual processing when possible [16], [23], [25], [26]. From this perspective, hand-motion control helps operators by lowering the cognition level required to drive the crane. Operators no longer need to think analytically about the sequence of buttons to push or to account for the swinging payload; they only need to move the hand-held device to the desired position or along a desired path. This allows the operators to perform simpler perceptual processing. The major contribution of this paper is the novel hand-motion control interface. The benefits of this interface are validated by human operator studies. Section II describes the novel inter- faces (the wand and glove). The control algorithms that are used in conjunction with the interfaces are discussed in Section III. This is followed by the operator studies in Section IV and conclusions in Section V. II. INTERFACES FOR HAND-MOTION CRANE CONTROL The application investigated in this paper is for a single- pendulum point-mass payload that is suspended from a motor- ized overhead crane. The 10-ton industrial bridge crane shown in Fig. 2 was used for experimental verifications. A bridge crane consists of a fixed overhead runway, a bridge that travels along the runway, and a trolley that runs along the bridge. Laser range sensors measure the trolley position along the runway and the bridge. The hook, which represents the payload, is suspended from the trolley by cables. A Siemens programmable logic controller is used to control the motor drives and acts as the central control unit. Commands to the crane can be issued with a push-button control pendent, the wand or glove for hand-motion control, or other devices [17]. A downward- pointing Siemens Simatic VS723-2 camera mounted on the Fig. 2. Typical bridge crane. Fig. 3. Driving a crane by moving a reflective wand. trolley measures the position of the hook. Reflectors mounted on the topside of the hook aid vision-detection algorithms [27]. There are two hand-held devices for hand-motion crane control: 1) The wand, shown in Fig. 3, is a reflective ball mounted to the end of a hand-held pole, and 2) the glove, shown in Fig. 4, has a circular reflector attached to the backside. Fig. 5 shows a schematic diagram of hand-motion control using machine vision. The crane-mounted camera is used to concurrently track the positions of the wand/glove and the hook. Because all reflectors appear as bright blobs in the camera image, a K -means clustering algorithm is used to distinguish the wand/glove reflectors from the hook reflectors [28]. The camera refresh rate is approximately 140 ms. The position of the wand/glove relative to the crane is used to generate an error signal to drive the overhead trolley. 1498 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 42, NO. 6, NOVEMBER 2012 Fig. 6. Standard pendent controller. Fig. 7. Standard push-button pendent control response. Fig. 4. Driving a crane by moving a reflective glove. Fig. 8. PD hand-motion controller. A. Standard Push-Button Pendent Control The block diagram for standard pendent control is shown in Fig. 6. The operator analyzes the workspace, considers the required manipulation goal, and then decides on a course of action. This plan is then implemented by pushing buttons on the control pendent. These buttons send energy to the motors and move the overhead crane trolley. The suspended payload is moved indirectly by the motion of the trolley. Fig. 5. Schematic of hand-motion crane control. III. HAND-MOTION CRANE CONTROLLERS Three control architectures were investigated. First, the stan- dard push-button pendent controller was used as the baseline for performance comparisons. Then, a proportional-derivative (PD) feedback controller was investigated for its suitability in hand-motion crane control. Finally, an input shaper was added to the PD controller in order to reduce payload swing. Note that, from the perspective of the control architecture, the wand and the glove are identical. Both devices are used to communicate the operator’s desired position to the controller. For this reason, there is no distinction between the wand and glove in the simulation and experimental verification results that are presented in this section. However, in terms of er- gonomics during operation, the wand has a greater reach and can drive the crane toward tight spaces, such as corners. On the other hand, the glove sacrifices range of reach for a smaller size and ease of use. Computer-simulated responses for point-to-point movements of approximately 2 and 3 m using the pendent controller are shown in Fig. 7. Pressing a pendent button for a certain amount of time issues a trapezoidal velocity command to the crane motors. Due to the pendulum-like nature of the payload, this type of trolley movement will, in general, induce significant payload oscillations. B. PD Hand-Motion Control The well-known and popular PD controller represents one of the simplest forms of feedback control. It is the most commonly used feedback method in industry and has been applied to the control of cranes [29], [30]. It provides a realistic choice for hand-motion crane controllers. The PD hand-motion control block diagram is shown in Fig. 8. The position of the wand or glove is compared to the position of the overhead crane (neglecting the vertical height difference) to generate the error signal e. The command generator converts the error signal (a positional measurement) into a velocity command that can be sent to the motor drives. If e is within the designed range specified by e0 and e100 , then the command generator linearly PENG et al.: USING MACHINE VISION AND MOTION CONTROL TO IMPROVE CRANE OPERATOR PERFORMANCE Fig. 9. Simulated PD controller with low gains. Fig. 10. Simulated PD controller with high gains. scales e. Otherwise, the command generator outputs either 0% or 100%. The values for e0 and e100 were 0.25 and 1.0 m. These were selected based on comfortable distances at which the crane followed the operator. The command generator is described as { 0% : e ≤ e0 Command = 100% × e e??0 : e0 < e < e100 (1)  1499 100% 100 e0 : e ≥ e100 . Fig. 11.  Starting and stopping with hand-motion control. A PD control law is then applied, and the result is passed through a saturator to ensure that crane velocity and accel- eration limits are not exceeded. Note that the position of the crane trolley, rather than of the payload, is used for feedback. This is because, in practice, sensing the position of the trolley (using laser range sensors) is much more reliable than sensing of the payload (using machine vision). Furthermore, the single- pendulum payload is an inherently stable plant: The payload will always come to rest directly beneath a stationary crane. Therefore, correct final positioning of the crane trolley ensures correct final positioning of the payload. 1) Simulation Verification: A crucial design challenge is the selection of PD gains. Computer simulations were constructed to aid the gain-selection process. Hand-motion trajectories were specified as ramps in position with gradients equivalent to the maximum velocity of the 10-ton industrial crane (0.3577 m/s). This is approximately the speed of a slow walk and mimics the typical hand-motion trajectories from a human operator. Figs. 9 and 10 show the simulation results for PD hand- motion controllers with low and high feedback gains, respec- tively. These two figures show the inherent tradeoff in using the PD controller: With low gains, the crane was slow to respond, but the payload oscillation was small; with high gains, the crane moved quickly but at the expense of large payload oscillations. 2) Experimental Verification: The hand-motion control sys- tem was implemented on the 10-ton bridge crane. The wand/glove trajectories produced by human operators were similar to those used in the simulations. The ramp gradient was approximately equivalent to the maximum velocity of the crane, and the move distance was approximately 2 m for the tests reported here. Fig. 11 shows an operator using hand-motion control to start and stop the crane. To start moving, the operator can expose Fig. 12. Experimental PD controller with low gains. the wand/glove to the camera at some distance away from the crane. When the crane approaches the desired position, the operator lowers the wand/glove, which becomes undetectable by the camera. When the camera is unable to locate the position of the wand/glove, e is set to zero. Because the position of the wand/glove may be unknown at certain times, there are breaks in the curves that are labeled “Wand/Glove” in the experimental response plots. Figs. 12 and 13 show the experimental results for PD hand- motion controllers with low and high feedback gains using the glove interface, respectively. The experimental data re- inforce the results that were established by the simulations: Low gains produced slow crane movements and small payload oscillations, while high gains yielded fast crane movements but large payload oscillations. For safety reasons, minimizing payload oscillation is normally a higher priority than fast crane n n 1500 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 42, NO. 6, NOVEMBER 2012 Fig. 13. Experimental PD controller with high gains. Fig. 14. PD with input shaper hand-motion controller. movements. Therefore, practical implementations of PD hand- motion controllers should only use low gains. C. PD With Input Shaper Hand-Motion Control Section III-B demonstrated the inherent weakness in using PD hand-motion controllers (the tradeoff between low and high gains). However, performance can be improved with the addi- tion of an input shaper that modifies the shape of the command signal to reduce oscillation. Fig. 14 shows the new control block diagram that shows an input shaper inserted between the saturator and the crane blocks. 1) Input Shaping: Input shaping is a technique that reduces the residual vibration of flexible systems by properly shaping the commands. This is accomplished by convolving the base- line input command with a series of impulses, known as an input shaper. The result is a shaped command that will reduce  Fig. 15. Simulated PD with input shaper controller. Fig. 16. Experimental PD with input shaper controller. ω is the natural frequency of the system, ζ is the damping ratio, and Ai and ti are the ith impulse amplitude and time, respectively. Equation (2) gives the ratio of vibration with input shaping to that without input shaping. A constraint on residual vibration amplitude can be formed by setting (2) less than or equal to a tolerable level of residual vibration at the modeled natural fre- quency and damping ratio [32]. For the simplest zero vibration (ZV) shaper, the tolerable amount of vibration is set to zero. This results in a shaper of the form [31], [33] [ ] [ ] residual vibration. In order to determine the impulse amplitudes and time loca- tions of an input shaper, certain design constraints must be sat- isfied. The primary design constraint is a limit on the amplitude of vibration caused by the shaper. The normalized percentage ZV = where Ai ti 1 K 1+K 1+K = 0 √ π (5) ω 1?ζ 2 √ ?ζπ residual vibration (PRV) amplitude of an underdamped second- order system from a sequence of n impulses is given by [31] √ P RV = V (ω, ζ ) = e?ζωtn [C (ω, ζ )]2 + [S(ω, ζ )]2 (2) where ∑ √ C (ω, ζ ) = Aieζωti cos(ωti 1 ? ζ 2 ) (3) i=1 ∑ √ S(ω, ζ ) = Aieζωti sin(ωti 1 ? ζ 2 ) (4) i=1 K = e 1?ζ2 . (6) 2) Simulation and Experimental Verifications: The goal of combining high-gain PD feedback with a ZV input shaper is to obtain fast crane response and low-amplitude payload oscillations. The hand-motion controller in this section uses high PD gains (identical to the ones from Section III-B) to move the crane trolley quickly, combined with a ZV input shaper to cancel payloa