Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!wuarchive!gem.mps.ohio-state.edu!usc!ucsd!ucbvax!VENUS.YCC.YALE.EDU!BUEHLER%KOBOT From: BUEHLER%KOBOT@VENUS.YCC.YALE.EDU ("buehler%kobot.decnet@venus.ycc.yale.edu") Newsgroups: comp.sys.transputer Subject: real time vision frontend Message-ID: <8909241836.AA13173@tcgould.TN.CORNELL.EDU> Date: 24 Sep 89 18:40:00 GMT Sender: daemon@ucbvax.BERKELEY.EDU Organization: The Internet Lines: 150 David Lowe writes: // I am looking for information from anyone who has experience interfacing // a transputer network to some type of image digitizer for near real-time // computation. // // The problem, of course, is that your average TV camera outputs data // at the rate of about 8 megabytes/sec when digitized. Here at the // Laboratory for Computational Vision, we already have Datacube image // processing equipment that can digitize and do local processing on // this data at video rates. However, for higher-level processing the // data must be transferred over a VME bus at far slower rates. Has // anyone looked at parallel input to transputers from a MAXBUS or // other video data source? Are there general high-speed IO solutions // for transputers? Any information appreciated. We need real time vision in our robotics work in the Yale Robotics Lab. About a year ago we evaluated numerous commercial vision systems. None of the systems excited us enough to buy it (cost, performance, interfacing, flexibility, complexity, etc.) so we built a simple but powerful distributed vision system based on Transputers/A110 ourselves. Below you find a short abstract-description as it was sent to the Natug89 meeting. A more detailed description can be found in the Natug-1 proceedings (Natug-1 Salt Lake City, UT, April 5-6, 1989, to obtain ($32)from G.S.Stiles, Dept. of El.Eng., Utah State University, Logan, UT, 84322, stiles@cc.USU.EDU), or else I can email you the latex version of the paper. At this opportunity I also would like to ask if anybody else built a "real-time" vision system, succeeded to interface cleanly to a commercial system or has any thoughts about real-time vision hardware in general. I would be very interested in seeing some comments on this net. martin (buehler%kobot.decnet@venus.ycc.yale.edu) ============================================================================= The Cyclops Vision System (ABSTRACT) Abstract submitted for the North American Transputer Users Group Meeting. April 5/6, 1989 Salt Lake City, Utah M. Buehler, C. J. Taylor, N. Vlamis, and A. Ganz Center for Systems Science Yale University, Department of Electrical Engineering We describe a real-time distributed vision system frontend for transputer based systems. The need for real time vision arose out of our robotics research with dynamical tasks, specifically tracking balls for robot juggling: The system must be able to track at least two objects in 3D (stereo-vision) at field rates (60 Hz). For computation and robot control we are using networks of our Yale transputer based distributed control nodes, the XP/DCS (Proc. 1988 National OCCAM Users Group Meeting). The motivation to develop our own vision system as opposed to buying some commercial unit is similar to the reasons why we chose to build our own distributed control node two years ago: * networking via the transputer links achieves flexibility and virtually unlimited expandability * system is built to satisfy all our requirements * by using OCCAM and the standard development environment we avoid the development of system software * high performance and simplicity is achieved by employing state of the art devices (Transputer,A110,AD9502,FastSram) * low cost (entry boards cost: 10k for stereo 60Hz, without A110 Filter Bd.) As Cyclops incorporates the transputer based XP/DCS, it ideally matches our XP/DCS based control system for interconnection via communication links. Furthermore a distributed vision system architecture connecting to a distributed control topology has another advantage for many applications: Results that are computed in one part of the vision system can be provided directly to the appropriate part of the control topology. Cyclops consists of five main elements: A camera, a digitizer board, a (optional) filter board and one or many parallel video memory boards, each of which connects to a XP/DCS CPU board. The functions are straightforeward --- the camera provides interlaced fields of video data to the digitizer, which converts the analog signal into an eight bit pixel stream, accompanied with some synchronization signals. The video bus containing the pixel stream is brought to the filter board. Here any two dimensional filter or convolution with a 6x14 pixel size window can be applied to the data in real time. Finally the video data is written into one or several video memory boards, where it can be accessed by its XP/DCS transputer. All memory boards attach to the video bus in parallel. Only its drive and noise limitations restricts the maximum number of video memory boards. Even though Cyclops satisfies a specific need, it is very flexible in its operation so as to allow its application to most vision tasks. In order to illustrate its generality, we will describe three different ways to distribute the video fields to the video memory boards and the implications for update rate and latency. By update rate we mean the rate at which results are computed by any of the processing boards on the video bus whereas latency refers to the total processing time from raw video data to end result. **Group Mode: the video fields are loaded into all memory boards simultaneously. Depending on the application, each processor then picks a dynamic or static partition of the full field for processing. A new field is loaded after all processors have finished processing --- thus latency is equal to update rate. **Scan Mode: for many applications, the video data is not easily separable into partitions without requiring extensive inter-processor communication and decrease performance. In this case one can achieve an increased update rate while keeping the same latency as with only one memory board by sequencially scanning successive fields into different memory boards. Maximally eight memory boards can receive different fields. **Mixed Mode: this mode combines scan and group mode. Successive fields are scanned into groups of boards as opposed to into single board as in scan mode. This reduces latency when compared to the scan mode. A monocular Cyclops (without the filter board, which is currently under construction) was built and tested by Taylor and Vlamis as a semester project for a computer architecture class offered during fall 1988 by the last author. It consists of one digitizer board and one video memory board, and tracked two ping pong balls at 30 Hz. (Using very unsophisticated initial software and dark background. With two video memory boards the update rate is 60 Hz, the latency worst case is 1/30 sec). So we have not pushed the performance yet. The critical ingredients that made this project a successful one were the existing design experience with the transputer in the Yale robotics lab, the availability of the XP/DCS CPU board, high speed CCD cameras with electronic shutter, single chip eight bit video digitizers, INMOS 2d transform integrated circuits (A110), and affordable high speed static memories. ===============================================================================