USF Libraries
USF Digital Collections

System approach to embedded system design

MISSING IMAGE

Material Information

Title:
System approach to embedded system design
Physical Description:
Book
Language:
English
Creator:
Mehendale, Vikram Prabhakar
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
FPGA
Cyclone II
CMOS sensor
VHDL
Video surveillance
Dissertations, Academic -- Electrical Engineering -- Masters -- USF   ( lcsh )
Genre:
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: During this research, the concepts of Systems Engineering were applied to embedded system design. The objective was to apply the Systems Engineering methodology to the design of a particular embedded system. A Video Surveillancesystem was chosen as the particular embedded system. Systems Engineering concepts provide the foundation for an optimized design process and for the coordination between system modules. The functionality of the Video Surveillance system was achieved through the partitioning of the overall system functionality into three separate modules. The three modules were Image Capture, Image Processing and Image Transmission. The methodology employed resulted in a system that was flexible and portable. The three modules were designed using their own set of specifications and with completely defined linking interfaces. Following a concrete set of specifications resulted in a system, which can be modified at any later stage without the necessity of changing the whole architecture. The Video Surveillance system fulfilled the overall system requirements as well as those imposed by the subsystems. The partitioning of functionality resulted in ease of implementation and better upgradeability. Design based on Systems Engineering concepts provides for ease of integration. In addition, for modules that follow the same protocol, the existence of well defined interfaces enables connectivity to a variety of external units.
Thesis:
Thesis (M.S.)--University of South Florida, 2007.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Vikram Prabhakar Mehendale.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 49 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001935142
oclc - 225866019
usfldc doi - E14-SFE0002282
usfldc handle - e14.2282
System ID:
SFS0026600:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001935142
003 fts
005 20080421150209.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 080421s2007 flua sbm 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0002282
040
FHM
c FHM
035
(OCoLC)225866019
049
FHMM
090
TK145 (ONLINE)
1 100
Mehendale, Vikram Prabhakar.
0 245
System approach to embedded system design
h [electronic resource] /
by Vikram Prabhakar Mehendale.
260
[Tampa, Fla.] :
b University of South Florida,
2007.
3 520
ABSTRACT: During this research, the concepts of Systems Engineering were applied to embedded system design. The objective was to apply the Systems Engineering methodology to the design of a particular embedded system. A Video Surveillancesystem was chosen as the particular embedded system. Systems Engineering concepts provide the foundation for an optimized design process and for the coordination between system modules. The functionality of the Video Surveillance system was achieved through the partitioning of the overall system functionality into three separate modules. The three modules were Image Capture, Image Processing and Image Transmission. The methodology employed resulted in a system that was flexible and portable. The three modules were designed using their own set of specifications and with completely defined linking interfaces. Following a concrete set of specifications resulted in a system, which can be modified at any later stage without the necessity of changing the whole architecture. The Video Surveillance system fulfilled the overall system requirements as well as those imposed by the subsystems. The partitioning of functionality resulted in ease of implementation and better upgradeability. Design based on Systems Engineering concepts provides for ease of integration. In addition, for modules that follow the same protocol, the existence of well defined interfaces enables connectivity to a variety of external units.
502
Thesis (M.S.)--University of South Florida, 2007.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 49 pages.
590
Advisor: Wilfrido Moreno, Ph.D.
653
FPGA.
Cyclone II.
CMOS sensor.
VHDL.
Video surveillance.
690
Dissertations, Academic
z USF
x Electrical Engineering
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.2282



PAGE 1

System Approach to Embedded System Design by Vikram Prabhakar Mehendale A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Electrical Engineering Department of Electrical Engineering College of Engineering University of South Florida Major Professor: Wilfrido Moreno, Ph.D. James Leffew, Ph.D. Paris Wiley, Ph.D. Date of Approval: April 2, 2007 Keywords: FPGA, Cyclone II, CMOS sensor, VHDL, Vide o surveillance Copyright 2007, Vikram Prabhakar Mehendale

PAGE 2

DEDICATION I dedicate this thesis to my parents.

PAGE 3

ACKNOWLEDGEMENTS I offer my deepest and most sincere “thank you” to my major professor, Dr. Wilfrido Moreno, for his encouragement and valuable guidance during this research. I also extend my gratitude to Dr. James Leffew and Dr Paris Wiley for agreeing to serve on my supervisory committee. I am grateful to Mr. Ronnie Leighty who helped me throughout this research. I would also like to tha nk my friends and colleagues for their support during this research experience.

PAGE 4

i TABLE OF CONTENTS LIST OF FIGURES.................................... ................................................... ....................iii ABSTRACT........................................... ................................................... .........................iv CHAPTER 1: INTRODUCTION AND MOTIVATION............. .......................................1 1.1 Introduction................................... ................................................... ..................1 1.2 The System of Systems Approach................. ................................................... .2 1.3 Design Methodology............................. ................................................... ..........2 1.4 Top Down Design................................ ................................................... ...........3 1.5 Embedded System Design......................... ................................................... .....6 1.6 Programmable Logic............................. ................................................... ..........7 1.7 Embedded Processors in Programmable Logic...... ...........................................7 1.8 Overview of Development Kits Used.............. ..................................................8 1.8.1 Altera DE2 Board............................ ................................................... .........8 1.8.2 Xilinx XUP V2PRO Board...................... ................................................... .9 1.9 Motivation..................................... ................................................... ................10 CHAPTER 2: BACKGROUND.............................. ................................................... .......11 2.1 Related Work................................... ................................................... .............11 2.2 Analysis of Requirements....................... ................................................... ......13 2.3 Altera Cyclone II Device....................... ................................................... .......19

PAGE 5

ii 2.4 Kodak KAC-9630 CMOS Image Sensor............... ..........................................20 2.5 The IrDA Interface............................. ................................................... ...........20 2.6 Altera DE2 Development and Education Board..... .........................................21 2.7 Design Software and Support.................... ................................................... ...22 CHAPTER 3: IMPLEMENTATION.......................... ................................................... ...23 3.1 Image Capture Module........................... ................................................... ......23 3.2 Image Processing Module........................ ................................................... .....25 3.2.1 System Initialization....................... ................................................... ........26 3.2.2 Pixel Comparison............................ ................................................... ........27 3.2.3 Error Detection............................. ................................................... ...........27 3.3 IrDA Transmission.............................. ................................................... ..........29 CHAPTER 4: TESTING AND VERIFICATION................ .............................................30 4.1 FPGA Verification.............................. ................................................... ..........30 4.2 VHDL Test Bench................................ ................................................... ........32 CHAPTER 5: CONCLUSION AND FUTURE WORK.............. .....................................35 5.1 Conclusion..................................... ................................................... ...............35 5.2 Future Work.................................... ................................................... ..............36 REFERENCES......................................... ................................................... ......................37 APPENDICES......................................... ................................................... .......................39 Appendix A: VHDL Source Code...................... ................................................... 40 Appendix B: VHDL Test Bench....................... ................................................... ..45

PAGE 6

iii LIST OF FIGURES Figure 1: Top Down Design Flow.................... ................................................... .............4 Figure 2: Embedded System......................... ................................................... .................6 Figure 3: Altera DE2 Board........................ ................................................... ..................8 Figure 4: Xilinx XUP V2PRO Board.................. ................................................... ..........9 Figure 5: Typical Video Surveillance System....... ................................................... ......15 Figure 6: Altera Cyclone II FPGA.................. ................................................... ............19 Figure 7: Proposed Interface...................... ................................................... .................24 Figure 8: Image Data Port......................... ................................................... ..................24 Figure 9: Image Processing Module................. ................................................... ...........25 Figure 10: Port Map for the Image Processing Module ..................................................2 6 Figure 11: FPGA Design Flow........................ ................................................... ............30 Figure 12: VHDL Test Bench......................... ................................................... .............32

PAGE 7

iv SYSTEM APPROACH TO EMBEDDED SYSTEM DESIGN Vikram Prabhakar Mehendale ABSTRACT During this research, the concepts of Systems Engi neering were applied to embedded system design. The objective was to apply the Systems Engineering methodology to the design of a particular embedded system. A Video Surveillance system was chosen as the particular embedded system Systems Engineering concepts provide the foundation for an optimized design proc ess and for the coordination between system modules. The functionality of the Video Sur veillance system was achieved through the partitioning of the overall system func tionality into three separate modules. The three modules were Image Capture, Image Process ing and Image Transmission. The methodology employed resulted in a system that was flexible and portable. The three modules were designed using their own se t of specifications and with completely defined linking interfaces. Following a concrete set of specifications resulted in a system, which can be modified at any later sta ge without the necessity of changing the whole architecture. The Video Surveillance sys tem fulfilled the overall system requirements as well as those imposed by the subsys tems. The partitioning of functionality resulted in ease of implementation an d better upgradeability. Design based on Systems Engineering concepts provides for ease o f integration. In addition, for

PAGE 8

v modules that follow the same protocol, the existenc e of well defined interfaces enables connectivity to a variety of external units.

PAGE 9

1 CHAPTER 1: INTRODUCTION AND MOTIVATION 1.1 Introduction The motivation behind this research was the desire to facilitate the embedded system design process by applying the concepts of S ystems Engineering. The embedded systems consist of many modules, which are comprise d of software components, hardware components and interfaces. All these modu les can be independently modeled as complex systems. In order to achieve a correct implementation of a project all of the independent designs must work in synergy. Therefor e, application of system design principles to the design of embedded system can dra matically streamline the design work and avoid future problems involved with integrating the modules to constitute a larger system. The embedded system is a system in which the proces sing unit is actually embedded between its peripherals and the system is designed to perform some predefined tasks. Being dedicated to certain tasks, the embed ded system provides a very efficient solution compared to their general purpose counterp arts. The embedded systems are currently used in many applications from automobile s to home appliances. Development of powerful microcontrollers and programmable logic has made the embedded system a very useful solution. Due to continuous developmen t of processors and peripherals the

PAGE 10

2 use of embedded systems is rising exponentially. U sing embedded processors and programmable logic makes it simpler to update the s ystem without changing much of the hardware. The use of EDA (Electronic Design Automa tion) tools make it simpler to debug the system and provide patches for future pro blems through the use of advanced synthesis and simulation tools. 1.2 The System of Systems Approach When confronted with designing complex systems it i s beneficial to approach the design via an architecture, which is structured as a system of systems. In this approach the designers identify the system requirements for subsystems, which are based on overall system requirements. The subsystems are de signed independently and then interfaced to achieve the completed system architec ture. This approach simplifies the procedures related to testing, debugging and integr ation of the subsystems, which are required to insure proper design of the whole syste m. Approaching a complex system as a system of systems helps to divide the functionality and verify the working of the subsyst ems independently. This design methodology also helps to enhance the interoperabil ity and portability of the design. 1.3 Design Methodology A good design methodology can help the system desig n process in many ways. It can help to verify the system for functionality and for errors. It can help the design team to coordinate the design effort. The design proces s should provide a time line for the designers and the deliverables, which are due at di fferent times. The design methodology

PAGE 11

3 involves the global approach of the chief designers to the system design. Since the projects usually involve many teams working on the project at the same time, this methodology can help with coordination, which can b e very fruitful for achieving the design goals. The Systems Engineering methodology helps realize a solution, which achieves all the goals of the system including manu facturing cost, performance and power consumption. 1.4 Top Down Design The Top Down Design approach involves design proced ures, which are initiated from requirements for system integration. This app roach involves arriving at the right solution after considering all the possible alterna tives. Every so often, the design effort may focus on trying to fit the solution within the available resources. This approach, which may provide short term gains, may lead to com plications while fulfilling future needs and not necessarily arriving at an optimum im plementation. The problem may have a large number of possible solutions and selec tions for the optimum solution based on the target application can provide a number of a dvantages. This can be considered as the selection of an optimum solution from a larger design space. The idea of fitting the problem into an already ava ilable solution may sound lucrative to the designer due the ease of implement ation. There is always a possibility of some other better way to implement the project. Th e Top Down approach looks at the requirements in an objective way. In this approach the designer will be faced with a large number of different possible ways and then the opti mum way is selected.

PAGE 12

4 The selection of an optimum solution is a very impo rtant decision since it involves the study of all design categories such as available resources, non-recurring engineering cost, size and power. The selection pr ocess for choosing an optimum solution provides a number of advantages in terms o f lowering the required resources and better functionality of the solution. Figure 1 ill ustrates the investigative flow associated with the Top Down Design process. [11] Figure 1: Top Down Design Flow The requirements are abstract descriptions of the s ystem, which involves functional as well as nonfunctional requirements. The requirements are the customer’s expectations about what the system has to achieve. The requirements may put monetary and timing constraints on the design, which will ha ve to be considered along with the technical specifications. The designers need to in corporate these requirements and realize a system, which can perform the expected ta sks. The requirements have to be

PAGE 13

5 validated throughout the design process including f inal verification at the end product level. The system specifications are more focused on syste m implementation. They offer the designer a role map for the design of the system. The specifications have to be written carefully to ensure that they meet requirem ents. The specifications must be comprehensible and unambiguous so that the designer knows exactly what has to be built. Ambiguous specifications can lead to an incorrect i mplementation, which can defeat the whole purpose of using a system approach. The architecture describes how the functions are im plemented in the system. The architecture defines the structure of the system. The architecture is the framework on which the technical aspects of the system will be i mplemented. The architecture will dictate what resources are used and how the subsyst ems are interfaced. The architecture design also needs to consider the constraints impos ed by the skills of the designers. If the designers are not skilled in some particular techno logy used in the architecture, the cost of providing training to the designers has to be co nsidered. The components are designed with the architecture i n mind. The components can be hardware components as well as software componen ts. The specifications for the component are developed with the architecture speci fications in mind. The components perform specific system tasks and together they per form the architecture tasks. The integration of the system is where the benefits of using a system approach are particularly visible. If the specifications and ar chitecture are designed correctly and tested, then this stage can be very simple. The wh ole system will be put together and the working system realized. The Systems Engineering c oncepts help to keep track of

PAGE 14

6 specifications for the separate modules and allow t he modules to be tested separately, which simplifies the system integration process. 1.5 Embedded System Design The embedded system consists of an embedded process or, hardware peripherals such as RAM, ROM, LCD, keypad, some communication c hannel for the processor to communicate with other devices such as RS232, Ether net, USB, wireless transceiver and a software code which runs on the processor to cont rol all the peripherals. The various elements associated with the embedded system are pr esented in Figure 2. Hardware Peripherals Software Code to execute on the processor Communication Devices Embedded System Design Figure 2: Embedded System Embedded system design involves division of system tasks between hardware and software. Some functions are better implemented in hardware whereas some functions

PAGE 15

7 give better performance if implemented in software. Therefore, the designer has to carefully divide the system tasks between these two components in order to get optimum system performance and meet all system requirements The selection of communication channels and the processor has to be accomplished i n accordance with system requirements. 1.6 Programmable Logic Programmable logic is hardware, which can be progra mmed to perform the required task using a Hardware Description Language (HDL). This programmability makes it a very good choice since it becomes easy t o enhance the functionality or provide a patch for the system. The programmable logic mak es it possible to accept last minute design changes, if necessary. The FPGA (Field Prog rammable Gate Arrays) provide an ideal platform of implementing digital logic circui ts. With the advent of FPGA technology, the operating speed of these devices ha s increased dramatically. This makes it possible for the designer to infer high speed lo gic on these devices. The use of Intellectual Property (IP) cores can relieve the de signer of some tasks and can significantly speed up the process. The FPGA devic es from Xilinx Inc. and Altera Inc. were employed for this research. 1.7 Embedded Processors in Programmable Logic The FPGA devices, which were used as the basic bloc ks for this research, can implement both hard core processors and the soft co re processors. The Intellectual Property Core is provided by various vendors and th at IP core can be used to instantiate a

PAGE 16

8 processor in the embedded system. The designer can also design a processor core, capable of executing the required instruction set, using the Hardware Description Languages. The hard core processor is implemented on the FPGA chip. The advantage of using such a hard core processor is enhanced perfor mance. Xilinx Inc. provides a hard core PowerPC core with some of its FPGA devices. With a soft core processor, the user can modify the HDL code in order to achieve specific processor requirements. The soft core opt ion provides the user more choices. The soft core processor offered by Altera Inc. is t he NIOS II. 1.8 Overview of Development Kits Used 1.8.1 Altera DE2 Board The DE2 (Development and Education 2) board, provid ed by Altera features the Cyclone II-2C35-FPGA in a 672 pins package. The DE 2 board is pictured in Figure 3. Figure 3: Altera DE2 Board Courtesy: Altera Corporation Inc.

PAGE 17

9 The DE2 board provides a ready to use development p latform with many peripherals already connected to the FPGA pins. Th e NIOS II embedded processor on this FPGA can be used to develop embedded applicati ons. The Quartus II web edition design software can be used to program this board. Altera also provides a NIOS II IDE to create embedded applications. This board also h as SDRAM, SRAM and Flash memory, which can be used by the NIOS II core. 1.8.2 Xilinx XUP V2PRO Board The Xilinx XUP V2PRO board features a Virtex-2 Pro XC2VP30 FPGA with 30,816 Logic Cells, 136 18-bit multipliers, 2,448Kb of block RAM, and two PowerPC Processors. Figure 4: Xilinx XUP V2PRO Board Courtesy: Xilinx Inc.

PAGE 18

10 The Xilinx ISE foundation edition can be used to pr ogram this board. The Embedded Development Kit (EDK) must be used to deve lop embedded applications using the PowerPC cores. The microblaze soft core processor, provided by Xilinx, can also be used to develop embedded applications with this board. 1.9 Motivation Use of embedded systems in all fields related to el ectrical design is rising exponentially. The design process is getting more and more complex with development of processors and peripherals. Therefore, to strea mline the design process and to make it easy for keeping the designs portable and interoper able, the System Approach can prove useful. The thesis will demonstrate the applicatio n of the System Approach concepts to an example system design.

PAGE 19

11 CHAPTER 2: BACKGROUND This Chapter provides a short description of previo us work on Systems Engineering. In addition, it describes how those c oncepts are applied to the design of embedded systems and how they were applied to the d esign of the example system. 2.1 Related Work The design of complex systems is always a difficult task since the designers must be sure, while deciding on the specifications, that the design of the proposed system is feasible. If it becomes impossible for some of the subsystems to follow the specifications then the specification set for the whole system of systems has to be changed. While spelling out the specifications, it is very importa nt for the designers to be very certain about what is expected of the subsystem designers. As a guideline for this procedure, Dale Scot Caffall et al. proposed “a system-of-syst ems construct in which we consider a system-of-systems infrastructure that is composed o f controlling software for the systemof systems, an information transport network, and c ontract interfaces” [1]. Dale Scot Caffall et al. also demonstrated application of for mal methods to system of systems development for verification purpose in their publi cation [2]. The application of the system approach to managemen t was studied by Group Captain W. W. Robinson in his publication ‘A system approach to management’ [3]. Dr.

PAGE 20

12 John Boardman et al. presented distinguishing chara cteristics that can help in identification and implementation of System of Syst ems (SoS) [5]. The development of a Video Surveillance system usin g programmable logic design has been undertaken by some researchers sinc e the programmable logic devices provide an optimum platform to develop a prototype system and implement different architectures on that platform. Suh Ho Lee et al. have implemented a motion detection system using a CMOS image sensor and ARM processor [4]. The concept of a Video Surveillance system involves more work than simple motion detection. The advanced Video Surveillance systems try to detect the object, localize the object and classify it so as to add mo re functionality to the Video Surveillance system. M.H. Sedky et al. proposed th e classification of such smart Video Surveillance systems [6]. The use of FPGA devices in the design of surveillan ce systems is well documented. Fei Wang et al. implemented a FPGA bas ed driver drowsiness detection system in order to detect if the driver has closed his eyes [7]. Patrick Dickinson et al. have designed an infant surveillance system in orde r to detect sleeping infants using a subtraction algorithm implemented on a Xilinx Virte x II PRO FPGA [8]. Jason Schlessman et al. implemented a tracking syst em based on optical flow using a Xilinx Virtex II PRO FPGA [9]. Michael McE rlean proposed a hierarchical motion estimator for embedded object tracking using a FPGA [10]. Another motivation for this research work was the c oncepts put forward in the book “Power to the Edge”, which explains about empo wering the nodes for better performance. This research work aimed to implement those concepts in the design of

PAGE 21

13 embedded systems. This research attempted to apply those concepts for the design of embedded system and one of the goals of this resear ch was the development of intelligent nodes, which could work independently in case of sy stem failure. The book “Computers as Components” by Wayne Wolf ex plains the concepts related to embedded system design. The book has be en very helpful in the analysis and planning of this research work [11]. 2.2 Analysis of Requirements During the study of any system, the requirements ar e the most important part in the design. The design of the system has to fulfil l all the requirements. In order to streamline the development of subsystems the specif ications of the subsystems have to be clearly defined. Any mistake in defining the speci fications can lead to future problems. Validation of these concepts was given the highest priority during the design of the Video Surveillance system. The system design h as to fulfill the system requirements. Therefore, the planning stage is the most important stage in the design cycle since it considers all the available resources and then assi gns specific tasks to subsystems by delineating, very specifically, the specifications. The technical capabilities and available resources were also considered during this stage. The specifications were formulated by considering all the possible solutions and then ide ntifying the optimal solution for the current problem. The architecture of the Video Surveillance system d epends on the requirements. If the proposed system is to be used in some wareho use, then it needs to have some memory where the system can store some images. The cameras in shops need to have a

PAGE 22

14 larger memory so that they can store more data. In these types of systems it is more important to store the data rather than to process and detect some object. In some security applications it may be very important to d etect the incoming object to enable quick response from the user. Thus, the architectu re of the Video Surveillance system totally depends on the requirements. The requireme nts guide the design team in assigning priorities and level of effort to the var ious components and capabilities of the design such as deciding whether it is more importan t to spend more design effort on data compression for storage or data processing capabili ty. Therefore, it is very important to write unambiguous specifications for the system bef ore a team of engineers begins designing. A complete specification derived from t he set of requirements is very important for the considerations involved in optimi zing the design effort. In the event that specifications are not completely defined and are changed during the design cycle, the design effort can result in wasting valuable ma n hours and other resources. The example system included in this thesis was desi gned for applications where detection and capture of motion was considered more important to the user than being able to look at the captured snapshot and take imme diate necessary action. The system was designed for those applications where the user was not expecting any particular type of object so there was not any need to run an objec t detection algorithm. A diagram of a typical Video Surveillance system is presented in F igure 5.

PAGE 23

15 Image Capture Memory Processing Unit Figure 5: Typical Video Surveillance System The criteria involved in the selection of various c omponents of a Video Surveillance system are explained below. The worki ng of the system will be explained later. The following paragraph explains the compon ent selection process as a function of the requirements. The effort and resources allocat ed to designing the different components are decided by going over the system req uirements. The digital camera represents any image sensor device. The device can be chosen to be a high speed device or a high resolution device. The primary criterion for selection of the image sensor device is the operating speed of the image sensor. The operating speed is specified in terms of frames per second. The resolution is also important and is specified by rows multiplied by columns. If the user needs to detect smaller objects, it may be necessary to select a sensor with more resolution. There is als o another constraint on the image sensor selection, which derives from the operating frequen cy of the clock. For example, if an 8 bit parallel interface for a CMOS image sensor is s elected and the operating frequency is

PAGE 24

16 10 MHz, then the maximum data rate, which can be re ad by the system, will be 80 Mbps. A color image sensor may or may not be specified an d is most often a function of the financial constraints. The example system was designed with two sensors. A CMOS grayscale image sensor was chosen, which possessed a resolution of 126 X 98 and a speed of 580 frames per second. In addition, a CGA Red Green Blue sens or with a resolution of 640 X 480 and a speed of 60 frames per second was selected. The system was capable of employing the CMOS sensor if higher frames per second were sp ecified or the VGA sensor if the requirements demanded better resolution. The Image Capture block is responsible for setting up the sensor, extracting the frames and storing them in separate memories or arr ays. The Image Capture block has to take input data from the sensor as per the requirem ents. This was specified because sometimes it may not be necessary to read the data at the maximum rate. If the system is only supposed to store the snapshots at regular int ervals it can work reasonably well even with a data rate of one frame per second. These ta sks are performed by the Image Capture block. The Image Capture block in the exam ple system works at a clock rate of 50 MHz and reads data from the sensor only when req uired. The data read from the sensor is processed by the logic implemented on the Altera Cyclone II device. The system is not required to work as very high speed. Therefore, the Image Processing block reads a new frame only when the processing un it can accept another frame and a new frame is available from the sensor. The other frames are simply discarded. The memory block has to be designed carefully since memories are costly in terms of operating time. Memories can slow down th e whole system if not designed

PAGE 25

17 carefully. The design of the memory block involved an estimate of the required memory. If the system was required to keep many past snapsh ots in the memory, this block would have larger capacity. The system, which is used in some departmental stores, may need to have a lot of memory since it will need to keep a record of all the visitors for the stipulated time. The memory block in the example system was very sma ll since there was no need to keep track of past events. Moreover, since the system transmits the results to another entity the memory block could also be used there, i f required. The memory used in the system was required to store only two frames. One reference frame and an error frame. The working of the example system will be explained in Chapter 3. The processing unit can vary in complexity from a s imple comparator, in case of motion detector, to complex object detection or fac e recognition algorithm processors. The processing unit also decides the type of proces sor, which has to be used. If the processing unit requirements are not computationall y expensive, then a simple microcontroller can be used. However, for algorith ms such as face recognition, a very powerful processor could be required. The processi ng unit can be designed in software as well as in the hardware. If the system places more emphasis on record keepin g, the processing unit will be the least worked on part of the system. If the sys tem is supposed to be used in an object detection smart surveillance system then the proces sing unit will be the most resource consuming part of the system. In any circumstance the processing unit will decide the functionality of the system.

PAGE 26

18 The processing unit in the example system was imple mented in an Altera FPGA device. The processing unit consisted of comparato rs, arithmetic logic and buffer memory. The processing unit performs a pixel by pi xel comparison over a period of 6 frames. Depending on the results of the comparison the processing unit detects the presence of another object. In case of detection o f an unwanted object the snapshot, with the object in the frame, is transmitted over the wi reless network to another entity where the frame can be further analyzed to localize, clas sify and detect the object. The system thus consists of an Altera Cyclone II FP GA based development board, a CMOS image sensor, a VGA sensor and a wireless tr ansmission channel. The system was intended to be a prototype for testing various object detection algorithms. The different modules were designed separately. Theref ore, any module could be used in another system with small modification in order to meet requirements, if required. The example system follows the philosophy of power to the edge [12]. This involves empowering the nodes of the system to atta in better functionality. The idea is the algorithms can be implemented at the nodes of t he system, which reduces the data that has to be transmitted over the network. This can help in reducing the network traffic and it can also be helpful in making the system mor e robust. Since the intelligence is distributed between various nodes, even is one node stops working the system can still function with the remaining nodes. This philosophy can greatly reduce system failures and improve performance. The only disadvantage may be that more processing units for use at nodes may be required, which will add up to the overall system cost.

PAGE 27

19 2.3 Altera Cyclone II Device The FPGA development board used for the design of t he system was based on the Cyclone II FPGA from Altera. The FPGA device offer s many features which can be used to implement advanced algorithms. The Cyclone II device was selected because it makes this system a very good platform for the impl ementation of prototypes. The Altera Cyclone II FPGA is pictured in Figure 6. Figure 6: Altera Cyclone II FPGA The Cyclone II device family provides from 4,608 to 68,416 logic elements (LE). The EP2C35F672C6 device contains 33,216 logic eleme nts. The 105 M4K RAM blocks contain 483,840 total RAM bits, 35 embedded multipl iers and 4 PLLs. This is sufficient for the current requirements since the simple algor ithm employed used only 30% of these resources. This device provides a platform capable of implementing an advanced algorithm. This device provides the necessary resources to des ign a smart Video Surveillance system processing unit. The Cyclone II device also comes with a soft core, NIOS II, embedded processor, which can be used for running software code. The p rocessor provides the designer the freedom to choose whether the implementation will b e hardware based or software based.

PAGE 28

20 The Cyclone II device comprises the most important part of the Video Surveillance System. 2.4 Kodak KAC-9630 CMOS Image Sensor The KAC-9630 provides a very high speed (580 frames per second) sensor. The disadvantage of this sensor is its low resolution. The resolution the sensor is 126 X 98 pixels, which does not allow for processing any det ails from the captured snapshot. The KAC-9630 sensor also possesses a parallel interface which was easy to use for interfacing with the FPGA device. The KAC-9630 sen sor can be very useful for the capture of high speed objects due to its very high frame rate. The Video Surveillance System employs this sensor t o implement simple pixel by pixel processing algorithms. The KAC-9630 sensor i s present on the designed printed circuit board and will provide the designer with th e option to select one of the two sensors in future research work. 2.5 The IrDA Interface The Video Surveillance System used the IrDA interfa ce to transmit the captured snapshot to another entity. The IrDA interface was implemented on the Altera DE2 board, which was connected to the Cyclone II device The IrDA system on the DE2 board was implemented us ing the HDSL 3201 transceiver. The interface is capable of data rate s up to 115.2 Kbps. The HDSL 3201 was selected due to availability constraints.

PAGE 29

21 The system checks the incoming frames for the prese nce of any unwanted objects. Since the system is not intended to look for any pa rticular object, it will check for dissimilarities between consecutive frames. If the system discovers that any frame differs significantly from the previous frame, the frame wi th dissimilarities will be transmitted over the IrDA to another entity where object detect ion algorithms are implemented. 2.6 Altera DE2 Development and Education Board The Altera DE2 board, which is pictured in Figure 3 of Chapter 1, provides an excellent platform for a broad range of academic re search, industrial research and prototyping applications. The board is based on th e Cyclone II device and comes with a USB blaster interface for download of a program int o an FPGA. The DE2 board also provides many other features such as an Ethernet 10 0/10 Mbps, RS232 and PS/2 interfaces. In addition, the DE2 board contains an IrDA, which was used during this research. The DE2 board also provides LED and DIP switches, which can be very helpful for testing small subset modules of the sys tem. The NIOS II processor provides an excellent soft core processor, which can be used to interface different peripherals and implement different algorithms. The DE2 board also comes with on board SDRAM, SRAM, Flash and a SD card connector. These features can be utilized if the d esigner runs out of the on chip memory resources on the Cyclone II device. The DE2 board provides a very good platform for both the design and for the testing stage of the pr ototype.

PAGE 30

22 2.7 Design Software and Support The support material provided by Altera Inc. is ver y helpful in any project. The University program with Altera Inc. provides Intell ectual Property (IP) cores for use with the DE2 board. Altera Inc. also provided, for down load from their website, the Quartus II synthesis tool, required to program the FPGA, an d the Modelsim software, required for simulation of the design. These resources along wi th the hardware make this platform suitable for use in research work.

PAGE 31

23 CHAPTER 3: IMPLEMENTATION The Video Surveillance system consists of three dif ferent modules from the implementation point of view. The modules are name d Image Capture, Image Processing and Image Transmission. The source code was writte n in VHDL and the Modelsim simulator was used for functional and timing simula tion. The Quartus II web edition software was used as the synthesis tool. The VHDL code was tested with a VHDL test bench for functional verification. 3.1 Image Capture Module The Image Capture module was implemented partly on the dedicated Printed Circuit Board and partly on the Cyclone II FPGA dev ice. The function of this module is to set up the sensor and to provide the processing module with separate frames of data. This module also provides the sensor with the appro priate clock signal. The data is sent to the processing module with the necessary handsha king signals. Figure 7 presents a block diagram of the interface between the Image Ca pture module and the Image Processing module.

PAGE 32

24 Image Capture Module Image Processing and Transmission unit 8 bit Data pins Clock new_frame Figure 7: Proposed Interface The Image Capture module provides the specified int erface to the Image Processing block. The KAC-9630 is equipped with a parallel digital image data port. The details of the data port are presented in Figur e 8. Figure 8: Image Data Port The sensor output is presented at the input of an 8 bit A/D converter and the output of the A/D converter, along with the synchro nization signals, is placed on the image data bus. This data is synchronized to the p ositive edge of the clock. The hsync ( horizontal synchronization) signal is used for sy nchronization of ROW data. The vsync (vertical synchronization) signal is used for synch ronization of frame data in video mode and is also used as input in the snapshot mode. Th e hsync and vsync signals are used by all the other modules. The function of the Image Capture block is to read the data from the sensor and to transmit it in required format to the Image Process ing block. The clock, which is used by this block, is provided by the Image Processing blo ck in order to maintain

PAGE 33

25 synchronization. This arrangement presents no prob lems as long as both blocks are implemented on the same device. If required, the I mage Capture block can be made to work with different clocks. The vsync signal is provided by the module as the n ew frame signal. The same clock was used for both Image Capture and Image Pro cessing modules. The data bus at the output of the digital image bus is the output d ata port. 3.2 Image Processing Module A block diagram of the Image Processing module is presented in Figure 9. LOAD REFERENCE FRAME PIXEL COMPARISON ERROR DETECTION IrDA TRANSMISSION Figure 9: Image Processing Module The Image Processing module only accepts data from the Image Capture module when it is ready for processing of the next frame. The function of the Image Processing module is to compare the incoming frame with the ex isting reference and check for errors. Whenever a determination of error occurs, the Image Processing module transmits the suspected frame over the wireless net work. This block performs the processing on a pixel by pixel basis. A port map, of the input and output signals for the Image Processing module, is presented in Figure 10.

PAGE 34

26 IMAGE PROCESSING CLOCK_50 RESET_N CLOCK_OUT DATA_IN FRAME_READY REFRESH_FRAME _MEMORY ERROR_IN_CURRE NT_FRAME Figure 10: Port Map for the Image Processing Modul e 3.2.1 System Initialization The Image Processing module has two memories, one f or storing the frame under analysis and the other for store of the reference f rame. Initially, the system loads the first incoming frame in the reference frame memory so tha t further analysis can be performed. The reference memory can be refreshed anytime by as serting a load reference memory signal. The system uses two counters for keeping track of r ows and columns. The data coming in from the Image Capture module is stored i n the reference frame memory. The reference frame memory was instantiated using the m emory bits available in the Cyclone II device. Once the reference frame is loaded an i nternal signal is asserted, which paves the way for further processing of the incoming fram es.

PAGE 35

27 3.2.2 Pixel Comparison Once the reference frame is loaded, in the referenc e memory, future data can be compared to the data in the reference frame. The r ow counters and column counters were used to keep the track of current frame. Once the system receives a byte of data on its input lines, the corresponding pixel value in the r eference frame memory is fetched by providing a correct address to the reference frame memory address bus and the two pixels are compared. The comparison is not bit by bit. T he required sensitivity can be defined by the designer. If the incoming pixel value is fo und to be out of the tolerance range of the system, an error counter is incremented. The e rror counter keeps track of the number of pixels in every frame, which are out of the tole rance zone. 3.2.3 Error Detection The error detection counter was used to decide the threshold value. If the number of pixels, which are different than the previous fr ame, is more than the threshold value the system flags the particular frame as a suspecte d frame and transmits it for more processing. If the threshold value is set to a ver y small value the system may flag some frames as suspected due to noise and changes in ill umination. Therefore, deciding the threshold is very important for correct operation. The threshold value for error detection decides the minimum object size that can be detecte d. If a frame is marked as suspected, that frame is ta ken out of frame memory and sent over the IrDA channel. If a frame is not susp ected, then the frame memory is overwritten. Since the frame memory has to be acce ssed frequently it was implemented using the on chip resources. The frame memory is w ritten to during almost every clock

PAGE 36

28 cycle and the reference frame memory is read during every clock cycle. Therefore, the memory access times play a major role in deciding t he combinational delay of the module. It has to be noted that a smaller object may appear bigger in frame if it is close to the camera. Therefore, the minimum size is the siz e which is detected anywhere in the range of the Video Surveillance system. The smalle r objects may be detected if they are too close to the camera and the larger objects may go unnoticed if they are far away from the camera. Thus, while providing the specificatio ns for the Video Surveillance system it is very important to know the altitude of the camer a and the area which is required to be covered. Furthermore, an object may go unnoticed i f it is not captured in any frame. Therefore, the frame rate places an upper limit on the speed of motion of the object being detected. This speed can not be specified without considering the operating condition. An object moving near the camera will appear to be moving at a higher speed than an object moving at the same speed but further away fr om the camera. The threshold value was kept programmable so that a ny changes which may be required during operation may be performed. The sy stem will perform better if it is fine tuned by considering the operating conditions. The sensitivity can be increased by changing the co mparison operation in the processing unit. If a bit by bit comparison is per formed, the system will be more sensitive. However, bit by bit comparison may gene rate many more false alarms. Therefore, the comparison operation and the error d etection threshold have to be decided by careful review of the requirements. In addition the nature of the objects to be detected and the environment where the system is go ing to be used must be considered

PAGE 37

29 carefully and their impact on the requirements take n into account. If the system is supposed to be used inside a warehouse then the cha nges due to the daylight will be negligible since the system will always work under artificial light. However, a system which is to be deployed in an outdoor environment w ill have, as a primary consideration, factors associated with the changes in illumination In order to be able to adapt to changing conditions the refresh frame memory signal was provided as an input port. The referenc e frame memory can be refreshed every few minutes when the system is in an outdoor environment in order to take care of the illumination issues. The refresh frame memory signal has to be efficiently controlled to obtain optimum performance of the system under d ifferent kinds of requirements. 3.3 IrDA Transmission If any frame is found to be suspect it is sent over the IrDA channel. The IrDA channel has adata rate of 115 Kbps. The IrDA trans mission module is activated by the error signal from the Image Processing module. The IrDA reads the frame memory and transmits the data in the event of an error.

PAGE 38

30 CHAPTER 4: TESTING AND VERIFICATION The functional verification was the most important stage in the design flow since this stage is where logical errors can be identifie d and rectified. The functional verification involved testing the design to see if it operated the way it was supposed to operate. The verification of the VHDL code was per formed by a VHDL test bench. 4.1 FPGA Verification A block diagram of the design flow for testing dur ing FPGA verification is presented in Figure 11. Figure 11: FPGA Design Flow Courtesy: Xilinx Inc. The design entry stage included generating the VHDL code for the design. The behavioral simulation did not consider the combinat ional delays and the routing delays.

PAGE 39

31 The behavioral simulation was intended only for det ecting logical inaccuracies. Once the design was cleared by the behavioral simulation it was synthesized by the synthesis tool. The synthesis tool translated the HDL (Hardware Des cription Language), description to a logic circuit. The synthesis tools used the logic components from its component library. Therefore, once the synthesis process was complete the design had to be simulated again to check whether it worked at the desired frequency After successful synthesis and behavioral simulatio n the design moved to the implementation stage. During implementation the co mbinational delays, associated with the logic components, were considered and the desig n was, once again, simulated to check for any timing violations. If the design fai led to follow the timing constraints it could be fixed, if required, by editing the VHDL co de. During this stage the synthesis tool can also assume the routing delay in order to provide a designer a rough estimate of the delay, which will be present when the device is actually downloaded to the FPGA board. Next, the EDA tool routes the design on the target FPGA device and provides the designer with fairly exact numbers with respect to routing delays and combinational delays. At this stage the timing simulation is per formed and if the design is cleared by this simulation it can be downloaded to the FPGA. The EDA tools for synthesis and simulation provide the user with very accurate delay estimates and functionality checks, which avo ids future problems related to timing violations and design failures. The design was tes ted once again, with the aid of a Logic Analyzer, after it was downloaded to the FPGA board

PAGE 40

32 In the design flow, verification is the most import ant block and at times it takes more resources than the design team. The verificat ion is extremely important for large scale orders since it may prove to be very costly t o fix any bugs in the design after the solutions are shipped. During this research, all verification was performe d by the Modelsim tool and logic analyzer. The VHDL test bench was used to pe rform behavioral simulation, functional simulation and timing simulation. Once the design passed these tests it was ready to be downloaded to the FPGA board. 4.2 VHDL Test Bench The VHDL test bench concept is presented figurativ ely in Figure 12. TEST BENCH ENTITY UNDER TEST OUTPUT PORTS INPUT PORTS Figure 12: VHDL Test Bench A VHDL test bench is VHDL code, which drives the in put ports with the simulated input signals and checks the output ports for expected data. This procedure can assure the designer of correct functionality so the design can be taken to next stage.

PAGE 41

33 Simulator software can be used for functional verif ication. For this research, the Modelsim simulator from Mentor Graphics Inc. was em ployed. The VHDL test bench simulated the input signals and checked for the output signals to determine functionality of the system. During this research, separate test benches were developed for different modules. The intention was to ensure correct functionality of every module before assembly of th e whole system. To verify the Image Capture block, the test bench s imulated the Image Processing block. The VHDL test bench code was used to drive the input signals such as clock and check for the new frame and data signals. The fram es received from the Image Capture block were saved in raw format and checked. Since the generation of the image depended on the sensor, this module had be tested i n hardware. It was not possible to complete the functional verification of this module without the actual hardware board. The test bench for the Image Processing block simul ated the signals from the Image Capture module and Image Transmission module. Two files on the computer hard drive were read by the test bench as input data and the output observed. The test bench, for the Image Processing block, involved the use of the VHDL package, TextIO. Different image files on the computer hard drive we re read and provided as inputs to the image processing block. The objective was to check whether the module could identify the differences in two image frames. The advantage of writing a test bench is that the s ame test bench can be used for all simulations. The test benches developed during this research were all used from the behavioral simulation stage to the timing simulatio n stage.

PAGE 42

34 The final system could not be completely tested by a VHDL test bench. Therefore, for testing of the final system, a hardw are testing platform was employed.

PAGE 43

35 CHAPTER 5: CONCLUSION AND FUTURE WORK 5.1 Conclusion The concepts of the system approach were applied to the design of a Video Surveillance system and it yielded results as per t he system requirements. Separate modules were designed to implement the functionalit y of the system. As a result of the application of Systems Engineering concepts, the sy stem is easy to modify and debug. It is very easy to use some module of this system as a subsystem in another design. The overall system requirements were studied and se parate module system specifications were written. The various requireme nts were divided into separate modules. Due to the modular approach a designer ca n target specific characteristics of the system for improvement without modifying the co mplete system. For example, if it is desired and/or required that the resolution of the system be improved, only the Image Capture block would need to be modified. The chang es which would be required in the Image Processing block are simple, since generics w ere used in the VHDL code. Hardware changes would not occur since the same FPG A device can be used to implement any new functionality.

PAGE 44

36 Any effort to increase the intelligence of this sys tem will be directed at the Image Processing block. Since programmable logic was use d, further revisions simply involve reprogramming. The system can be interfaced to any wireless transm ission medium by simply changing the transmission module. 5.2 Future Work Some improvements could be performed in the future to enhance the functionality of the system: The Image Capture module could be implemented compl etely on one PCB board. The Printed Circuit Board has a MSP240 proc essor, which could be used for setting up the sensor and providing the output rather than using the Cyclone II device. The Image Processing block could be modified to red uce the number of false alarms. The system could be made immune to c hanges in illumination. Rather than the current scheme of pi xel by pixel processing, block processing could be implemented, which would help to localize the object and could reduce the amount of data sent ove r the network. An Ethernet interface could be provided for transmi ssion.

PAGE 45

37 REFERENCES 1. Caffall, D.S. and Michael, J.B., “Formal Methods in a System-of-Systems Development”, Systems: Man and Cybernetics, 2005 IE EE International Conference, Volume 2, Page(s): 1856-1863, 10-12 Oct 2005 2. Caffall, D.S. and Michael, J.B., “Architectural Framework for a System-ofSystems”, Systems: Man and Cybernetics, 2005 IEEE I nternational Conference, Volume 2, Page(s): 1876-1881, 10-12 Oct. 2005 3. Robinson, W.W., “A Systems Approach to Managemen t”, Engineering Management Journal, Volume 6, Issue 4, Page(s):172176, Aug. 1996 4. Eisner, H., “A Systems Engineering Approach to A rchitecting a Unified System of Systems”, Systems: Man and Cybernetics, 1994, “Huma ns, Information and Technology”, 1994 IEEE International Conference, Vo lume 1, Page(s): 204-208, Oct. 1994 5. Boardman, J. and Sauser, B., “System of Systems the Meaning of System of Systems Engineering”, 2006 IEEE/SMC International C onference, Page(s): 24-26, 6 April 2006 6. Sedky, M.H., Moniri, M. and Chibelushi, C.C., “C lassification of Smart Video Surveillance Systems for Commercial Applications Ad vanced Video and Signal Based Surveillance”, 2005 IEEE Conference on AVSS, Page(s): 638-643, 15-16 Sept. 2005 7. Fei Wang and Huabiao Qin, “A FPGA Based Driver D rowsiness Detecting System”, Vehicular Electronics and Safety, 2005. IE EE International Conference, Page(s): 358-363, 14-16 Oct. 2005 8. Dickinson, P., Appiah, K., Hunter, A. and Ormsto n, S., “An FPGA-Based Infant Monitoring System”, 2005 IEEE International Confere nce on Field-Programmable Technology, Page(s): 315-316, 11-14 Dec. 2005

PAGE 46

38 9. Schlessman, J., Cheng-Yao Chen, Wolf, W., Ozer, B., Fujino, K. and Itoh, K., “Hardware/Software Co-Design of an FPGA-Based Embed ded Tracking System” 2006 Conference on Computer Vision and Pattern Reco gnition Workshop, Page(s):123-123, 17-22 June 2006 10. McErlean, M., “Hierarchical Motion Estimation f or Embedded Object Tracking”, 2006 IEEE International Symposium on Signal Process ing and Information Technology, Page(s): 797-802, Aug. 2006 11. Wolf, W.H., “Computers as Components”, Principl es of Embedded Computing System Design, Morgan Kaufmann Publishers, San Fran cisco, CA, 2001 12. David S. Alberts and Richard E. Hayes, “Power t o the Edge: Command and Control in the Information Age”, CCRP Publications, 2003

PAGE 47

39 APPENDICES

PAGE 48

40 Appendix A: VHDL Source Code LIBRARY IEEE; USE IEEE.std_logic_1164. ALL ; USE IEEE.std_logic_unsigned. ALL ; USE IEEE.std_logic_arith. ALL ; ENTITY object_detect IS GENERIC ( ROWS: INTEGER:=98; COLUMNS: INTEGER:=126; VECTOR_LENGTH: INTEGER := 13); -the gene ric to pass the number of pixels to be read PORT ( clock_50 : IN std_logic; -system clock from on board source reset_n : IN std_logic; -system reset signal clock_out : OUT std_logic; -output c lock to read data from the frame capture block data_in : IN std_logic_vector(7 DOWNTO 0); -data from the frame read frame_ready : IN std_logic; -the fram e ready signal from the frame capture block refresh_frame_mem : IN std_logic; error_in_current_frame : OUT std_logic; DUMMY_ref_frame_loaded : OUT std_logic; -DUMMY signal for testing data_read_out1 : OUT std_logic_vector(7 DOWNTO 0); new_frame_analysis1 : OUT std_logic; -DUMMY signal for testing -------------------------------------------------------------------------address_frame_mem : IN std_logic_vector(VECTOR_LENGTH DOWNTO 0); data_frame_mem : OUT std_logic_vector(7 DOWNTO 0)); -output signal showing presence of some unwanted objects -in the image END ENTITY object_detect;

PAGE 49

41 Appendix A: (Continued) ARCHITECTURE object_detect_1 OF object_detect IS SIGNAL clock_enable, start_clock_enable, clock_out1, sig1 : std_logic; -the cloc k enable signal for the output clock going to the f rame capture block SIGNAL clock_enable_counter : std_logic_vector(VECTOR_LEN GTH DOWNTO 0); -counter to ensure complete frame is read in SIGNAL rdaddress,wraddress : std_logic_vector(VECTOR_LENG TH DOWNTO 0); -the addr ess to drive address bus of referance frame SIGNAL ref_frame_loaded, load_new_frame, new_frame_analys is : std_logic; -signals for loading a new frame and to specify that new fra me is loaded SIGNAL data_read_out,write_databus_refmem : std_logic_vec tor(7 DOWNTO 0); -the read and write databusses for the DPRAM SIGNAL data_in1, data_in2, data_in3 : std_logic_vector(7 DOWNTO 0); -the regi stered data_in to compensate for the registered out put from the DPMEM SIGNAL error_count : std_logic_vector(VECTOR_LENGTH DOWNTO 0); -the coun ter to count how many pixels are in error BEGIN -object_detect_1 ----------------------------------------------------------------------------DUMMY signals for testing --------------------------------------------------------------------------------------------------------------------------------------------------------DUMMY_ref_frame_loaded <= ref_frame_loaded; data_read_out1 <= data_read_out; new_frame_analysis1 <= new_frame_analysis; ----------------------------------------------------------------------------The clock enable control and reference frame loa ding processes ------------------------------------------------------------------------------purpose: this process controls the clock, which is going as output to the frame --capture block clock_enable_process: PROCESS (clock_50, reset_n) IS BEGIN -process clock_enable_process IF reset_n = '0' THEN -asynchronous reset (active low) clock_enable <= '0'; clock_enable_counter <= ( OTHERS => '0'); ELSIF clock_50'event and clock_50 = '1' THEN -rising clock edge IF frame_ready = '1' THEN clock_enable_counter <= conv_std_logic_vector((ROWS*COLUMNS),VECTOR_LENGTH+ 1); clock_enable <= '1'; END IF ;

PAGE 50

42 Appendix A: (Continued) IF clock_enable_counter/=conv_std_logic_vector(0, VEC TOR_LENGTH+1) THEN clock_enable_counter <= clock_enable_counte r '1'; ELSE clock_enable <= '0'; END IF ; END IF ; END PROCESS clock_enable_process; clock_out1 <= clock_50 WHEN clock_enable = '1' else '0'; clock_out <= clock_out1; -purpose: this process accepts data coming in fro m the frame capture block accept_frame: PROCESS (clock_50, reset_n) IS BEGIN -process accept_frame IF reset_n = '0' THEN -asynchronous reset (active low) load_new_frame <= '0'; wraddress <= ( OTHERS => '0'); ref_frame_loaded <= '0'; ELSIF clock_50'event and clock_50 = '1' THEN -rising clock edge IF (refresh_frame_mem AND frame_ready) = '1' THEN load_new_frame <= '1'; wraddress <= conv_std_logic_vector((ROWS*COLU MNS),VECTOR_LENGTH+1); ref_frame_loaded <= '0'; END IF ; IF load_new_frame = '1' THEN IF wraddress /= "0000000000000" THEN wraddress <= wraddress '1'; write_databus_refmem <= data_in; ELSE ref_frame_loaded <= '1'; load_new_frame <= '0'; END IF ; END IF ; END IF ; END PROCESS accept_frame;

PAGE 51

43 Appendix A: (Continued) -----------------------------------------------------------------------------------------------------------------------------------------------------Component instantiation for the DP ref memory ------------------------------------------------------------------------------------------------------------------------------------------------------Reference_memory_DPRAM : ENTITY work.ref_frame PORT MAP ( clock => clock_50, data => write_databus_refmem, rdaddress => rdaddress, wraddress => wraddress, wren => load_new_frame, q => data_read_out); sig1 <= new_frame_analysis OR load_new_frame; current_frame_mem_DPRAM : ENTITY work.ref_frame PORT MAP ( clock => clock_50, data => data_in, rdaddress => address_frame_mem, wraddress => rdaddress, wren => sig1, q => data_frame_mem); -----------------------------------------------------------------------------------------------------------------------------------------The processing after loading the ref frame -------------------------------------------------------------------------------------------------------------------------------------------purpose: this process takes the input bit stream and compares it with data inside the reference memory block processing_data: PROCESS (clock_50, reset_n) IS BEGIN -process processing_data IF reset_n = '0' THEN -asynchronous reset (active low) rdaddress <= ( OTHERS => '0'); error_count <= ( OTHERS =>'0'); ELSIF clock_50'event AND clock_50 = '1' THEN -rising clock edge IF ref_frame_loaded = '1' THEN IF frame_ready = '1' THEN rdaddress <= conv_std_logic_vector((ROWS *COLUMNS), VECTOR_LENGTH + 1); new_frame_analysis <= '1'; END IF ; IF rdaddress /= conv_std_logic_vector(0, VECTOR_LENGTH + 1) THEN

PAGE 52

44 Appendix A: (Continued) rdaddress <= rdaddress '1'; data_in1 <= data_in; data_in2 <= data_in1; data_in3 <= data_in2; IF data_in3(7 DOWNTO 5) /= data_read_out(7 DOWNTO 5) THEN error_count <= error_count + '1'; END IF ; ELSE new_frame_analysis <= '0'; IF error_count > conv_std_logic_vector(200, VECTOR_LE NGTH + 1) THEN error_in_current_frame <= '1'; ELSE error_in_current_frame <= '0'; END IF ; error_count <= ( OTHERS => '0'); END IF ; END IF ; END IF ; END PROCESS processing_data; END ARCHITECTURE object_detect_1;

PAGE 53

45 Appendix B: VHDL Test Bench LIBRARY IEEE; USE IEEE.STD_LOGIC_1164. ALL ; USE IEEE.STD_LOGIC_UNSIGNED. ALL ; USE IEEE.STD_LOGIC_ARITH. ALL ; USE STD.TEXTIO. ALL ; USE IEEE. ALL ; ENTITY object_detect_tst_img IS END ENTITY object_detect_tst_img; ARCHITECTURE object_detect_tst_a OF object_detect_tst_img IS SIGNAL clock_50, reset_n, clock_out, frame_ready, refresh _frame_mem, SIGNAL error_in_current_frame : STD_LOGIC :='0'; -signals for the ports SIGNAL data_in : STD_LOGIC_VECTOR(7 DOWNTO 0); SIGNAL DUMMY_ref_frame_loaded : STD_LOGIC; -Signal t aken out of entity for testing SIGNAL done : STD_LOGIC := '0'; -flag set when simulat ion finished CONSTANT number_of_pixel_values : STD_LOGIC_VECTOR(13 DOWNTO 0) := "00000000001111"; -generic value SIGNAL address_frame_mem : STD_LOGIC_VECTOR(13 DOWNTO 0) := "00000000000000"; SIGNAL data_frame_mem : STD_LOGIC_VECTOR(7 DOWNTO 0); CONSTANT N : INTEGER := 12347; --Number of bytes in file m inus one SUBTYPE file_element IS STD_LOGIC_VECTOR(7 DOWNTO 0); TYPE mem_array IS ARRAY (N DOWNTO 0) OF file_element;

PAGE 54

46 Appendix B: (Continued) COMPONENT object_detect IS GENERIC ( ROWS : INTEGER := 98; COLUMNS : INTEGER := 126; VECTOR_LENGTH : INTEGER := 13); PORT ( clock_50 : IN STD_LOGIC; -system clock from on board source reset_ : IN STD_LOGIC; -system reset signal clock_out : OUT STD_LOGIC; -output c lock to read data from the frame capture block data_in : IN STD_LOGIC_VECTOR(7 DOWNTO 0); -data from the frame read frame_ready : IN STD_LOGIC; -the fram e ready signal from the frame capture block refresh_frame_mem : IN STD_LOGIC; error_in_current_frame : OUT STD_LOGIC; DUMMY_ref_frame_loaded : OUT STD_LOGIC; data_read_out1 : OUT STD_LOGIC_VECTOR(7 DOWNTO 0); -------------------------------------------------------------------------address_frame_mem : IN STD_LOGIC_VECTOR(VECTOR_LENGTH DOWNTO 0); data_frame_mem : OUT STD_LOGIC_VECTOR(7 DOWNTO 0); new_frame_analysis1 : OUT STD_LOGIC); --output signal showing presence of some unwanted --object in the image END COMPONENT ; SIGNAL tmp_data : STD_LOGIC_VECTOR(7 DOWNTO 0); -temporary data SIGNAL new_frame_analysis1 : STD_LOGIC; BEGIN -object_detect_tst_a U1 : COMPONENT object_detect PORT MAP ( clock_50 => clock_50, reset_n => r eset_n, clock_out => clock_out, frame_read y => frame_ready, refresh_frame_mem => refresh_frame _mem, error_in_current_frame => error_in _current_frame, data_in => data_in, DUMMY_ref_frame_loaded => DUMMY_re f_frame_loaded, address_frame_mem => address_ frame_mem, data_frame_mem => data_fra me_mem, new_frame_analysis1 => new_fram e_analysis1); clock_50 <= NOT clock_50 AFTER 10 ns;

PAGE 55

47 Appendix B: (Continued) Reset_control: PROCESS IS BEGIN -process Test reset_n <= '0'; WAIT FOR 150 ns; reset_n <= '1'; WAIT ; END PROCESS Reset_control; load_new_frame : PROCESS IS BEGIN -process load_new_frame WAIT UNTIL reset_n = '1'; frame_ready <= '1'; refresh_frame_mem <= '1'; WAIT UNTIL (clock_50'event AND clock_50 = '1'); frame_ready <= '0'; refresh_frame_mem <= '0'; data_in <= "10101010"; WAIT UNTIL DUMMY_ref_frame_loaded = '1'; WAIT FOR 300 ns; frame_ready <= '1'; WAIT UNTIL (clock_50'event AND clock_50 = '1'); frame_ready <= '0'; data_in <= "01010101"; WAIT ; END PROCESS load_new_frame; read_file : PROCESS IS -read file_io.in (one time at start of simulation ) SUBTYPE INTEGER_8bit IS INTEGER range 0 TO 255; TYPE char_type IS FILE of INTEGER; -file type for declaring the file VARIABLE ch : INTEGER; VARIABLE tmp1, tmp2, tmp3, tmp4 : INTEGER; -temporary to extract byets -the char acter being read from the file FILE my_input : char_type OPEN READ_MODE IS "pirates.raw"; FILE my_check : char_type OPEN READ_MODE IS "pirates.raw"; FILE my_check2 : char_type OPEN WRITE_MODE IS "output.raw"; BEGIN WAIT UNTIL reset_n = '1'; WAIT UNTIL refresh_frame_mem = '1'; LOOP EXIT WHEN (endfile(my_input) OR DUMMY_ref_frame_loaded = '1'); read(my_input, ch); tmp4:= ch MOD 256; WAIT UNTIL clock_50 = '1' AND clock_50'event; WAIT FOR 5 ns;

PAGE 56

48 Appendix B: (Continued) data_in <= CONV_STD_LOGIC_VECTOR (tmp4, 8); ch := ch / 256; tmp3:= ch MOD 256; WAIT UNTIL clock_50 = '1' AND clock_50'event; WAIT FOR 5 ns; data_in <= CONV_STD_LOGIC_VECTOR (tmp3, 8); ch := ch/256; tmp2:= ch MOD 256; WAIT UNTIL clock_50 = '1' AND clock_50'event; WAIT FOR 5 ns; data_in <= CONV_STD_LOGIC_VECTOR (tmp2, 8); ch := ch/256; tmp1:= ch; WAIT UNTIL (clock_out'event AND clock_out = '1'); WAIT FOR 5 ns; data_in <= CONV_STD_LOGIC_VECTOR (ch, 8); -to_stdlogicvector(ch_vector); END LOOP ; REPORT "out of the ref frame loading loop" SEVERITY note; WAIT FOR 250 ns; WAIT UNTIL frame_ready = '1'; LOOP EXIT WHEN endfile (my_check); Read (my_check, ch); tmp4:= ch MOD 256; WAIT UNTIL clock_50 = '1' AND clock_50'event; WAIT FOR 5 ns; data_in <= CONV_STD_LOGIC_VECTOR (tmp4, 8); ch := ch / 256; tmp3:= ch MOD 256; WAIT UNTIL clock_50 = '1' AND clock_50'event; WAIT FOR 5 ns; data_in <= CONV_STD_LOGIC_VECTOR (tmp3, 8); ch := ch/256; tmp2:= ch MOD 256; WAIT UNTIL clock_50 = '1' AND clock_50'event; WAIT FOR 5 ns; data_in <= CONV_STD_LOGIC_VECTOR (tmp2, 8); ch := ch/256; tmp1:= ch; WAIT UNTIL (clock_out'event AND clock_out = '1'); WAIT FOR 5 ns; data_in <= CONV_STD_LOGIC_VECTOR (ch, 8); -to_stdlogicvector (ch_vector); END LOOP ;

PAGE 57

49 Appendix B: (Continued) REPORT "Simulation complete!!!!!!!" SEVERITY note; ------------------------------------------------------------------------reading from the cuddern frame memory -----------------------------------------------------------------------WAIT UNTIL new_frame_analysis1 = '0'; address_frame_mem <= CONV_STD_LOGIC_VECTOR (1 2347,14); WAIT FOR 2 ns; WHILE address_frame_mem > "00000000000000" LOOP address_frame_mem <= address_frame_mem '1 '; WAIT UNTIL clock_50'event AND clock_50 = '1'; ch := CONV_INTEGER (data_frame_mem); write (my_check2, ch); END LOOP; REPORT "reading the current memory complete" SEVERITY note; WAIT ; END PROCESS read_file; END ARCHITECTURE object_detect_tst_a;