USFDC Home  USF Electronic Theses and Dissertations   RSS 
Material Information
Subjects
Notes
Record Information

Full Text 
xml version 1.0 encoding UTF8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchemainstance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd leader nam Ka controlfield tag 001 001498279 003 fts 006 med 007 cr mnuuuuuu 008 041209s2004 flua sbm s0000 eng d datafield ind1 8 ind2 024 subfield code a E14SFE0000567 035 (OCoLC)57724081 9 AJU6884 b SE SFE0000567 040 FHM c FHM 090 TA145 (ONLINE) 1 100 Yang, Jidong. 0 245 Road crack condition performance modeling using recurrent Markov chains and artificial neural networks h [electronic resource] / by Jidong Yang. 260 [Tampa, Fla.] : University of South Florida, 2004. 502 Thesis (Ph.D.)University of South Florida, 2004. 504 Includes bibliographical references. 500 Includes vita. 516 Text (Electronic thesis) in PDF format. 538 System requirements: World Wide Web browser and PDF reader. Mode of access: World Wide Web. Title from PDF of title page. Document formatted into pages; contains 110 pages. 520 ABSTRACT: Timely identification of undesirable pavement crack conditions has been a major task in pavement management. Up to date, myriads of pavement performance models have been developed for forecasting pavement crack condition with the traditional preferred techniques being the use of regression relationships developed from laboratory and/or field statistical data. However, it becomes difficult for regression techniques to predict the crack performance accurately and robustly in the presence of a variety of tributary factors, high nonlinearity, and uncertainty. With the advancement of modeling techniques, two innovative breeds of models, Artificial Neural Networks and Markov Chains, have drawn increasing attention from researchers for modeling complex phenomena like the pavement crack performance.In this study, two distinct models, a recurrent Markov chain, and an Artificial Neural Network (ANN), were developed for modeling the performance of pavement crack condition with time. A logistic model was used to establish a dynamic relationship between transition probabilities associated with the pavement crack condition and the applicable tributary variables. The logistic model was then used conveniently to construct a recurrent Markov chain for use in predicting the crack performance of asphalt pavements in Florida. Florida pavement condition survey database were utilized to perform a case study of the proposed methodologies. For comparison purpose, a currently popular static Markov chain was also developed based on a homogeneous transition probability matrix that was derived from the crack index statistics of Florida pavement survey database.To evaluate the model performance, two comparisons were made; (1) between the recurrent Markov chain and the static Markov chain; and (2) between the recurrent Markov chain and the ANN. It is shown that the recurrent Markov chain outperforms both the static Markov chain and the ANN in terms of oneyear forecasting accuracy. Therefore, with high uncertainty typically experienced in the pavement condition deterioration process, the probabilistic dynamic modeling approach as embodied in the recurrent Markov chain provides a more appropriate and applicable methodology for modeling the pavement deterioration process with respect to cracks. 590 Coadviser: Lu, Jian John. Coadviser: Gunaratne, Manjriker. 653 deterioration. logistic model. transition probability. stochastic process. deterministic. 690 Dissertations, Academic z USF x Civil Engineering Doctoral. 773 t USF Electronic Theses and Dissertations. 4 856 u http://digital.lib.usf.edu/?e14.567 PAGE 1 Road Crack Condition Performance Modeling Using Recurrent Markov Chains And Artificial Neural Networks by Jidong Yang A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Civil and Environmental Engineering College of Engineering University of South Florida CoMajor Professor: Jian John Lu, Ph.D. CoMajor Professor: Manj riker Gunaratne, Ph.D. Ram Pendyala, Ph.D. Edward Mierzejewski, Ph.D. Lihua Li, Ph.D. Date of Approval: November 17, 2004 Keywords: transition probability, logistic model, deterioration, deterministic, stochastic process Copyright 2004 Jidong Yang PAGE 2 DEDICATION To my wife Rui Dai and my son Andrew Yang. PAGE 3 ACKNOWLEDGEMENTS I am grateful to my major professor Dr Jian John Lu, for his continuous guidance to my academic studies, and assistance with my researches for the past five years. I would like to express thanks to Dr. Manjrike r Gunaratne for his constructive directions and continuous encouragement. This disse rtation is not possibl e without their support and invaluable advice. I must also tha nk all my committee members, Dr. Edward A. Mierzejewski, Dr. Ram Pendyala and Dr. Lihua Li, for taki ng their valuable time to review my work. Special thanks to Dr. Polzin Steve for serving as the chair of the final defense committee. This research is an ensuing effort of the project titled Application of Neural Network Models for Forecasting of Paveme nt Crack Index and Pavement Condition Rating, which is sponsored by Florida Depa rtment of Transportation (FDOT). The author would like to take this opportunity to thank the support from FDOT, especially for Mr. Bruce Dietrich and Ms. Sandra Kang for th eir technical assistance and suggestions. PAGE 4 i TABLE OF CONTENTS LIST OF TABLES.............................................................................................................iv LIST OF FIGURES............................................................................................................v ABSTRACT......................................................................................................................v ii CHAPTER 1 INTRODUCTION.....................................................................................1 1.1 Background...........................................................................................................2 1.1.1 Pavement Management System.....................................................................2 1.1.2 Techniques Related to Pavement Performance Modeling.............................5 CHAPTER 2 LITERATURE REVIEW..........................................................................7 2.1 Technical Review of Pavement Performance Modeling..........................................7 2.1.1 Deterministic Models.......................................................................................7 2.1.1.1 Pure Empirical Models.............................................................................8 2.1.1.2 Mechanisticempirical Models.................................................................8 2.1.1.3 Expert System Models............................................................................10 2.1.2 Probabilistic Models.......................................................................................11 2.1.3 Biologicallyinspired Models.........................................................................13 2.2 State Practice..........................................................................................................18 2.3 Summary................................................................................................................20 CHAPTER 3 METHODOLOGY..................................................................................23 3.1 Markov Chains.......................................................................................................23 PAGE 5 ii 3.1.1 Theoretical Background.................................................................................23 3.1.2 Stateoftheart Review of Transition Probabilities Estimation.....................26 3.1.3 Framework of the Recurrent Markov Chain..................................................29 3.1.4 Estimation of Transition Probabilities using Logistic Model.........................29 3.1.4.1 Logistic Model........................................................................................30 3.1.4.2 Maximum Likelihood Estimation of the Model Parameters..................32 3.2 Artificial Neural Network (ANN)..........................................................................34 3.2.1 Architecture....................................................................................................35 3.2.2 Neuron Activation Function...........................................................................36 3.2.3 Learning Method............................................................................................38 CHAPTER 4 MODEL DEVELOPMENT.....................................................................43 4.1 Data Description.....................................................................................................43 4.1.1 Computation of Equivalent Single Axle Loads (ESAL)................................46 4.1.2 FDOT Crack Rating.......................................................................................47 4.2 Development of the Logistic Model......................................................................52 4.2.1 Definition of Condition States........................................................................52 4.2.2 Variable Definitions........................................................................................53 4.2.2.1 Binary Response Variable.......................................................................53 4.2.2.2 Dummy Variables...................................................................................54 4.2.2.3 Quantitative Variables.............................................................................55 4.2.3 Model Selection..............................................................................................55 4.2.4 Parametric Analysis of the Logistic Model....................................................62 4.2.5 Analysis of Model Sensitivity........................................................................65 PAGE 6 iii 4.3 Recurrent Markov Chain........................................................................................67 4.4 Modeling using Artificial Neural Networks...........................................................71 4.4.1 Model Architecture Design.............................................................................73 4.4.2 Use of the Trained ANN in Forecasting.........................................................80 CHAPTER 5 MODEL PERFORMANCE EVALUATION...........................................81 5.1 Comparison between the Recurrent Mar kov Chain and the Static Markov Chain83 5.2 Comparison between the Recurrent Markov Chain and the ANN.........................84 5.2.1 Comparison of Forecasting Errors.................................................................84 5.2.2 Goodness of Fit...............................................................................................85 5.3 Case Study of a Typical Individual Section...........................................................88 CHAPTER 6 SUMMARY, CONCLUSIONS AND RECOMMENDATIONS.............89 6.1 Summary................................................................................................................89 6.2 Conclusions............................................................................................................92 6.3 Recommendations..................................................................................................93 REFERENCES.................................................................................................................94 ABOUT THE AUTHOR........................................................................................End Page PAGE 7 iv LIST OF TABLES Table 4.1 Excerpt from Traffic Information Data Set....................................................44 Table 4.2 Excerpt from Roadway Condition Data Set...................................................44 Table 4.3 Numerical Deductions for Cracking Survey (Confined to Wheelpaths (cw))48 Table 4.4 Numerical Deductions for Cracking Survey (Outside of Wheelpaths (co))..49 Table 4.5 Definition of Crack Condition State..............................................................53 Table 4.6 Insignificant Variables....................................................................................58 Table 4.7 Significant Variables......................................................................................58 Table 4.8 Overall Model Goodness of Fit (Hosmer and Lemeshow Test)....................61 Table 4.9 Parameter Estimation of Different Data Sets.................................................65 Table 4.10 KruskalWallis Test.........................................................................................67 Table 4.11 Weight Matrix between Input Layer and Hidden Layer..................................79 Table 4.12 Weight Matrix between Hi dden Layer and Output Layer...............................79 Table 5.1 Static Transition Probability Matrix...............................................................83 Table 5.2 Comparison of Forecasting Errors of the Static Markov Chain and the Recurrent Markov Chain.................................................................................84 Table 5.3 Comparison of Forecasting Errors of the Recurrent Markov chain and the ANN................................................................................................................85 PAGE 8 v LIST OF FIGURES Figure 1.1 Typical PMS Architecture..............................................................................2 Figure 1.2 Typical Operational Model of PMS...............................................................3 Figure 1.3 Illustration of the Effect of Maintenance Activities on Pavement Performance....................................................................................................5 Figure 3.1 Framework of the Recurrent Markov Chain................................................29 Figure 3.2 A Typical Threelayered Neur on Network with One Output Neuron..........36 Figure 3.3 Diagram of Artificial Neuron.......................................................................37 Figure 4.1 Historical Pavement Deterioration Distribution...........................................45 Figure 4.2 Histogram of Pavement Age.........................................................................50 Figure 4.3 Histogram of Pavement Cycle......................................................................50 Figure 4.4 Histogram of Traffic Loads (ESAL).............................................................51 Figure 4.5 Histogram of Thickness of Asphalt Overlay................................................51 Figure 4.6 Predicted Variation of Cr ack Index in Different Cycles...............................62 Figure 4.7 Predicted Variation of Crack I ndex with Different Levels of ESAL............63 Figure 4.8 Deterioration Impact of Pa vement Age with Different Cycles.....................64 Figure 4.9 Deterioration Impact of Pavement Age with Different Levels of ESAL.....64 Figure 4.10 Illustration of the Recurrent Markov Chain.................................................70 Figure 4.11 Training Errors of Diff erent Number of Hidden Neurons...........................74 Figure 4.12 Testing Errors of Diff erent Number of Hidden Neurons.............................74 PAGE 9 vi Figure 4.13 Connection Weights Histogr am (8HiddenNeuron Network)....................76 Figure 4.14 Connection Weights Histogr am (13HiddenNeuron Network)..................76 Figure 4.15 Architecture of Crack For ecasting Model (Flexible Pavements)................78 Figure 5.1 Goodness of Fit the Recurrent Markov Chain...........................................87 Figure 5.2 Goodness of Fit the ANN..........................................................................87 Figure 5.3 Comparison of Longterm Performan ce of the Recurrent Markov Chain and the ANN.......................................................................................................88 PAGE 10 vii ROAD CRACK CONDITION PERF ORMANCE MODELING USING RECURRENT MARKOV CHAINS AND AR TIFICIAL NEURAL NETWORKS Jidong Yang ABSTRACT Timely identification of undesirable pa vement crack conditions has been a major task in pavement management. Up to date myriads of pavement performance models have been developed for forecasting pavement crack condition with the traditional preferred techniques being the use of regre ssion relationships deve loped from laboratory and/or field statistical data. However, it becomes difficult for regression techniques to predict the crack performance accurately and robustly in the presence of a variety of tributary factors, high nonlinearity, and un certainty. With the advancement of modeling techniques, two innovative br eeds of models, Artificial Ne ural Networks and Markov Chains, have drawn increas ing attention from resear chers for modeling complex phenomena like the pavement crack performance. In this study, two distinct models, a recurrent Markov chain, and an Artificial Neural Networ k (ANN), were developed for modeling the performance of pavement crack condition with time. A logistic model was used to establish a dynamic re lationship between transition probabilities associated with the pavement crack condition and the applicable tributary variables. The logistic model was then used conveniently to construct a r ecurrent Markov chain for use in predicting PAGE 11 viii the crack performance of asphalt pavements in Florida. Florida pavement condition survey database were utilized to perfor m a case study of the proposed methodologies. For comparison purpose, a currently popular st atic Markov chain was also developed based on a homogeneous transiti on probability matrix that was derived from the crack index statistics of Florida pavement surv ey database. To evaluate the model performance, two comparisons were made; (1) between the recurr ent Markov chain and the static Markov chain; and (2) between the recurrent Markov chain and the ANN. It is shown that the recurrent Markov chain outperf orms both the static Markov chain and the ANN in terms of oneyear forecasting accuracy. Therefore, with high uncertainty typically experienced in the pavement condition deterior ation process, the probabilistic dynamic modeling approach as embodied in the recurrent Markov ch ain provides a more appropriate and applicable methodology for mo deling the pavement deterioration process with respect to cracks. PAGE 12 1 CHAPTER 1 INTRODUCTION The past three decades has witnessed a shift of emphasis on nationwide highway programs from construction of new highway infrastructures to rehabilitation, maintenance and preservation of the existing highway infrastructures. Transportation Equity Act in the 21 st Century (TEA21) calls for coordi nated efforts to collect, store, manage, and analyze transportation related data, which lay a solid foundation for the establishment of PMS. Due to the increasing challenges in pavement maintenance and rehabilitation, a pavement management syst em (PMS) has become a very beneficial management tool for highway agencies. Th e high expenditures in curred in highway construction imply a significant saving even from a slight improvement in management of the highway investment. With establishm ent of pavement management system (PMS) in many highway agencies across the State, quality pavement performance models have been recognized to be critical for successful application of a PMS. As a result, an increasing research interest thrives in impr oving performance of pavement deterioration models for the past decade. The inventory da tabase established in the initial stage of a PMS provides researchers an indispensable data resource for the development of the quality pavement performance models. PAGE 13 As a crucial component of a PMS, pavement performance models provide decision makers with a valuable means for predicting pavement future condition, and hence allow them to efficiently allocate the limited funds for future pavement maintenance and rehabilitation. 1.1 Background 1.1.1 Pavement Management System A functional Pavement Management System consists of four basic components: inventory, analysis, output, and feedback, as shown in Figure 1.1. 2implementation programs. Feedback occurs when M&R are actually implemented; the Figure 1.1 Typical PMS Architecture Inventory provide a solid data basis, analysis component operates on inventory to identify financial need either at network level or project level. Output component is an organized form of analysis results, based on which decisions can be made regarding overall maintenance and rehabilitation (M&R) strategies, and detailed priority Feedback Inventory Analysis Output Database Performance Model PMS Summarization PAGE 14 3 project level. implemented improvements need to be updated in the inventory database. In addition, feedback is also used to track and evaluate the effects of various M&R measures. Pavement management typically operates at two levels, network level and At the network level, a priority program and work schedules are developed within overall budget constraints. On the other hand, at the project level, specific physical improvements are implemented according to the network decisions. Pavement performance model, which acts as the hub of the analysis component, is the engine of the whole management activities. The activities include: at the network level, (1) prediction of the future conditions of the pavement, (2) prediction of the future funding needed to keep the pavement network at an acceptable level, (3) comparison of the effects of various funding scenarios on the pavement network, and (4) justification of annual budget for rehabilitation; at the project level, (1) identification of the candidate projects for rehabilitation, (2) generation of rehabilitation alternatives for each candidate project, (3) technical and economic analysis of each alternative, and (4) justification of project rehabilitation activities. Figure 1.2 illustrates in detail a typical operational model of PMS. Figure 1.2 Typical Operational Model of PMS Network Level: a) New construction programs nce programs System data olicies/Financing ection data tandard Specification b) Maintena P c) Rehabilitation programs Project Level: a) Economic Analysis ral Design S S Budgets b) Structu c) Implementation PAGE 15 4 As it can be nly a technical tool but als seen, the pavement performa nce model is not o o one that has significant economic implications. Trad itionally, pavement performance has been referred to as servi ceability performance, a concept defined by Carey and Irick, which represents performance as the history of pavement serviceability with time. Since then, the concept of paveme nt performance has been widely analyzed and discussed by many researchers. Typica lly, pavement performance models or pavement deterioration models relate pavement condition, represented by any one indicator of pavement condition, to a set of e xplanatory variables, such as traffic loads, environmental, design, construction, and maintenance practices to simulate the mechanism of the pavement deterioration proc ess. If measured expl anatory variables are furnished, pavement performance models can predict the future condition of the pavement, based on which future management activities are schedu led. In order to make a decision as to when maintenance ac tivities are necessary, it is important to establish an action threshold in terms of the pavement cond ition. Usually, the rationale to set up the threshold is based on the deterior ation rate. Empirically the period of first several years after construction represents the slowest deteri oration period for a pavement. As time progresses, pavement condition becomes worse, and the deteri oration rate begins to increase until it comes to a reflection poin t after which the pavement deteriorates so quickly that it is no longer efficient to renov ate rather than rebuild it. However, the threshold value can vary depending on the rati ng systems and specific indicator that is used for pavement condition evaluation. A graphic illustration of the effect of maintenance activities on the pavement performance is shown in Figure 1.3. PAGE 16 Pavement Condition Meas ure Year Mainear Threshold Cycle 1 Cycle 2 tenance Y 5igure 1.3 Illustration of the Effect of Maintenance Activities on Pavement Performance 1.1.2 Techniques Related to Pavement Performance Modeling The magnitude, randomness, and complex interactions of the factors involved in the pav F ement deterioration process make it a complex phenomenon to model. It is impossible to find a mathematical function to accurately describe the mechanism underlying this phenomenon. With the advent of pavement management system (PMS), modeling tasks start to take a datadriven face. Myriads of researches have been accomplished regarding the pavement performance modeling. Traditional approaches are characteristic of regressionoriented modeling, such as pure empirical models and mechanisticempirical models. Pure empirical models assume the pavement condition to be a linear or polynomial function of a single variable such as age or cumulative traffic loading. Mechanisticempirical models include more mechanisticrelated variables, such as the type of base, strain energy at the bottom of asphalt layer, etc. As a result, PAGE 17 6 multivariate regression technique is often applied to estimate the model parameters. However, to apply the multivariate regression technique, linear parameters usually need to be assumed. On the other hand, recently, as an identifiable trend, two new nonlinear approaches, Markov chains and Artificial Neur al Networks, have been taking territory from the traditional regressionbased models. Artificial Neural Networks do not need to specify a function form, capable of abstra cting the underlying relationship between the dependent and independent variab les from the exemplar data pairs and express it in the form of weight matrices. Markov chains are ty pical of a stochastic process, which treats the pavement condition as a random variable, and are able to account for the inherent uncertainty associated with the pavement condition deterioration process. In the following section, a detailed review of the researches regarding pavement performance modeling is presented. PAGE 18 7 CHAPTER 2 LITERATURE REVIEW 2.1 Technical Review of Pavement Performance Modeling The last three decades witnessed an incr easing interest in the development of pavement performance models. Although pavement performance models may take different forms, typically, they relate the indicators of pavement conditions, such as cracking index, roughness, or rutting, to explan atory variables such as traffic loads, environmental factors, cycle, age, and pave ment structure. The purpose of a pavement performance model is to establish a causal relationship between the pavement condition and any of the factors that influences perf ormance of pavements over time. Three broad categories of pavement performance models currently exist. These are deterministic models, probabilistic models, a nd biologicallyinspired models. 2.1.1 Deterministic Models For deterministic models, the functional form is assumed to be explicitly specified. Deterministic models can be further divided into three subcategories, which are pure empirical models, mechanisticempirical models, and expert system models. PAGE 19 2.1.1.1 Pure Empirical Models Pure Empirical model is one of the most widely used models for pavement performance forecasting. A massive database is required in the modeling effort. A typical empirical model takes the form of a nonlinear polynomial curve that obeys specific boundary conditions as shown in Eq.2.1. 332210XaXaXaaPCR (2.1) where: PCR= pavement condition rating, X = pavement age in years, and = regression parameters. 3210,,,aaaa To assure the accuracy of such models, pavements need to be classified into families with each family having a unique set of parameters capturing its own characteristics. 2.1.1.2 Mechanisticempirical Models Historically, engineering knowledge of pavement behavior under traffic loading has been mostly based on mechanistic analyses of pavement structures. Mechanistic models are developed based on the mechanistic relationship among loading, stresses, strains, and deflections. Due to the complexity of the interactions among the factors relevant to pavement performance, only a few of this type of models have been successfully developed so far. Instead, the hybrid breed of mechanisticempirical models 8 PAGE 20 becomes popular. The mechanisticempirical model is the combination of the empirical method and the mechanistic knowledge. In particular, it involves a mechanistic model to calculate the pavement response (stresses, strains, deflections) under traffic loading, and an empirical function relating the pavement response to the pavement performance (cracking, roughness, and rutting etc.). An example of the models in this category is a pavement roughness model provided by Queiroz (1983) as shown in Eq.2.2. ))(10(08.9))(10(22.9297.1)log(23STAGEQI ))(log1}(10(57.5))(10(03.742NSENRH (2.2) where: QI = roughness (counts/km), AGE = pavement age in years, ST = surface type dummy variable (0 for as constructed and 1 for overlaid), RH = state of rehabilitation indicator (0 for as constructed and 1 for overlaid), SEN1 = strain energy at bottom of asphalt layer (10 4 kgf cm), and N = cumulative equivalent single axle loads (ESAL). By taking into account of the mechanistic characteristics of pavements, the mechanisticempirical models are able to perform better than the empirical models. A major drawback of this type of models is the considerable efforts involved in data acquisition. 9 PAGE 21 2.1.1.3 Expert System Models It is recognized that pure empirical models and mechanisticempirical models are both models demanding massive data support. In cases where data are deficient, experts can supplement knowledge. Expert models are developed based on the opinions of experienced engineers who are familiar with the deterioration patterns of different types of pavements. In practice, the amount of expert knowledge that enters these models varies depending on the highway agency. South Dakota Department of Transportation used this approach to develop their deterioration models (SD9314). In their effort, first, a scaling system was applied to develop the deduct values associated with each severity and extent classifications associated with defined distress types. Then, experienced engineers were asked to provide estimates of the ages of pavements to reach particular conditions in terms of severity and extent for different distress type. With these data, a regression analysis was performed to determine the coefficients for the specified model, which could take the following form: (2.3) cbtaPCI where: PCI = pavement condition index, a = the maximum value of the index, b = slope of the deterioration curve, c = exponent coefficient, and t = age of the pavement. 10 PAGE 22 11 The expert system model is an example of the intelligent systems that are designed to maximize the utilization of th e expert knowledge. However, it may pose a dangerous situation when the experts are actually wrong. Although many successful applications have been accomplished in many medical diagno stic systems, its application in modeling pavement performance is still limited. 2.1.2 Probabilistic Models The deterministic model assumes that the pavement behavior follow a predetermined pattern that can be formulated by a specific equation relating the pavement performance indicator to one or more explan atory variables. This may oversimplify the pavement deterioration process since the uncer tainty observed in pavement deterioration is not accounted for. An alternative appr oach, known as probabilis tic models, treats pavement condition as a random variable, is capable of taking into account the uncertainty associated with pavement deterioration. The most popular probabilistic modeling a pproach is through Markov chains. For the application of the Markov chains, a se t of transition probabi lities needs to be estimated. Historically, two methods were employed for derivation of these transition probabilities depending on the quantity of available pavement condition survey data. Due to the scarcity of data in th e initial stage of a PMS, paveme nt expert knowledge is usually consulted to obtain the stationary transition pr obability matrix. Considering the subjective nature of pavement expert knowledge and the variety of pavement deterioration patterns across the associated variables, the stationary transition probability matrix is generally questioned for the appropriateness. In a wellfunctioning PMS that has accumulated a PAGE 23 12 relatively sizable database; the transition proba bility matrix is usually deduced from the statistics of pavement condition survey data. Wang et al (1994) developed new transition probability matrices from the statistics of survey data for Network Optimization System for use by Arizona Department of Transportation. More recently, econometric methods have been attempted to make use of the available data resource for estimating the tr ansition probabilities. A number of studies have been identified involving the applicat ion of econometric methods in estimating infrastructure condition transiti on probabilities. Several typical applications in this field are discussed in detail as follows. Madanat et al (1995) proposed an ordered probit model for estimating infrastructure transition probabilities from infrastructure condition data. In this research, an incremental discrete deterioration mode l was constructed using an ordered probit model. The model treated faci lity deterioration as a latent variable, recognized the discrete ordinal nature of condition ratings, exp licitly links infrastructure deterioration to several explanatory variables, hence allows for computation of the nonstationary (i.e. time dependent) transition probability matr ix. As a case study of the methodology, a concrete bridge deck deteri oration model was formulated and estimated using Indiana State Bridge Inventory data base. Comparison was performed between modeled and observed frequency, it has been shown that the proposed methodology results in more accurate transition probabilities than the expectedvalue approach. Based on the previous work, Madanat et al (1997) formulated a randomeffects probit model, which is able to capture the heterogeneity in the data by accounting for PAGE 24 13 differences across infrastructure units that may not be appropriately reflected in the available explanatory variables. Ariaratnam et al (2001) presents a me thodology for predicting the likelihood that a particular infrastructure system is in a deficient state, using logistic regression models. The methodology is illustrated in a case study involving the ev aluation of the local sewer system of Edmonton, Alberta. Canada. Variable s of age, diameter, material, waste type, and average depth of cover are modeled. The outcome of the model does not produce a prediction of condition rating but rather provides decisionmakers with a means of evaluating sewer sections for the planning of future scheduled inspection, based on the deficiency probability. 2.1.3 Biologicallyinspired Models With deeper understanding of biologica l phenomena, such as functioning of human brain, nature evolution, etc., a new br eed of modeling methodologies has begun to thrive, which is generally named biologicallyi nspired models. Typical models in this category are genetic algorithms (GA) and artificial neural networks (ANN). A genetic algorithm derives its concept from the process of evolution in nature. First, a population of characteristic candidates for the optimi zation problem is created. Each of these candidates is termed as an individual. Th en, the individuals in the population go through a process of evolution. The evolution is usually achieved in a manner that is similar to the biological evolution: (1) eval uate the fitness of all indi viduals in the population; (2) create a new population through three key operations: crossover, reproduction, and mutation on individuals in old population; (3) discard the old population, and iterate PAGE 25 using the new population. One iteration is referred to as a generation. The three operations play a crucial role in the process of evolution. Reproduction allows the copy of better individuals to appear in the new population. Crossover allows different individuals to be created in the successive generation by merging material from individuals from the previous generation. Mutation is the operation that can infuse new information in a random way to the genetic search process. An application of genetic algorithm in the pavement performance modeling is done by Andrei et al (2000). In the research, a roughness performance model was developed by using the genetic programming algorithm. Various published Longterm pavement performance (LTPP) distress data and early results of ROLTPP data were utilized for the modeling. After running about 50 generations, the best model was finally obtained, which is expressed as: 14) (log1101SNRRRttt (2.4) where, R t = roughness of pavement at age t, R t1 = roughness of pavement at age t1, and SN = structural number modified for subgrade strength. As noticed, it is an iterative model. With the initial roughness R 0 and the pavement roughness condition at age t provided, R t can be forecast iteratively. Another important biologicallyinspired approach is artificial neural networks (ANN). ANN stems from understanding of the functioning of the human brain. It can be regarded as highly simplified models of the human brain system, which emulates PAGE 26 15 human brain abilities of lear ning, generalization, and abstra ction. Up to now, many ANN applications in modeling pavement perfor mance have been attempted, which produce inspiring results. Some typical applications in this field will be discussed in the following section in detail. A number of studies have i nvolved the application of arti ficial neural networks to model pavement performance over time. Four a pplications relevant to this research are discussed herein. AttohOkine et al. (1994) applied a ne ural network to develop a pavement roughness progression model. The training data were generate d from RODEMAN, a road deterioration and maintenance submodel of HDMIII. An empirical simulation model was used to generate roughness data. The neural network wa s then developed relating the pavement roughness to a set of factors causi ng pavement roughness: pavement structural deformation, incremental traffic loadings, extent of cracking a nd thickness of surface layer, incremental variation of rut depth, su rface defects such as patching and potholes, and environmental and other nontrafficrelate d variables such as road age etc.. Three different architectures of the neural network with one, two a nd three layers, respectively, were examined. The Backpropagation learni ng algorithm was used as the learning rule. The predicted results of the trained network were compared with the desired results in terms of the mean square error (MSE). It wa s concluded that the ap plication of neural networks in pavement deterioration modeli ng is feasible when a large database of pavement condition is available. On the ot her hand, since the modeling was accomplished using simulated data, it was recognized that the model mi ght not be general enough to perform well on other data sets, especially from pavements in service. PAGE 27 Shekharan (2000) developed ANN models to predict pavement conditions for five families of pavements: original flexible, overlaid flexible, composite, jointed, and continuously reinforced concrete pavements. The pavement condition was represented by pavement condition rating (PCR), a composite index derived by combining the distresses and roughness, formulated for the Mississippi Department of Transportation. In this approach, Genetic Adaptive Neural Network Training (GANNT) algorithm is employed. The explanatory variables that have been chosen as inputs to the neural network models are pavement structure, pavement history represented by pavement age in years, and traffic volume by cumulative 18kip equivalent single axle loads. In order to account for quality of maintenance activities, and to some extent the traffic volume, the classification according to Federal Aid System (FAS) is also included in the list of explanatory variables. To substantiate the predictive capability of ANN, the same data with the same explanatory variables are employed for developing regression models. Finally, comparison was made on ANN and regression modeling. The author concluded that for modeling purposes, artificial neural network algorithms are, in general, found to be a better tool as compared to regression techniques, for the simple reason that artificial neural networks provide a flexible form of mapping and can take into account any functional form of equation. OwusuAbabio (1998) applied neural networks to model performance of thick asphalt pavement (thickness152.4 mm (6 in.)). The database used for this study was developed through a survey of the Wisconsin Department of Transportation district offices and selected city governments. The indicator of pavement condition used in this study was the pavement distress index (PDI), which range from 0 to 100 with 0 being the 16 PAGE 28 17 best and 100 being the worst. The main fact ors assumed to affect the performance of nonoverlaid thick asphalt pavements include the pavement surface thickness, pavement age, traffic level (ESAL/day), base thickn ess, and roadbed condition. For comparison purposes, multiple linear regression (MLR) models were also developed. It was concluded that the ANN model outperforms th e MLR model in terms of standard error and R square value. In the research conducted by Lu et al at USF, a neural network model was developed to forecast pavement crack c ondition. In this study, the FDOT pavement condition database was used. Back propaga tion algorithm was employed for the network training. A threelayer neural network model was proposed for the modeling. Through trial and error, seven specific variables were selected as inputs. These are crack index time series variables, CI(t2), CI(t1), CI(t), which are the Crack Index in year t2, t1 and t, respectively, flexible type of pavement indicator (1 if flex ible, 0 otherwis e), rigid type of pavement indicator (1 if rigid, 0 otherwise), pavement cycle, and pavement age. The following years crack index (CI(t+1)) was pred icted as the output of neural network. For comparison purposes, a corresponding AR mode l was also developed. The comparison showed that the neural network model was mo re accurate than the AR model in terms of root mean square error (RMSE), average error and R square value. As the result of the research, the authors (Lou et al, 2001) concluded that the proposed neural network model could be an effective tool for pavement maintenance planning. PAGE 29 2.2 State Practice Although there are a variety of techniques available in developing pavement performance model, selection of a particular one depends on characteristics of local pavement deterioration experience, policies, and preference of local agencies. Colorado Department of Transportation developed various performance curves for each distress type. Three levels of performance curve are used, which are sitespecific, pavement family, default curve. The most desirable is sitespecific curve. If it is not available due to lack of data, family curves are used. If both are not available, default curves are applied. Washington State Department of Transportation (WSDOT) used performance equations for pavement condition forecasting. The generalized equation used by WSDOT is: mAgebcPSC)( (2.5) where, PSC=pavement structure condition Age = pavement age (time since new construction or last resurfacing) c = the maximum rating, b = slope coefficient m = exponential coefficient (controlling the degree of curvature of the performance curve) To ensure better fitted curve, various coefficients was developed for different localities across the State, such as Seattle, Wenatchee, Tumwater etc. 18 PAGE 30 Nevada Department of Transportation (NDOT) developed a set of performance models for the most commonly used maintenance and rehabilitation techniques in all NDOT districts. The data collected by NDOT personnel over the lifetime of each of these techniques were gathered and used to develop these models. The model uses traffic loads, environmental, material, and mixtures data in conjunction with actual performance data, as measured by PSI, to predict the longterm performance of a rehabilitation and maintenance technique. The following represent a typical performance model for asphalt concrete overlays. YEARESALSeFTTMINSNPMFDPTPSI14.0)71.7(0037.0078.027.019.023.083.0 (2.6) where, PSI = present serviceability index, DPT = depth of overlay, SN = structural number of existing pavement, PMF = percent mineral filler, TMIN = average minimum annual air temperature ( o F), ESALS = equivalent single axle loads, YEAR = year of performance, and FT = number of freezethaw cycles per year. Arizona Department of Transportation (ADOT) used Markov chain for pavement condition forecasting. The development of the award winning Network Optimization System by WoodwardClyde Consultants in 1980 for the ADOT was a pioneering effort 19 PAGE 31 20 to combine Markov process model with linea r programming. Subsequently, Connecticut Department of Transportation, Alaska De partment of Transportation, and Kansas Department of Transportation implemented Markovprocessbased prediction models in their pavement management systems. Two mathematical methods are curren tly used by Florida Department of Transportation (FDOT) for forecasting roadway conditions: (1) mean deterioration rate and (2) simple linear regression. In practice, one of the met hods that best fits the prior trend of the data is usually chosen. 2.3 Summary The literature review shows a series of researches that attempted to apply ANN in modeling pavement performance. Howeve r, due to the difficulty involved in interpretation of results, few of these m odels have been actually adopted by highway agencies. In contrast, Markov chain is a wellestablished approach, and has been extensively applied in the PMS of many highw ay agencies. Historically, homogeneous, i.e. timeindependent transitio n probability matrices were used in Markov chain for forecasting pavement condition deterioration over time. However, this may be contradictory to the nature of deteriorati on, which actually exhibits time dependence in the condition state transition. To overcom e this obvious weakness and improve model performance, various econometric methods ha ve been applied in estimating transition probabilities of infrastructure deterioration, su ch as bridge, sewer etc. In addition to account for the time dependence, these econometric methods attempted to capture various factors influencing pavement performance, such as material, structur e base, cycle, etc. PAGE 32 21 However, the Markov property, stated as limited historical dependency, has not been reflected in estimating the tran sition probabilities. As a critic al property, state dependence assumes that evolution of a Markov process at a future time, conditioned on its present and past value, depends only on its present value. To account for the state dependence, the lagged condition rating should be consid ered into estimation of the transition probabilities. With these considerations in mind, a logistic model is proposed for estimating the state transition probabilities. In the logistic model, the time dependence is accounted for by including pavement age as a predictor in the mo del specification. The state dependence is accounted for by expl icitly including the lagged condition rating as a predictor in the model specification. In addition, other explanatory variables, such as ESAL and cycle, are also included as th e predictors in the model specification. Finally, the logistic model is integrated into a recurrent Markov chain for forecasting pavement future conditions. As a case study, the logistic based recurrent Markov chain is used for forecasting the Florida pavement crack conditions. Impr oved model performance is expected since use of logistic models in Markov chain allo ws transition probabilities to respond to lagged pavement crack conditi on and various explanatory va riables as well, such as traffic load, age, cycle, etc. To illustrate the benefit of the pr oposed recurrent Markov chain over traditional static Markov chains, a transition probability matrix is derived from statistics computed on Florida pavement condition survey database, and is used in a homogenous Markov chain process for paveme nt crack condition forecasting. Forecasts from both models are compared. More accurate forecasts are expected from the recurrent Markov chains. PAGE 33 22 In addition to the Markov chains, recent research activities identified ANN as a potential technique for modeli ng pavement deterioration pro cess although it has not been practically implemented in any state PMS. For a comparative study, an ANN model is developed as well using the same data set as used in developing the recurrent Markov chain. Forecasts of the ANN model are comp ared with these of the recurrent Markov chain. Finally, discussions are made regarding pros and cons of each model and conclusion are drawn regarding the superiority of one model over the other. PAGE 34 23 CHAPTER 3 METHODOLOGY 3.1 Markov Chains Inherent variability of material proper ties, environmental conditions, and traffic characteristics cause the pavement performance to inherit characteristics of uncertainty. Probabilistic models treat pavement condition m easures such as crack index, ride index, and rut index as random variables, therefore, are able to account for the uncertainty associated with pavement deteriorati on. One popular probabilistic pavement performance model is the Markov chain, whic h is defined as a sp ecial case of Markov process where the state space of the process is discrete. As a discrete time stochastic process, Markov chains involve using transition probabilities for forecasting condition state transition over time sequence. 3.1.1 Theoretical Background A discrete time Markov process is defi ned by Parzen (1962) as a stochastic process with the state parameter X(t). Provided time series of t 1 t 2 , t n the conditional distribution of X(t n ) given the series of values of {X(t 1 ),X(t 2 ),,X(t n1 )} depends only on the immediate prev ious state value, i.e. X(t n1 ). This can be formulated as: PAGE 35 })(,...,)(,)()({112211 nnnnxtXxtXxtXxtXP })()({11 nnnnxtXxtXP (3.1) The set of possible values of a stochastic process defines its state space. A Markov process with discrete state space is called a Markov chain. In a nstate Markov chain, the state of the process at any time t is defined by a probability mass function that can be expressed as: 1};,...,,{)(21titnttpppptP (3.2) where, = probability that the process is in state i at time t. tip Given the process starting time of t, the probability mass function of the process at time (t+k) can be derived by multiplying the probability matrices for each of k transitive steps. This can be formulated as follows: ktktttttPPPtPktP,12,11,)()( (3.3) where, = the vector of probability mass function at any time t, )(tP = the vector of probability mass function at k )(ktP th step of the process, and jtit P = transition probability matrix from step t+i to step t+j. 24 PAGE 36 By assuming that transition probability functions depend only on the time difference, a stationary Markov chain process can be derived as shown in Eq.3.4. kttPtPktP))(()(1, (3.4) The transition matrix P t,t+1 can be expressed as: (3.5) 1,1,)1(1,21,11,)1(1,)1)(1(1,2)1(1,1)1(1,21,)1(21,221,211,11,)1(11,121,111,...........................ttnnttnnttnttnttnnttnnttnttnttnttnttttttnttnttttttppppppppppppppppP However, to model a deterioration process, a semiMarkov process is often used, where it is assumed that improvement in pavement condition is impossible unless maintenance or rehabilitation is implemented. Therefore, the transition probability matrix as described in Eq.3.5 is reduced as: 10...00...00..................0...1,)1(1,)1)(1(1,21,)1(21,221,11,)1(11,121,111,ttnnttnnttnttnttttnttnttttttpppppppppP (3.6) where, i = 1,2,3,n1. nijttijp11, 25 PAGE 37 The entry of 1 in the last row of the transition probability matrix corresponding to state n indicates a trapping state. The pavement condition cannot transfer further down from this state unless maintenance or rehabilitation is performed. Due to data limitations, it is difficult to estimate all the probabilities transferring from the present state to lower states. Instead, a simplified matrix is generally used in practice with the assumption that the condition can drop, at most, one state in a single duty cycle. With this assumption, the transition probability matrix can be further simplified to Eq.3.7. Nevertheless, this simplification assumption is not a critical constraint since either the duty cycle or the condition state can be arbitrarily defined to satisfy the assumption. (3.7) 10...00...00...............00000...1,)1(1,)1)(1(1,221,221,121,111,ttnnttnnttttttttttppppppP 261,1,tttt where, i = 1,2,3,,n1 1)1(iiiipp 3.1.2 Stateoftheart Review of Transition Probabilities Estimation This section reviews the stateoftheart methods that have been attempted for estimating the transition probabilities and serves as a detailed examination of studies specifically regarding the estimation of state transition probabilities. To model the pavement deterioration behavior, traditionally, the pavements are segmented according to PAGE 38 27 certain characteristics such as pavement type locality, etc. The purpose of segmentation is to capture the fact that transition probabi lities are a function of explanatory variables and to ensure consistent deterioration pa ttern within each group. As proposed by Carnahan et al (1987) and Jiang et al. (1988), for each group, a deterioration model with the condition state as the dependent variable and age as the independent variable is estimated by linear regression. Then, a tran sition probabilities matrix is estimated for each group by minimizing the sum of absolute (or squared) differences between the expected value of the condition state pr edicted by the regression model and the theoretical expected value derived from th e Markov transition probabilities. As pointed out by Madanat et al (1995), these models suffer from several methodological limitations and practical inconsistencies. First, it fails to capture the mechanism of the deterioration process because the change in condition with in an inspection period is not explicitly modeled as a function of explanatory variables. Second, segmentation results in a small sample size within each group, which restri cts the number of parameters that can be estimated. Finally, linking causal variables to facility condition ra ting directly does not recognize the latent nature of the infrastructure deterioration process. With panel data becoming available in the field, some researchers have recently applied econometric methodologies in modeling infrastructure deterioration. Combining wellestablished methodologies a nd quality facility characteristics data, these models are considered theoretically appropriate and practically feasible. Madanat et al. (1995) introduced an ordered probit model for estima ting transition probabilities from inspection data. The model assumes the existence of an underlying continuous random variable and therefore allows the latent nature of infrastructure performance to be captured. The PAGE 39 28 ordered probit model is used to construct an incremental discrete deterioration model in which the difference in observed condition rating is an indicator of the underlying latent deterioration. This model is used to compute a nonstationary (i.e. time dependent) transition matrix. Based on the previous work, Madanat et al. (1997) proposed an improved probit model with a randomeffec ts specification to account for the heterogeneity and extend the model to inves tigate the presence of state dependence. An implication of the research is that both he terogeneity and state dependence may need to be accounted for in developing probabilisti c infrastructure deterioration models. The stateoftheare review indicates a deficiency in modeling state dependence. This implies that traditional use of Ma rkov chain to model the pavement condition deterioration could be erroneous. In addition, Most of these studies were targeting to model bridge or sewer system deterioration. Few of econometric methods have been found in modeling pavement condition deterioration behavior over time. Most highway agencies, which adopted Markov chain as the pe rformance model in th eir PMS, still rely on static transition probabilities. However, as a totally different infrastructure, the mechanism of pavement condition deterioration may differ from that of bridges or sewer systems. One objective of this research is to establish a causal relationship between the transition probabilities and va rious explanatory variables through a logistic model. To actually account for the state dependency, th e lagged pavement crack condition index was explicitly included as a predictor in the model specification. In this research, a recurrent Markov chain model that is constructed base d on the logistic model was introduced and a corresponding procedure of applying the recu rrent Markov chain model in forecasting was established. PAGE 40 3.1.3 Framework of the Recurrent Markov chain The adjective recurrent refers to iterative process in applying the model for multiplestep forecasting. The model framework is illustrated in Figure 3.1. State condition Explanatory Variables Transition Probabilities (Cycle, Age, ESAL, etc.) 29 Figure 3.1 Framework of the Recurrent Markov Chain As shown in Figure 3.1, the recurrent Markov chain uses the transition probabilities, which are functions of explanatory variables and the lagged Pavement Condition Rating PCR(t), to forecast pavement condition in the next duty cycle PCR(t+1). For multiplestep forecasting, a recurrent process is applied, where the output of the process at one time step becomes the input at the next time step. The transition probabilities are estimated through a logistic model based on a set of explanatory variables and the lagged pavement condition rating. 3.1.4 Estimation of Transition Probabilities using Logistic Model Provided the assumption that pavement can only drop one state during one duty cycle, a binary choice situation exists for any pavement sections for next duty cycle, either remaining in current state or move to the next worse state. With this in mind, a logistic model is considered for establishing a relationship between the transition Estimated by the Logistic Model PCR(t) PCR(t+1) PAGE 41 30 probabilities and deterioration explanatory variables. The following section presents a theoretical background of the logistic model and how it can be derived from a utility function approach. 3.1.4.1 Logistic Model Discrete choice analysis is used to m odel the choice of one from a choice set comprised of a set of mutually exclusive alternatives. The multinomial logit (MNL) model (McFadden, 1973) is the most widely used discrete choice model. Binary choice model, a Logistic model in this study, is a reduced form of MNL where only two alternatives are included in the choice set. There are a number of interpretations of the underlying data generating process that produce the binary choice models. Generally, it is assumed that there are a set of measurable covariates, X, which can be used to help explain the choice of one alternative over the other. With de finition of an index function, X, the modeling of binary choice in th ese terms is typically done in one of three frameworks: utility function approach, latent regression approach, and conditional mean function approach. Among these, utility function approach is most convenient way to view migration behavior and economic oppor tunity. In the following context, utility function approach is used to illustrate the de rivation of a binary choice model, a logistic model. The utility function expresses the usefuln ess of an alternative in the choice makers consideration. Each utility functi on has two terms associated with it, (1) deterministic component and (2) disturbance component. Generally, a utility function can be written as: PAGE 42 ininnViU )( (3.8) where, = utility of alternative i for choice maker n, )(iUn = deterministic component of utility of alternative i for choice maker n, inV and in = disturbance component of utility of alternative i for choice maker n. Based on principles of utility maximization, the probability of choosing alternative i over j can be formulated as: (3.9) )(Pr)(Pr))()((Pr)(jnininjnjnjnininnnnVVobVVobjUiUobiP By assuming that the difference of disturbance terms ( injn ) is logistically distributed, a logistic model can be derived as shown in Eq.3.10. injnVVneiP11)( (3.10) Eq.3.10 suggests that the probability of choosing one alternative over the other depend only on the difference between utilities of two competing alternatives. Substitute index function X for deterministic component of utility function, logistic model is obtained: 31 PAGE 43 kmmnmXneiP011)( (3.11) kmmnmXnejP01)( 1 (3.12) where, )(iPn = probability of entity n choosing state i, )(jPn = probability of entity n choosing state j, n = entity n, X mn = the m th explanatory variable, and m = parameter associated with the m th variable. The index function X implies a linearityinparameter assumption, which offers great computational convenience for parameter estimation as will be shown in the next section. However, it is not necessarily a significant constraint for these variables that may have a nonlinear relationship to the utility function since a variety of function forms can be specified for the subject variables, such as Logarithm, exponents etc. 3.1.4.2 Maximum Likelihood Estimation of the Model Parameters Maximum likelihood method is usually used for parameter estimation. Assuming that observations in a statistical sample are drawn independently and randomly and the 32 PAGE 44 variables X n are nonstochastic, the logarithm likelihood function for the sample conditioned on the parameters can be written as: jninynynNniPiPL))(1()()(1 (3.13) where, )(iPn = probability of entity n choosing state i, ],...,,[10k N = sample size, y in = 1 if alternative i is actually chosen by entity n, otherwise 0, and y jn = 1 if alternative j is actually chosen by entity n, otherwise 0. By setting the first derivative of )( L with respect to equal to 0, a system of K nonlinear simultaneous equations with k unknowns, k ......, ,21 can be derived as follows: KkXiPyNnnknin,....1,0)]([1 (3.14) where, )(iPn = probability of entity n choosing state i, and X nk = vector of contributing variables. Solving the system of k nonlinear simultaneous equations, the maximum likelihood estimates of can be found. Since the log likelihood function is globally 33 PAGE 45 34 concave, the solution to the first order condi tions is the only solution to the problem under study. 3.2 Artificial Neural Network (ANN) An Artificial Neural Network (ANN) is a parallel informationprocessing system that has certain performance characteristics similar to biological neural networks. A neural net consists of a larg e number of simple processing elements called neurons. Each neuron is connected to other neurons by mean s of directed links and each directed link has a weight associated with it. The we ights acquired through the training process represent abstracted information from data set, which is used by the net to solve a particular problem. Some functi ons that neural networks are able to perform include: (1) classification making a decision on which category an input patt ern belongs to, (2) pattern matching given the input pattern, the neural network produces corresponding output pattern, (3) pattern comp letion presented with an incomplete pattern, the neural network produces the corresponding complete pattern, (4) optimizatio n provided with the initial values for a specific optimization pr oblem, the neural netw ork produces a set of variables that represent an acceptably optimized solution to the problem, and (5) simulation: presented with the current state ve ctor of a system or time series, the trained network generates structured sequence or patterns that simulate the behavior of the system with time. The capability that neural networks can execute such complicated tasks is attributed to its underlying parallel distributed com putational mechanism. The mechanism is supported by three crucial and interacting components: (1) pattern of PAGE 46 35 connection between neurons, which is referre d to as the architecture, (2) neuron activation function, and (3) method of determ ining the weight of the connections, which is referred to as learning algorithm. In order to construct a neural network for solving a particular problem, the above three key co mponents need to be determined first. 3.2.1 Architecture Significant efforts are needed to determin e the best architecture for a given ANN model. This includes determin ation of input and output vari ables, the number of hidden layers, and the number of hidden neurons in each hidden layers. Usually, a neural network with too few hidden neur ons is unable to learn sufficien tly from the training data set, whereas a neural network with too many hidden neurons will allow the network to memorize the training set instead of gene ralizing the acquired knowledge for unseen patterns. Haykin (1994) recommended using two hidden layers; the first one for extracting local features and the second one fo r extracting global features. However, with two hidden layers, a significant increase in the training time and a corresponding decrease in the efficiency of training process are e xperienced. Funahashi and Hornik et al. (1989) separately proved that any continuous func tion can be approximated with an arbitrary accuracy using a threelayered network. T hus, from a theoretical point of view, a threelayered network is adequate for purpose of function approxima tion. It has been shown in practice that onehiddenlayer ANN is sufficient for most applications. Due to the still vague understanding of the impacts of the variation of ANN architecture, a trial and error approach is conventionally empl oyed to select the appropriate number of PAGE 47 hidden neurons in the hidden layer for the problem under study. As an illustration, a typical threelayered neural network with one output neuron is shown in the Figure 3.2. Input Layer Hidden Layer Output Layer 36 X 1 X 2 . Y . . X n Figure 3.2 A Typical Threelayered Neuron Network with One Output Neuron 3.2.2 Neuron Activation Function A neural network consists of many neurons. Each neuron is an independent processing element (PE), having its own inputs and output. The term of distributed parallel computation is derived from the independence property of neurons. A typical neuron is shown in Figure 3.3. PAGE 48 x 1 Input w 1 From x 2 Other w 2 Processing Elements Output w n x n 37 Summation Transfer Figure 3.3 Diagram of Artificial Neuron The output shown in Figure 3.3 is calculated by the following equation: niiijwxfO1)( (3.15) where x i = the i th input, w i = the connection weight associated with i th input, O j = output of j th neuron, and f = the transfer function. As noticed, the processing of each neuron involves simply a weighted summation plus a function transfer. Five common transfer functions are generally used as neuron activation functions depending on the characteristics of the problem under study. These activation functions are linear, linear threshold, step, sigmoid and Gaussian. Among these, the most commonly used one is the sigmoid function due to its concise form and differentiability. The output of each neuron calculated by the sigmoid transfer function can be expressed as: PAGE 49 )(11)(yaeyfz (3.16) niiixwy1 (3.17) where, z = neuron output, y = input to the transfer function, a = gain of the sigmoid function, n = number of element in the input vector, x i = i th element in the input vector, and w i = weight of connection i. In this research, the sigmoid function was employed as the neuron activation function. 3.2.3 Learning Method The learning capability of ANN is achieved by adjusting the signs and magnitudes of their weights according to learning rules that seek to minimize a cost or error function. All learning methods can be classified into two categories: supervised learning and unsupervised learning. Supervised learning is a process that utilizes an external teacher and/or global information. Several popular supervised learning algorithms are error correction learning, reinforcement learning, stochastic learning, and hardwired systems. In the case of unsupervised learning, an external teacher or supervisor is not necessary. It 38 PAGE 50 39 relies only upon local information during the entire learning process by organizing presented data and discovering its emergent collective properties. The Backpropagation (BP) method, which is used in this research, falls into the category of supervised learning. It is the most widely used learning method in neural network modeling. It provides an opportunity for the multidimension vector mapping. Due to its generality, BP neural network can be used to tackle a wide array of problems. Moreover, BP method presents a clear mathematical concept and embraces ease of programming. These conveniences empower BP as a versatile and pragmatic mechanism to implement neural networks. Enormous software applications of neural networks use BP as the embedded learning law including Brainmaker as employed in this research effort. Once the architecture, neuron activation f unction, and learning method have been determined, a neuron network needs to be traine d using sample data in order to obtain the connection weight matrices, re presenting parameters of the network, which is required for real application. The training process cons ists of two steps. In the first step, the training patterns (a set of know n input and output pairs) obtai ned from a data source are fed into the input layer of the network. These inputs are then propagated through the network until the output layer is reached. The output of each neuron is computed by the transfer function in Eq.3.16, which squashes the range of input to be between 0 and 1.0. Then a forward preprocessing error is cal culated by using the following equation: PAGE 51 prmkrkrktotalYTE11221 (3.18) where, = square of the output error for all the patterns in the data sample; totalE p = the number of patterns in the data sample; m = the number of neurons in the output layer; = target value of neuron k for pattern r; and rkT =output of neuron k for pattern r based on the sigmoid function rkY )(yf In the second step, the above error is minimized by backpropagation of the error through the network. During this process, the individual error contribution caused by each layer is computed and distributed backward and the corresponding weight adjustments are made to minimize the error. Using a gradient descending method, the backpropagation weight adjustment for the connections between hidden layer and output layer can be expressed as Eq.3.19 ))1()()(()(1lwlwlwEllwlwjkjkjktotaljkjk (3.19) where, = the weight of link for training iteration l+1 between neuron j 1lwjk in the hidden layer and neuron k in output layer; 40 PAGE 52 = the weight of link for training iteration l between neuron j in the lwjk hidden layer and neuron k in output layer; 41 = the weight of link for training iteration l1 between neuron j in 1lwjk the hidden layer and neuron k in output layer; ) (l = positive constant termed the learning coefficient at iteration l; and )(l = momentum term used to achieve rapid convergence and avoid numerical vibration during training. Similarly, weight adjustment for the connections between input layer and hidden layer can be written as Eq.3.20 ))1()()(()(1lwlwlwEllwlwijijijtotalijij (3.20) where, = the weight of link for training iteration l+1 between neuron i 1lwij in the input layer and neuron j in hidden layer; = the weight of link for training iteration l between neuron i in the lwij input layer and neuron j in hidden layer; = the weight of link for training iteration l1 between neuron i in 1lwij the input layer and neuron j in hidden layer; PAGE 53 42EE The training approach discussed above is called batch training. In batch training, the weights are adjusted after all of the samples are processed. Batch training can guarantee to decrease gradually and speed up convergence as well. Training is considered complete when the overall error is lowered to an acceptable level. total total PAGE 54 43 CHAPTER 4 MODEL DEVELOPMENT 4.1 Data Description Two sources of data were utilized for the model development in this research. They are (1) Florida traffic information data, and (2) Florida roadway condition survey data. The Florida traffic data has been obt ained through the Florid a Traffic Information (FTI) CD published annually. This CD consists of traffic characteristic information on the roadways maintained by FDOT, such as p eak season factors, Kfactors, Dfactors, vehicle classification, truck percentage, historical Averag e Annual Daily Traffic (AADT), etc. The Florida roadway condition survey data is obtained from the FDOT State Materials Office, Gainesville, FL, which maintains a comprehensive roadway condition survey database. The database contains deta iled State roadway information, such as historical crack ratings, roadway identifica tion (RDWYID), section begin mileage (BMP) and section end mileage (EMP), roadway age, roadway type, number of lanes, district, system, maintenance cycle, asphalt overlay thickness, etc. Excerpts from each source of the data are illustrated in Table 4.1 and 4.2, respectively. PAGE 55 Table 4.1 Excerpt from Traffic Information Data Set CountFunctionSite%SectionLocationCountySiteYearAADTClassTypeTruck010500004.69301000119942550016P6.1010500005.69301000119952550016P1.9010500006.69301000119962550016P3.3010500007.69301000119973000016P3.1010500008.69301000119982900016P3.2010500009.69301000119993000016P3.10105000010.69301000120003100016P50105000011.69301000120013350016P4.40105000012.69301000120023300016P3.7 Table 4.2 Excerpt from Roadway Condition Data Set SectionBmpEmpSideAsThickSystemLanesTypeCycleAgeDistrictCrk1986Crk2003090100000.000 6.527 L2.512121517.71090100000.000 6.527 R2.5121215184.5090100006.527 15.686L312131017.77.5090100006.527 17.196R312121018.75.5120200003.830 4.354 L412121019.48120200003.830 4.354 R412121019.47.5120200004.354 5.133 L413131019.48120200004.354 5.133 R413131019.47.5120200005.133 5.716 L41313101105.5 Historical data on oneyear pavement condition deterioration from 1986 to 2003 was examined for 7434 flexible roadway segments. The percent distribution of pavement sections with respect to the deterioration on the condition rating scale is illustrated in Figure 4.1. 44 PAGE 56 1.59%0.23%0.11%0.07%98.00%00.10.20.30.40.50.60.70.80.9112345Deterioration on the Crack Condition Rating ScalePercent Figure 4.1 Historical Distribution of Flexible Pavement Deterioration As shown in Figure 4.1, the majority of flexible pavement sections, about 98 percent, deteriorate up to one integer in the condition rating scale within one duty cycle defined as one calendar year. Only two percent of pavement sections deteriorate more than one integer in the condition rating scale. This information verifies the assumption made in the proposed recurrent Markov chain that pavements deteriorate, at most, one state (one integer interval in the condition rating scale) for one duty cycle under normal traffic conditions. 45 PAGE 57 4.1.1 Computation of Equivalent Single Axle Loads (ESAL) Although some performance models include Average Annual Daily Traffic (ADT) as a predictor variable, ADT is not an appropriate representation of traffic loading because the traffic loading effect on the pavement condition deterioration is mainly caused by heavy vehicles such as trucks, and not passenger cars. Hence more accurate representation of the traffic loading is achieved using the Equivalent Single Axle Loads (ESAL). In this study, ESAL per lane were computed from the Average Annual Daily Traffic (AADT) for each roadway segment, and treated as a predictor variable of the proposed logistic model. As shown in Table 4.1 and 4.2, the two data sources can be integrated through a common roadway identification number and the milepost reference location number. This integration allows AADT and the truck factor to be identified and thus the ESAL per lane to be calculated for each roadway section. The FDOT ESAL computation equation developed for pavement design purposes is used for computing ESAL per lane for each roadway segment as: LFFNEDTAADTESAL36524 (4.1) where, ESAL = the number of 18kip (80kN) Equivalent Single Axle Loads; AADT = Average Annual Daily Traffic; T 24 = Percent heavy trucks during a 24hour period; D F = Directional split factor; 46 PAGE 58 47 E F = Load Equivalent Factor, and N L = Number of Lanes. 4.1.2 FDOT Crack Rating Among all roadway distress types, cracking is the most critical i ndicator that often governs the overall roadway c ondition. Visual surveys have been employed by FDOT to evaluate the pavement crack condition. The designated survey crew drives an inspection vehicle at a reduced speed to check visually the entire pavement section and record the overall crack condition of the section. To fac ilitate crack data collection, three distinct types of cracking have been considered by FDOT: Class IB: this category include s hairline cracks that are 1/8 inch (3.18 millimeters) wide either in the longitudinal or transverse direction. Class II: this category includes cracks with an open width from 1/8 inch (3.18 millimeters) to 1/4 inch (6.35 millimeters) either in the longitudinal or transverse direction. These cracks may have moderate sp alling or severe branching. It is also includes cracks with an open width less than 1/4 inch (6.35 millimeters) which have formed cells less than 2 feet (0. 61 meters) on the longest side (alligator cracking). Class III: this category includes cracks with open width 1/4 inch (6.35 millimeters) or greater and extend ing in a longitudinal or transv erse direction and those open to the base or underlying material. It al so includes progressi ve Class II cracking resulting in severe spalling with chunks of pavement breaking out. Severe PAGE 59 raveling (loss of surface aggregate) or patching would also be classified as Class III cracking. The crack rating (CR) is obtained by subtracting the negative deduct values associated with various forms of cracking from 10 as shown in Eq.4.2 CR = 10 (cw + co) (4.2) where, cw = deduct value confined to wheelpaths, and co = deduct value outside of wheelpaths Deduct values for flexible pavements are shown in Tables 4.3 and 4.4. A crack rating of 10 indicates a pavement without observable distress or with only minor observable distress. Table 4.3 Numerical Deductions for Cracking Survey (Confined to Wheelpaths (cw)) 1B Cracking DeductII Cracking DeductIII Cracking Deduct00050.00.00.006250.51.01.026501.01.52.051+1.52.03.0Percent of Pavement Area Affected by CrackingPredominate Cracking Class 48 PAGE 60 Table 4.4 Numerical Deductions for Cracking Survey (Outside of Wheelpaths (co)) 1B Cracking DeductII Cracking DeductIII Cracking Deduct00050.00.51.006251.02.02.526502.03.04.551+3.55.07.0Percent of Pavement Area Affected by CrackingPredominate Cracking Class In view of tremendous efforts associated with data integration and preprocessing, codes were developed in Visual Basic, which can import traffic data and roadway condition data into a MS Access database, where the two data sets were combined and an integrated database was created with both roadway characteristics data and traffic data. Then, the integrated data set was imported into the SAS system. Finally, SAS codes were developed for data preprocessing purposes. In view of the magnitude of the aggregated database, it is cumbersome to utilize the entire database for modeling. Moreover the amount of observations that can be handled by the modeling software is often limited. Therefore, a sample data set was drawn for convenient manageability. The objectives of data preprocessing include: (1) Removal of the observations with critical missing data, (2) Elimination of irrational condition rating data (improved conditions without rehabilitation), (3) Sampling of data for modeling purpose, and (4) Preparation of the data set for model validation. 49 PAGE 61 As the result of data preprocessing, data sets were prepared and made ready for model development and model validation. For the derived sample data set, histograms were drawn for each individual variable as shown in Figures 4.24.5. 050100150200250300350123456789101112131415Pavement Age (Year)Frequency of Sections Figure 4.2 Histogram of Pavement Age 020040060080010001200140016001234Pavement CycleFrequency of Sections Figure 4.3 Histogram of Pavement Cycle 50 PAGE 62 01002003004005006007008009001000<= 10,000 10,000 50,000 50,000 100,000 100,000 200,000 200,000 300,000> 300,000ESALFrequency of Sections Figure 4.4 Histogram of Traffic Loads (ESAL) 0100200300400500600700800900100011223344556Asphalt Thickness (Inches)Frequency of Sections Figure 4.5 Histogram of Thickness of Asphalt Overlay As shown in Figure 4.2 to Figure 4.5, the major variables in the sample data set adequately covered their typical range of values. Therefore, the sample data set is deemed as a good representation of the entire database. Crack condition survey data in 51 PAGE 63 52 2003, the latest crack condition data contained in the database, are reserved and used for model evaluation purpose. 4.2 Development of the Logistic Model The following sections discuss in detail the definition of variables as used for development of the logistic model for es timating pavement crack condition transition probabilities and the procedures used for the selection of model specifications. After the model specification was selected, a parametric analysis was performed to examine if the model is a rational representation of the pavement condition deterioration mechanism with respect to various explan atory variables. Subsequently a sensitivity analysis was performed to test the robustness of the model against different data sets. Finally, the application of the logistic m odel in a recurrent Ma rkov chain for realis tic forecasting is presented. 4.2.1 Definition of Condition States For application of the Markov chain in modeling pavement crack condition performance, a suitable definition of the condition states must be adopted. As discussed in section 4.1.1, Crack Index (CI) is rate d on a 010 scale where 10 indicates the best condition and 0 the worst. Therefore, the pave ment crack index was categorized into 10 states with one integer interv al representing each state, as shown in Table 4.5. In pavement management practices, a duty cycl e is normally defined as one year since seasonal climate change is cycled in one year and traffic is usually measured on an annual variation basis, using Average Annual Da ily Traffic (AADT). Hence the 10state PAGE 64 pavement condition discretization scheme assures that the pavement crack conditions would not drop more than one state in a single duty cycle (typically, one year) under normal traffic conditions. Table 4.5 Definition of Crack Condition State Crack Condition StateCrack Rating Range109< CI <=1098< CI <=987< CI <=876< CI <=765< CI <=654< CI <=543< CI <=432< CI <=321< CI <=210=< CI <=1 4.2.2 Variable Definitions It may not be appropriate to directly use the existing variables in the database. Sometimes, transformation or categorization of some variables may be necessary for the modeling purpose. This section describes in detail the variables that would be used for modeling, and how they were compiled before usage. 4.2.2.1 Binary Response Variable Binary response variables are those that only have two possible values. The status of the pavement crack condition can be considered as a binary response variable. If the assumption is made that a given pavement section can only drop one state in a duty cycle, the resulting crack condition after one duty cycle can be regarded as a binary variable 53 PAGE 65 54 which either remains in the current condition state or deteriorates to a lower condition state. This binary function can be formulated as follows: Y n,i = 1, Y n,(i+1) = 0 if i1 < CI(t+1) <= i given i1 < CI(t) <= i (i=2,,9) (4.3) Y n,i = 0, Y n,(i+1) = 1 if i < CI(t+1) <= i+1 given i1 < CI(t) <= i (i=2,,9) (4.4) where, i= condition state, n = pavement section number, Y n,i Y n,(i+1) = binary variable indicating the ne w state of the pavement section after one duty cycle, CI(t) = crack condition index at time t, and CI(t+1) = crack condit ion index at time t+1. 4.2.2.2 Dummy Variables Dummy variables are artificial variab les representing the categories of a qualitative variable. It is used under the assumption that no distance exists between categories. Each variable assume one of two values, 1 or 0, indicating whether an observation falls in a particul ar category or not. Pavement cycle is a nominal variable, which is defined as the number of overlays th at has been applied before reconstruction of pavements. In case where the nominal variable has more than two levels, multiple dummy variables need to be created to repr esent the nominal variable. The total number of dummy variables required is one less than the number of values of the original nominal variable since one nominal variable has to be specified as the base case for reference which does not appear in the model specification. In the current work, Cycle 1 PAGE 66 55 is referred to as the base case and hence three additional dummy variables were defined based on Cycle 1 as follows: Group 1: 1 when Cycle = 2, 0 otherwise; Group 2: 1 when Cycle = 3, 0 otherwise; Group 3: 1 when Cycle = 4, 0 otherwise. 4.2.2.3 Quantitative Variables The quantitative variables are these associated with numerical values. ESAL and crack index (CI) are the quantitative variables in this case. ESAL is calculated according to Eq.4.1, which represents cumulative traffic loading in one duty cycle. Due to the magnitude of ESAL, direct use of ESAL re sults in unbalanced parameter estimates. Therefore, a 10based logarithm transfor m of ESAL is used in the model. 4.2.3 Model Selection A backward stepwise elimination proce dure was employed for selecting variables to be included in the model specification. It starts with the complete model with all possible explanatory variables, and sequentially removes variables from the model one at a time, based on a specific criterion, such as statistical significance (ex: 0.05 significance level) or the improvement in the explained variance. Three types of Hypothesis tests were involved in the model sele ction process; (1) the significance test for each model para meter by performing a Wald test. (2) determination of significance of multiple parameters using a likelihoodratio test. (3) examination of the overall model fit using a Hosmer & Lemeshow goodness of fit test. The three Hypothesis tests are di scussed in detail as follows: PAGE 67 Wald tests are based on Chisquare statistics that tests the null hypothesis that a given parameter is 0, or in other words, that the corresponding variable has no significant effect given that the other variables are in the model. The likelihood ratio test is used for joint testing of several parameters. It compares two different model specifications by testing whether the extra parameters in the relatively more complex model equal zero. The test begins with a comparison of the likelihood scores of the two models. The test statistic can be formulated by Eq.4.5, which approximately follows a chisquare distribution with k degrees of freedom where k is the number of additional parameters in the more complex model. )log(log2)log(21010LLLL (4.5) where, L 0 = likelihood score of the simpler model, and L 1 = likelihood score of the more complex model. The assessment of the fittingness of a model is a very important component in any modeling procedure. Goodnessoffit tests try to evaluate how well modelbased predicted outcomes coincide with the observed data. However, in the logistic regression models, investigating the goodnessoffit is often problematic when continuous covariates are modeled, since the approximate chisquared null distributions for the Pearson test statistic is no longer valid. Categorization might provide a solution for this problem, but it is often not clear how the categories should be defined. Hosmer and Lemeshow (1980) were the first to propose a goodnessoffit test that can be used for logistic regression models with continuous predictors. It takes an alternative approach to grouping: it groups 56 PAGE 68 the predictions of a logistic regression model rather than the models predictor variable data, which is the Pearson statistics approach. In the implementation found in the Business Analysis Module, mode predictions are split into G bins that are filled as evenly as possible, sometimes called equal massing binning. Then the statistic can be computed using the following equation: GjjjjnjjnjoHL1)1(2)( (4.6) where, = total frequency of event outcomes in group j, jo = total frequency of subjects in group j, and jn j = average estimated probability of an event outcome in group j. The HosmerLemeshow statistic follows a Chisquare distribution with G2 degrees of freedom. However, caution should be exercised when the sample size is relatively small i.e. less than 400. In the model selection process, Wald test was performed on each parameter of the model to investigate the significance of the individual parameters. Table 4.6 lists those variables that do not meet the 0.05 significant level criterion, and therefore have been removed. Table 4.7 shows the variables that meet the 0.05 significance level criterion, and hence are included in the final model specification. 57 PAGE 69 Table 4.6 Insignificant Variables WaldVariableStatisticSignificanceThickness0.38260.5362CI*Cycle0.00060.9801Age*Thickness0.01000.9205Cycle*Thickness0.03700.8475CI*Thickness0.07860.7792CI*Log(ESAL)0.94720.3304Log(ESAL)*Thickness1.00500.3161 Table 4.6 also indicates that the new asphalt overlay thickness is not a significant variable by itself. Neither do all the interaction effects related to it. This is not a surprising finding from a structural mechanistic viewpoint since the thickness of the new asphalt overlay is not as critical as the pavement base or subgrade. The difference in thickness will therefore have a minor effect on pavement deterioration. The model is expected to be improved if the thickness of base enters the model. Unfortunately, this information was unavailable in a readytouse form. Table 4.7 Significant Variables ParameterWaldVariableEstimateStatisticSignificanceConstant8.4246 10.4599 0.0012CI0.7134 28.3502 0.0000Age1.3485 16.3611 0.0000Log(ESAL)2.0418 23.9186 0.0000Cycle 21.5347 11.5328 0.0007Cycle 31.0964 5.6401 0.0176Cycle 41.5278 8.0936 0.0044Age*Age0.0337 31.4722 0.0000Age*CI0.0503 14.7651 0.0001Age*Log(ESAL)0.2191 18.5292 0.0000Cycle Age0.1327 7.9318 0.0049Summary Statistics:Number of Observations2552 L(C)1220.468L(B)1050.911 58 PAGE 70 59 Table 4.7 lists the variables that are signi ficant at the 0.01 level except for cycle 3, which is significant at the 0.05 level. Negative sign of the crack condition reveals that the better the current condition the lower the pr obability of deterioration is. Positive signs of age and logarithm of ESAL indicate older pavements with higher traffic loading tend to have higher probability of deterioration. Furthermore, positive coefficients of the dummy variable for the second cycle, the thir d cycle and the fourth cycle indicate higher deterioration propensity of pavements in these cycles than those in the first cycle, which reflects a totally new condition. These result s are intuitively expected. However, an unexpected result occurs when comparing the effects of different cycles on the deterioration. The magnitudes of coefficient of different cycles reveal that the pavement sections in the third cycle te nd to deteriorate slower than those in the second cycle. However, pavement sections in the fourth cycle have almost the same deterioration probability as those in the second cycle. This may be explained by the definition of cycle. According to the definition, a new cycle begins after the application of an asphalt overlay. Therefore, it can be deduced th at the cycle is a func tion of two variables, (1) cumulative damage (compared to the new facilities) and (2) improvements (new surface condition and thicker pavement, resulti ng in a stiffer pavement). A higher cycle implies higher cumulative damage and also an in creased stiffness. Therefore, the effect of cycle on pavement deterioratio n is a resultant contribution of the two competing factors. With this in mind, the complexity can be well explained. The pavement sections in the second cycle have a higher dete rioration probability in gene ral than those in the first cycle because the pavements in the second cycl e have a more dominant contribution from the cumulative damage than from the impr ovements. The pavements in the third cycle PAGE 71 60 still have a higher deterioration probability than those in the first cycle, but lower deterioration probability than those in the seco nd cycle. This implies that in the third cycle, the contribution from cumulative dama ge has been overcome by the improvements compared to the second cycle. The pavements in the fourth cycle seem to have almost the same deterioration probability as those in the second cycle because the cumulative damage tends to cancel the increased stiffness due to the improvements. The likelihood ratio test was perfor med to examine the overall model specification and check if all the parameters other than the constant term are significant or not. As shown in Table 4.7, L(C) = 1220.468, L(B)=1050.911. The likelihood ratio can be computed as L = 2(L(C)L(B)) = 339.114 > 23.21 (critical ChiSquare value with 10 degree of freedom at 0.01 significance level). Therefore, the null hypothesis that all the parameters are equal to zero is rejected. The HosmerLemeshow goodness of fit test was used to test the overall model fittingness. The results are shown in Table 4.8. The Null hypothesis for this test is that the data fits the specified model. In view of the high pvalue (0.3027), the Null hypothesis is not rejected. Thus, the conclu sion may be drawn that the data fit the specified model. PAGE 72 Table 4.8 Overall Model Goodness of Fit (Hosmer and Lemeshow Test) State RemainState DropGroupTotalObservedExpectedObservedExpected125734.20254252.80225639.47253246.5332571715.76240241.2442562724.22229231.7852553735.56218219.4462555046.89205208.1172555760.62198194.3882557472.77181182.2392569887.14158168.8610250105114.33145135.67Hosmer and Lemeshow Goodness of Fit Test:ChiSquare = 9.4901Degrees of Freedom = 8pvalue = 0.3027 As the result of foregoing modeling efforts, the logistic model is finally obtained, and is expressed as follows: ),,),((11])()1([ESALAgeCycletCIfneitCIitCIP (4.7) ),,),((11])()1()1([ESALAgeCycletCIfneitCIitCIP (4.8) where, i = present crack condition state, t = present duty cycle, n = pavement section n, ])()1([itCIjtCIPn = probability of deteriorating to the next lower state i1 given present condition is in state i, ])()1([itCIitCIPn = probability of remaining in present state i given present condition is in state i, and 61 PAGE 73 AgeCycleLogESALAgetCIAgeAgeCycleCycleCycle E SAL L og A ge t CI E SAL A geCycle t CI f *1327.0*2191.0)(*0503.00337.045278.130964.125347.1)(0418.23485.1)(7134.04246.8),,),((2 4.2.4 Parametric Analysis of the Logistic Model To further evaluate the soundness of the model, a parametric analysis was performed to verify the estimated model parameters. The impact of each variable is evaluated by holding other variables constant at their mean values. Then, relationships were drawn for each influencing variable. Figure 4.6 Predicted Variation of Crack Index in Different Cycles Figure 4.6 shows the probability of remaining in the current state versus crack condition index. It can be seen that pavements in good condition have a higher probability of remaining in the current state than those in a poor condition. This finding concurs with the observations. It also shows that pavements in cycle 1 have the highest 62 PAGE 74 probability of remaining in the current state, and pavements in cycle 4 have the lowest probability of remaining in the current state. Pavements in cycles 2 and 3 lie in between these in cycles 1 and 4. Due to the complex interaction effect of damages and improvements inherited in each cycle that was discussed in section 4.2.3, pavements in cycle 3 have a higher probability of remaining in the same state than those in cycle 2. The variation of crack condition index at different levels of ESAL is plotted in Figure 4.7. The three levels of ESAL represent the pavements with low, medium, and high traffic loading, respectively. Figure 4.7 indicates that pavements with higher ESAL tend to have a lower probability of remaining in the current state. Figure 4.7 Predicted Variation of Crack Index with Different Levels of ESAL The variation of crack condition deterioration with pavement age in different cycles and levels of ESAL are illustrated in Figures 4.8 and 4.9. Figures 4.8 and 4.9 indicate that older pavements tend to have a lower probability of remaining in the current 63 PAGE 75 state and a higher probability of deteriorating to the next lower state. Similar patterns in the crack condition index across different cycles and levels of ESAL were observed for the pavement age as shown in Figures 4.6 and 4.7. Figure 4.8 Deterioration Impact of Pavement Age with Different Cycles Figure 4.9 Deterioration Impact of Pavement Age with Different Levels of ESAL 64 PAGE 76 4.2.5 Analysis of Model Sensitivity The objective of the sensitivity analysis is to test the reliability of the model structure using different data sets. In this analysis, two logistic models were developed under two scenarios using two different data sets, i.e. 80 % and 90% of the original data set selected randomly. The two models were subsequently compared to the original logistic model. For comparison purposes, the estimated model parameters using all three data sets are presented in Table 4.9. Table 4.9 Parameter Estimation of Different Data Sets 100% data sample90% data sample80% data sampleVariableEstimateSignificanceEstimateSignificanceEstimateSignificanceConstant8.42460.00128.52910.00258.41140.0060CI0.71340.00000.74970.00000.76060.0000Age1.34850.00001.38440.00001.41130.0004Log(ESAL)2.04180.00002.00490.00001.97470.0000Cycle 21.53470.00071.50910.00141.53100.0025Cycle 31.09640.01761.07710.02621.15270.0264Cycle 41.52780.00441.49230.00861.50970.0137Age*Age0.03370.00000.03260.00000.03700.0000Age*CI0.05030.00010.05190.00010.05100.0006Age*Log(ESAL)0.21910.00000.21860.00000.21180.0002Cycle Age0.13270.00490.12590.01040.12960.0147Sample Size255222972042 It can be seen that the coefficients estimated from the three data sets agree reasonably well in terms of both the sign and the magnitude (within 10% of each other). The Wald statistics for the coefficients were significant at a relatively lower level for the models based on 80% and 90% sample sets. To support this finding and statistically show that there is no difference among these three models, the KruskalWallis test was performed under the following Hypotheses: 65 PAGE 77 H 0 : The models are equal (there is no significant difference between models). H a : the models are different. To apply the KruskalWallis test, the following procedure needs to be followed: 1. Combine all the samples into one large sample, sort the result in the ascending order, and assign ranks. 2. Find r i the sum of the ranks of the observations in the ith sample. 3. Compute the test statistic KW using Eq.4.9 kiiiNnrNNKW)1(3)1(122 (4.9) 4. Under H 0 KW follows an approximate ChiSquare distribution with k1 degrees of freedom. 5. Reject the null hypothesis that all k models are the same if KW >. 21,k Projections of the probabilities of the pavement sections remaining in the current state and the corresponding rank measures across different ages for the three data scenarios are listed in Table 4.10. 66 PAGE 78 Table 4.10 KruskalWallis Test 67178 PavementProbability of Remaining in Current StateRank MeasureAge100% sample90% sample80% sample100% sample90% sample80% sampleCombined10.99410.99640.996543444513220.98730.99170.991840414212330.97450.98240.982337393811440.95280.96540.964534363510550.91940.93680.93473133329660.87330.89370.89062830298770.81670.83590.83282527267880.75500.76710.76652224236990.69500.69440.699620192160100.64330.62600.640218161751110.60420.56820.59421481335120.58020.52480.564710472130.57230.49720.55319351140.58100.48600.559711161150.60580.49100.58441521229Sum of ranks:3573273511,035KW:0.195 As shown in Table 4.10, the KW statistic is calculated to be 0.195 using Eq.4.9 and compared with the tabulated =4.61. Therefore, the null hypothesis is not rejected, indicating that no significant difference exists among the three models. Thus the conclusion that the proposed model is stable and may be deemed as a good representation of the data set can be drawn. 2 2,01.0 4.3 Recurrent Markov Chain Application of the Markov chain for forecasting the pavement condition requires a mechanism that can convert discrete states combined with transition probabilities back to the pavement condition rating. Condition state value provided in terms of the pavement crack index and probabilities associated with each condition state (probability mass function) can be used to compute the expected value of pavement crack condition in the next duty cycle using the following equation. PAGE 79 nijttijjpSItCI1,)1( (4.10) where, t = present duty cycle; t+1 = next duty cycle; CI(t+1) = pavement crack index in next duty cycle; = value of pavement crack condition state j; jSI = transition probabilities from state i to state j, and 1,ttijp n = number of states. In case where state distances are uniform, i.e. SI j+1 SI j =d (j=1, 2,n1), Eq.4.10 can be rewritten as: (4.11) nijttijipijdSItCI1,)()1( where, = mean value of current state i, and iSI d = uniform state distance. As indicated in Eqs.4.10 and 4.11, state value of the pavement crack condition, usually the mean pavement crack condition index of the subject state, is used in the Markov chain to convert transition probabilities back to crack conditions. This poses a serious limitation in the forecasting capability of Markov chains since variations in pavement crack conditions within a state are not accounted for. As discussed in Section 68 PAGE 80 4.2, the lagged condition index was introduced into the logistic model as a predictor for estimating transition probabilities, which results in a varying state distance, i.e. transition probabilities are functions of the present pavement crack condition and the state distance from the present crack condition to the next lower condition state depends on the present pavement crack condition, and should be calculated as d(t) = CI(t) . Accordingly, the actual present crack condition index CI(t) should be used instead of the state valuein Eq.4.11. With these considerations, Eq.4.11 is further transformed into: iSI iSI nijttijjpijSItCItCItCI11,))()(()()1( (4.12) Moreover, considering the assumption that pavement crack condition can drop only one state for one duty cycle, Eq.4.12 can be simplified as: (4.13) 1,)1(,1))(()()1(ttiiipSItCItCItCI In this research, Eq.4.13 was employed in the recurrent Markov chain for forecasting the evolution of pavement crack condition over time. The mechanism of the recurrent Markov chain is illustrated in Figure 4.10. 69 PAGE 81 33.544.555.566.577.588.599.5100123456789101112131415Duty Cycle (Year)Pavement Crack Index (PCI) State (i1) State iCI(t)State Value SI(i1) State Value SI ( i ) d2d1 Pavement Performance Curvet t+1 CI(t+1)d1 = dynamic state distanced2 = static state distance Figure 4.10 Illustration of the Recurrent Markov Chain As shown in Figure 4.10, d1 represents the dynamic crack condition state distance depending on the present pavement crack condition rating CI(t), and d2 represents the static crack condition state distance. As implied in the specification of the logistic model (Eqs.4.7 and 4.8), the transition probabilities are a function of the present crack condition index CI(t), age, cycle, and ESAL. Use of the logistic model in the recurrent Markov chain process is considered to be theoretically appropriate because it satisfies the Markov property assumption that the condition in the current duty cycle depends only on the condition in the previous duty cycle. In addition, it is practically feasible since the transition 70 PAGE 82 71 probabilities are dynamically linked to the appropriate explanatory variables so that variation of each explanatory variable can be captured in the transition probabilities. Therefore, the recurrent Markov chain model is expected to overperform its static counterpart in forecasting pavement crack condition. This will be substantiated by comparing the observed pavement crack co nditions in 2003 with forecasts of the proposed recurrent Markov chain and a static Markov chain de veloped for this purpose. 4.4 Modeling using Artificial Neural Networks In addition to the recurrent Markov chain, an ANN model is also developed. This section presents in detail the development of the ANN model. Similar to the traditional modeling process, where the objective is to esti mate a set of coefficients for a particular functional form of specification, the main objective of modeling with ANN was to attain a set of weight matrices, which represents the abstracted underlying knowledge from the example data after many loops of training. However, to use neural network to solve a particular reallife problem, appropriate architecture needs to be designed first according to the characteristics of the problem under st udy. The objective of ar chitecture design is to determine the number of layers, the number of neurons in each layer, variables to be included in the input layer a nd the output layer, etc. Once the ANN architecture design is completed, the ANN models are ready for trai ning, testing, and finally validation. Training a neural network i nvolves repeatedly presenti ng a set of example data pairs to the neural network. The neural network adapts its connection weights between the neurons in different layers according to the learning law. Eqs.3.18 and 3.19 were used as the learning law for this research. The result of training is a set of weight matrices, PAGE 83 72 which stores the knowledge gained from the ex ample data set. Testing a neural network is almost the same as training it, except that the trained network is presented with the examples it had not seen during the training pr ocess, and no weight adjustments are made during testing. The results of ANN testing can only expl ain how well the ANN performs with the data set used for training and testing. To further evaluate the validity of the ANN, a separate data set independent of these used fo r training and testing is used. This is called the validation data set. Validation adds an other layer of quality control to the ANN model. PAGE 84 73 4.4.1 Model Architecture Design Selection of the ANN architecture is not a clearcut decisionmaking process. Most of the time, trial and error combined with engineering judgment ar e jointly employed to determine the appropriate architecture for a par ticular problem. In this study, a threelayer ANN was adopted. Similar to the traditional models, variables entered in the output layer represent the dependent variab les, and variables entered in the input layer represent independent variables. Weights between layers represent the parameters to be estimated. First, dependent variables in the output layer are decided accordi ng to the objective of modeling. Then a statistical analysis is usually employed to identify these variables highly related to the dependent variables. A trial and error pr ocedure is often followed to identify the input combination that produces the minimum training and testing error. To determine the optimum number of neurons in the hi dden layer, a trial and error procedure is employed due to the still va gue understanding of the effects of the variation of network structures on the network performance. In practice, a sequentia l numbers of hidden neurons are tried, and the one that produces the minimum average or rootmeansquare test error is often chosen. As a comparative study, these explanatory variables identified in the logistic model were entered into th e input layer of the ANN model used in this study. Interaction terms were eliminated since the effects of the interactions are expected to be captured in the c onnection weights during networ k training. The average and rootmeansquare training and testing errors are plotted against th e number of hidden neurons as shown in Figure 4.11 and 4.12, respectively. As it can be seen, the architecture with 8 hidden ne urons produced the smallest tr aining and testing errors. In PAGE 85 addition, the architecture with 13 hidden neurons also produced comparable small training and testing errors. Figure 4.11 Training Errors of Different Number of Hidden Neurons Figure 4.12 Testing Errors of Different Number of Hidden Neurons 74 PAGE 86 75 According to the guidelines provided by Br ainmaker users manual, the shape of the connection weight histograms indicates if the number of hidden neurons is appropriate. The horizontal axis of the hi stogram graph represents the values of connection weights; the vertical axis represents the number of weights. Prior to training, the connection weights were initialized with small random values representing the nave brains. The histogram of weights at the initial point usually l ooks like a steep bell shape, with all weights clustered around the center zero point. As training progresses, the weights are adjusted according to the learning rules, resulting in more and more weights with larger values, which are reflected in the histogram as a flattingout trend of bell shapes. Therefore, the histogram is a percep tive way to examine the stage of the learning process of a neural network. Usually, the following rules of thumb can be used to determine whether a neural network reach es its optimum learning power or not. If, at the end of training, the histograms are still bell curve shaped, which means that the network is healthy and still has the capacity to learn, the number of hidden neurons can be reduced, which may improve the network's predictive powers. If histograms are relatively flat, the number of hidden neurons is probably close to the optimum number. However, if the histogra ms are bunched up at th e left and/or right side of the graph, with a few near the mi ddle, the network is pr obably braindead, and will never learn. Hence more hidden neurons may need to be added to increase the learning power of the network. PAGE 87 Figure 4.13 Connection Weights Histogram (8HiddenNeuron Network) Figure 4.14 Connection Weight Histogram (13HiddenNeuron Network) 76 PAGE 88 77 Figure 4.13 and 4.14 show the connecti on weight histogram of two trained threelayer network with 8 hidden neurons and 13 hidden neurons, respectively. The flattingout shape histogram of the 8hiddenneuron network indicates that the network reaches an optimum learning power. The bell shape histogram of the 13hiddenneuron network indicates that the network still has power to learn and it is possible to reduce hidden neurons to improve netw orks predictive capability. As illustrated in Figure 4.13 and 4.14, the architecture with 8 hidden neurons produced the structure with smallest training and tes ting error. Although 13 hidden neurons also produce comparably small error, the structure with 8 hidden neurons is finally selected in light of the greater gene ralization power associated with fewer hidden neurons. The final proposed ANN archit ecture is illustrated in Figure 4.15. PAGE 89 Input Layer Hidden Layer Output Layer (6 neurons) (8 neurons) (1 neuron) CI(t) Age . Log(ESAL) . CI(t+1) 2 n d Cycle 3 r d Cycle 4 t h Cycle Figure 4.15 Architecture of Crack Forecasting Model (Flexible Pavements) 78 PAGE 90 79 As results of the network training, two we ight matrices were derived as shown in Tables 4.11 and 4.12. The weight matrices repr esent the knowledge abstracted from the example data. Table 4.11 Weight Matrix between Input Layer and Hidden Layer Input Layer ConstCI(t)AgeLog(ESAL)2nd Cycle3rd Cycle4th Cycle 1234567 1 1.52304.12220.24925.70760.10721.20705.3316 2 1.34422.50861.97161.30840.06061.77701.5584 3 2.21740.54421.94140.13920.06320.05463.1552 4 0.20320.26341.92720.05904.71220.22446.7312 5 2.60261.16161.41703.49140.79701.86262.9356 6 4.28821.02744.00720.60600.04641.22341.2072 7 2.07060.06042.04800.50660.13200.00622.5916 8 3.57960.98544.90344.60025.16044.31847.5846Hidden Layer Table 4.12 Weight Matrix between Hidden Layer and Output Layer Hidden Layer Const12345678 1.56061.74265.14100.47901.00941.74422.59240.21501.5180Output PAGE 91 80 4.4.2 Use of the Trained ANN in Forecasting Once the training and testing is successfully completed, the neural network attains the capability of simulating pavement conditio n deterioration mechanism and thereby is able to forecast future pavement conditi ons. Use of the trai ned ANN for forecasting involves a forward propagation process, whic h is similar to that encountered in the training process. To forecast future pavement condition, the inputs are prepared and fed into the input layer of the ne twork; these inputs are then pr opagated forward through the hidden layers, and finally reach the output laye r. The computed network output represents the predicted value of the neural network. For application of the ANN in multipleyear forecasting, the output at one time step are fe d back to the input at the next time step. PAGE 92 CHAPTER 5 MODEL PERFORMANCE EVALUATION Once the model specification is determined, the parameters associated with the explanatory variables are estimated, the model development is considered to be complete. Another critical step prior to the real application of the developed model is to evaluate the performance of the model against a separate data set that is independent of the data used for the model development. For this purpose, the dataset, including the FDOT pavement condition data for year 2003, is utilized. To obtain unbiased evaluations, irrational data that erroneously showed unrealistically improved pavement conditions with time were discarded. Two comparisons were involved in this endeavor. One is between the recurrent Markov chain and the static Markov chain; while the other is between the recurrent Markov chain and the ANN. The comparison are based on the three criteria: average absolute error, rootmeansquare error, and goodness of fit measure (R 2 ). The measurements of the three criteria are defined as follows: The average absolute error is computed using Eq.5.1. Average absolute error = nponiii1 (5.1) 81 PAGE 93 where, n = number of observations, o i = observed value of observation i, and p i = predicted value of observation i. RMSE is computed using Eq.5.2. RMSE = nponiii12)( (5.2) where, RMSE = root mean square error, n = number of observations, o i = observed value of observation i, and p i = predicted value of observation i. The goodness of fit measure, R 2 is calculated using Eq.5.3. ])(/)([1222avgactpredactCICICICIR (5.3) where, actCI = actual value of CI; = model predicted value of CI; and predCI = average actual value of CI. avgCI 82 PAGE 94 5.1 Comparison between the Recurrent Markov Chain and the Static Markov Chain To show the benefits of the recurrent Markov chain versus a static Markov chain, a homogenous transition probability matrix was developed and applied in a Markov chain process for prediction of the pavement crack condition deterioration over time. The transition probabilities were derived from crack condition statistics of the FDOT pavement condition survey database. More specifically, these probabilities were calculated based on the timebased distribution of the frequencies of pavement sections in each condition state. The obtained transition probability matrix is shown in Table 5.1. Table 5.1 Static Transition Probability Matrix State10987654321100.90120.098890.67970.320380.58330.416770.64240.357660.52730.472750.66670.333340.82500.175030.74580.254220.66670.333311.0000 For comparison, crack condition of the pavement in 2003 was forecasted using both the recurrent Markov chain and the static Markov chain. Forecasting errors were computed and compared in terms of absolute average error and rootmeansquare (RMS) error across crack condition states. The results are summarized in Table 5.2. 83 PAGE 95 Table 5.2 Comparison of Forecasting Errors of the Static Markov Chain and the Recurrent Markov Chain ConditionStatic Markov ChainRecurrent Markov ChainStateAverage ErrorRMS ErrorAverage ErrorRMS Error100.66140.68500.10210.126590.78510.80930.21010.228280.66450.70980.22620.246470.71560.75760.26710.294760.77050.80950.30030.328250.46140.49390.20130.241740.36810.40830.22200.263830.81290.81290.35850.434320.75370.77160.15870.173310.50000.50000.19160.2603Total0.67150.70440.15660.1948 As expected, the recurrent Markov chain produced more accurate forecasts than those of the static Markov chain. Therefore, linking the transition probabilities to explanatory variables associated with the pavement crack condition deterioration provides a sensible, adaptive, and more accurate means to estimate those transition probabilities than the simple frequencybased approach. 5.2 Comparison between the Recurrent Markov Chain and the ANN 5.2.1 Comparison of Forecasting Errors The pavement crack condition data in year 2003 were not used in the model development and used only for verification purposes. To assess the performance of the recurrent Markov chain versus the ANN, both models were applied for forecasting pavement crack conditions in 2003. To test multipleyear forecasting capability of the models, pavement crack condition in 2003 were forecasted using data from years 2002, 2001, 2000, 1999, and 1998 in one year, two year, three year, four year, and five year 84 PAGE 96 forecasting, respectively. It can be seen that the recurrent Markov chain is more accurate than the ANN in terms of average absolute error and the rootmeansquare error (RMSE), and it is as expected that the forecasting errors increase as the forecasting period become longer. Table 5.3 Comparison of Forecasting Errors of the Recurrent Markov chain and the ANN ForecastingAverage ErrorRMS ErrorPeriodRMCANNRMCANN1 year0.28900.53910.35660.60832 year0.42971.07080.51571.19143 year0.57441.64960.93291.92244 year0.78112.31051.45032.68435 year1.35992.71572.55523.0654 5.2.2 Goodness of Fit Goodness of fit is a commonly used approach for evaluating performance of models. In this evaluation, crack conditions forecasted for 2003 were plotted against the field observed conditions. The coefficient of determination was calculated using Eq.5.3, which assumes the regression line to be y = x (predicted = observed). In this evaluation, the correlation plot serves as a perceptive qualitative control over the fittingness of the models to the observed crack conditions. The coefficient of determination serves as a quantitative measure of the fittingness of the models to the observed crack conditions. The model performance was evaluated by comparing the goodness of fit of the recurrent Markov chain and the ANN. As an illustration, oneyear forecasts by both the recurrent Markov chain and the ANN are plotted against the observed crack conditions. As shown in Figure 5.1 and 5.2, the recurrent Markov chain produces higher R 2 than the 85 PAGE 97 86 ANN. The computed R 2 values based on Eq.5.3 are 0.95 and 0.86 for the recurrent Markov chain and the ANN, respectively. In add ition, the shapes of the plots reveal that for the recurrent Markov chain model the representative data points are more evenly distributed around the regression line. In contrast to the recurrent Markov chain, an identifiable Sshape trend is shown by the representative data points of the ANN. The Sshape data trend indicates that the ANN te nds to underpredict th e conditions of those pavements in a good condition, but overpredict the conditions of those pavements in a poor condition. PAGE 98 012345678910012345678910Observed CIPredicted CI Rsquare = 0.95 Figure 5.1 Goodness of Fit the Recurrent Markov Chain 012345678910012345678910Observed CIPredicted CI Rsquare = 0.86 Figure 5.2 Goodness of Fit the ANN 87 PAGE 99 5.3 Case Study of a Typical Individual Section A typical section was selected and used for comparing longterm forecasting performance of the recurrent Markov chain and the ANN. The crack conditions forecasted by the two models on an annual basis from one year to 18 years are plotted together with the observed crack conditions. As shown in Figure 5.3, the recurrent Markov chain tends to follow the pavement deterioration trend more closely than the ANN. The observed slow deterioration during the initial stages of new pavements can be better modeled by the recurrent Markov chain than by the ANN. Concurrent with the findings of the goodnessoffit evaluation discussed previously, the ANN tends to underpredict the crack conditions of the pavements in a good condition, and overpredict the crack conditions of the pavements in a poor condition. 66.577.588.599.510123456789101112131415161718AgeCrack Index (CI) Observed Recurrent Markov Chain ANN Figure 5.3 Comparison of the Longterm Performance of the Recurrent Markov Chain and the ANN 88 PAGE 100 89 CHAPTER 6 SUMMARY, CONCLUSIONS AND RECOMMENDATIONS 6.1 Summary This dissertation documents the rese arch that was conducted to develop appropriate pavement crack performance m odels based on recurrent Markov chains and Artificial Neural Networks (ANN). Pavement pe rformance models play a crucial role in a pavement management system (PMS) at th e network level where forecasting results provide key information for highway ag encies in making decisions on overall maintenance and budget planning. Therefore, improved accuracy of pavement performance models could make a consid erable difference in the expenditure on pavement maintenance and rehabilitation. Although many highway agencies still use regression models in their PMS, a noticeable tr end can be observed in attempts to achieve higher forecasting accuracy using more advan ced and innovative modeling techniques. Pavement performance models can generally be categor ized as either deterministic or probabilistic. Deterministic modeling assumes that the pavement behavior follows a predetermined pattern that can be formulated by a specific mathematical equation relating the considered pavement performance indicator to one or more explanatory variables. Historically, the deterministic models have been adopted by many highway agencies in their PMSs. The deterministic models are straightforward, PAGE 101 90 easy to understand and implement. However, theoretically, the deterministic models generally oversimplify the problem since the uncertainty observed in pavement performance is unaccounted for. The pavement deterioration is widely known to be a complex phenomenon characterized by an array of variables associated with it. The underlying mechanisms are still vaguely underst ood. Therefore, an inherent outcome of the complexity required to account for all possible variables pertaining to pavement deterioration is uncertainty. In summary, it would be difficult to successfully model pavement performance in a deterministic wa y unless all the variab les pertaining to the pavement deterioration are clearly defi ned and appropriately accounted for. In response to the above need, the proba bilistic models have emerged as an alternative to the deterministic models. In contrast to deterministic models, the probabilistic models treat pavement condition as a random variable and hence they are capable of accounting for the unc ertainty associated with the pavement deterioration. One of the most popular probabilistic models is the Markov chain. As a stochastic process, Markov chain has been extensively applied in modeling the physical phenomena plagued with uncertainty. Due to its advantages, such as conceptual conciseness, stochastic nature, ease of implementation, etc., the Markov chain has been adopted by many highway agencies in their PMSs as well. The major defect encountered in modeling using Markov chains is the difficulty in obtaini ng rational condition transition probabilities. In the initial stage of PMS, when pavement condition data is scarce, expert knowledge is often consulted to estimate the c ondition transition probabilities. It is this subjective nature of transi tion probabilities that has limited Markov chains from widespread application. Various statistical methods have been attempted to estimate the PAGE 102 91 condition transition probabilities by agencies which benefit from established extensive pavement condition databases. In contrast, in this study, a logistic model was developed to link the transition probabilities to a set of explanatory variable s. As a result, a recurrent Markov chain was constructed in such a way that the logistic model can be dynamically integrated into the Markov chain. As an adaptive process, the recurrent Markov chain is able to realize the true dyna mics not only in the estimation of these transition probabilities but also in the application of them for realistic forecasting. It has been shown that the new recurrent Markov chain overperforms the traditional static Markov chain in term of forecasting accuracy. As the computer industry advances, the computing speed would not be a major concern for extensive computation. This a llows more sophisticat ed algorithm to be implemented with ease for modeling purposes. An artificial neural network (ANN) is one of these. ANN represents typical applications of parallel computation technique inspired by the understanding of the functioning of hu man brain. As a computation intensive method, the artificial ne ural network is di fficult to be categorized into either deterministic or probabilistic models although the computation mechanism makes it more like a deterministic model because the weight ma trices derived from the network training simulate the parameters estimated in the tradit ional deterministic model. As part of this study, a Backpropagation neur al network was developed. The performance of the developed neural network was compared with that of the recurrent Markov chain. The comparison of forecasts by both models leads to a better understanding of the mechanisms underlying the two distinct methodologies. The artificial neural network tends to overestimate the paveme nt condition deterioration in PAGE 103 92 the initial stages of pavement life, but underestimate the pavement condition deterioration in the latter stages of pavement life. On the othe r hand, the recurrent Markov chain produces more consistent foreca sts of crack conditions. In addition, the higher goodness of fit (Rsquare = 0.95) was obtained from the recurrent Markov chain compared to the ANN (Rsquare = 0.86). 6.2 Conclusions The recurrent Markov chain is consider ed a theoretically appropriate model because the model formulation satisfies th e Markov property of limited historical dependency and its characteristics coincide with the very nature of the uncertainty associated with the pavement deterioration process. In addition, the model is also deemed practically feasible since it made use of various explanatory variables in the estimation of transition probabilities. The model is also constructed in a way that allows for the realization of the dynamics in these transition probabilities. Compared with the recurrent Markov chain, the ANN does not require a function form to be specified. ANN is often viewed as a black box function. Therefore, it is hard to evaluate the effect of the input variables and the impact of the input variables on the output. Due to its generality of the modeling structure, the model performance is highly dependent on the data used for training. Hen ce, more strict data processing is usually required for successful training. In addition, the training process can be timeconsuming, and intervention may be necessary for adjustment of parameters, such as the learning rate and momentum, during training based on empirical judgment. PAGE 104 93 6.3 Recommendations Data processing plays an important role in any modeling effort. Although the model structure may be theore tically sound, the model estima tion can only be as good as the quality of the data being used. Ther efore, it is recommended that the pavement condition survey procedure should be as uniform and consistent as possible over time and the annual survey data need to be carefully examined for the irregu larities before the PMS database is updated. Timely updates of the model parameters us ing newly collected data are necessary in order to capture the deterioration pattern revealed in the u pdated data set. This can be accomplished by reestimating the model para meters or retraining the network with newly available data. The met hodologies as documented in this research are quite general in themselves. They could be used for modeling the performance of other pavement distresses, such as ride, rut, etc. The ANN model used in this research as a comparison to recurrent Markov chain is a feedforward threelayer Backpropagation neural network. It may not be appropriate to be used in a recursive manner for multip leyear forecasting although it is trained with time series of multipleyear crack data. For recursive modeling, a recurrent neural network may be more suitable th an a traditional BP network. Although multiplestate transition probabiliti es can be derived from the twostate transition probabilities, it is highly r ecommended that multiplestate transition probabilities should only be used when this trend is supported by the data. PAGE 105 94 REFERENCES 1. Adi Andrei, Dragos Andrei and Michael Aceves, Conception and Development of an Evolutionary Algorithm for Predicting Road Distress. Computational Intelligence Applications in Pavement and Geomechanical Systems 2000, pp147151. 2. American Association of State Hi ghway and Transportation Officials (AASHTO1993), AASHTO Guide for Design of Pavement Washington D. C., 1993. 3. A.R.Shekharan, Pavement Performance Prediction by Artificial Neural Networks. Computational Intelligence A pplications in Pavement and Geomechanical Systems 2000, pp8998. 4. Asphalt Institute (AI) (1982), Research and Development of the Asphalt Institutes Thickness Design Manual (MS1) 9 th Edition, 1982. 5. AttohOkine, N. O., Predicting Roughne ss Progression in Flexible Pavements Using Artificial Neural Networks. Third International Conference on Managing Pavements Conference Proceedings Vol. 1, 1994, pp.5562. 6. Banan, M. R. and Huelmstad, K. D., Neural Networks and AASHO Road Test, Journal of Transportation Engineering Sep. 1996, pp358366. 7. Butt, A. A., Feighan, K. J., Shahin. M. Y., and Carpenter, S. Pavement performance prediction model using the Markov Process. Transportation Research Record 1123 Transportation Research Board, Washington, D.C., 1219, 1987. PAGE 106 95 8. Carey, W. N. and P.E. Irick Pavemen t ServiceabilityPerformance Concept. HRB, Bulletin 250, 1960, pp. 4058 9. Carnahan, J. V., Davis, W. J., and Shahin M. Y. Optimal maintenance decisions for pavement management. Journal of Transportation Engineering, ASCE, 113(5), pp. 554572, 1987. 10. Chen, D.H., Zaman, M., and laguros, J. G., Assessment of Distress Models for Prediction of Pavement Service Life, the 3 rd Material Engineering Conference 1994, pp10731080. 11. Eldin, N. N. and Senouol, A. B., A Pavement ConditionRating Model Using Backpropagation Neural Networks, Microcomputers in Civil Engineering 10, 1995, pp.433441. 12. Emanuel Parzen, Stochastic Processes, HoldenDay, Inc. 1962. 13. Florida Department of Transportati on, Pavement Condition Survey Handbook, April 1994. 14. GarciaDiaz, A, and Riggins, M., Ser viceability and Distress Methodology for Predicting Pavement Performance. Transportation Research Record 997 pp1723, 1984. 15. Ghassan AbuLebdeh, Rick Lyles, Gilbert Baladi, and Kamran Ahmed, Development of Alternative Pavement Di stress Index Models, Research Report, Department of Civil & Environmental En gineering, Michigan State University, November. 2003. 16. Hass, R. and Hudson, W. R., Pavement Management Systems McGrawHill, Inc., 1978. 17. Hass, R., Hudson, W. R., and Zaniewski, J., Modern Pavement Management Krieger publishing Company, Malabar, FL, 1994. PAGE 107 96 18. Haykin, S., Neural Networks A Comprehensive Foundation. Macmillan College Publishing Company, New York, 1994. 19. Jiang, Y., Saito, M., and Sinha, K. C., Bridge performance prediction model using the Markov chain. Transportation Research Record 1180 Transportation Research Board, Washington, D.C., 2532, 1988. 20. J. Yang, J. J. Lu, M. Gunaratne, and Qiaojun Xiang, Forecasting Overall Pavement Condition with Neural Netw orks, Application on Florida Highway Network. Transportation Research Record 1853, Transportation Research Board, Washington, D.C., 312, 2003. 21. J. Yang, J. J. Lu, M. Gunaratne, App lication of Neural Network Models for Forecasting of Pavement Crack index and Pavement Condition Rating, Final Report for Florida State Department of Transportation, Research Study BC353, March 2003. 22. K Funahashi, On the Approximate R ealization of Con tinuous Mappings by Neural Networks. Neural Networks Vol. 2, No. 3, pp. 183192, 1989. 23. K. Hornik, M. Stinchcombe, and H. Wh ite, Multilayer Feedforward Networks Are Universal Approximators. Neural Networks, Vol. 2, No. 5, pp. 359366, 1989. 24. Lawrence, J. and Fredricson, J., BrainMaker Users Guide and Reference Manual 7 th Edition, California Scientific Software, Nevada City, CA, 1993. 25. Madanat, S., and Wan Ibrahim, W. H., Poisson and negative binomial regression models for the computation of infr astructure transition probabilities, Journal of Transportation Engineering, ASCE 121(3), pp. 267272, 1995. 26. Madanat, S., Karlaftis, M., and McCarthy, P., Probabilistic infrastructure deterioration models with panel data, Journal of Infrastructure Systems, ASCE, 3(1), pp.49, 1997. PAGE 108 97 27. Madanat, S., Mishalani, R., and Wa n Ibrahim, W. H., Estimation of infrastructure transition probabi lities from condition rating data, Journal of Infrastructure Systems, ASCE 1(2), pp. 120125, 1995. 28. McFadden, D., Conditional Logit Analysis of Qualitative Choice Behavior. In Frontiers in Econometrics, ed. P. Zare mbka. Academic Press, New York, 1973. 29. McFadden D., Econometric Models of Pr obabilistic Choice. In Structural Analysis of Discrete Data, eds. C. F. Manski and D. McFadden. The MIT press, Cambridge MA. 30. Mei, X., J. J. Lu, Evaluation of Tec hniques and Methodologies Applicable for Automatic Detection of Pavement Crack Depth on Florida Roadways. Interim Report for Florida State Depart ment of Transportation, 1999. 31. Nunez, M.M. and Shahin, M.Y., Pavement Condition Data Analysis and Modeling. Transportation Research Record 1070 pp125132, 1986. 32. Paterson, W.D.O., Road Deterioration a nd Maintenance Effects: Models for Planning and Management. Baltimore, John Hopkins University Press, 1987. 33. Project Traffic Forecasting Handbook, October 2002. 34. Queiroz, C. A Mechanistic Analysis of Asphalt Pavement Performance in Brazil. Journal of Association of Asphalt Paving Technology 52, pp474488, 1983. 35. Ronald P. Cody and Jeffrey K. Smith, Applied Statistics and the SAS Programming Language, Fourth Edition, PrenticeHall, Inc. 1997. 36. Samuel T. Ariaratnam, Ashraf ElA ssaly, and Yuqing Yang, Assessment of Infrastructure Inspection Needs Using Logistic Models, Journal of Infrastructure Systems, ASCE 7(4), pp.160165, 2001. PAGE 109 98 37. Shahin, M. Y., Pavement Management fo r Airports, Roads, and Parking Lots, Chapman & Hall, New York, 1994. 38. S.OwusuAbabio, Application of Neural Networks to Modeling Thick Asphalt Pavement Performance. Artificial Intelligen ce and Mathematical and Mathematical Methods in Pa vement and Geomechanical Systems 1998, pp2330. 39. Z. Lou, J. J. Lu, M. Gunaratne, Roa d Surface Crack Condition Forecasting Using Neural Network Models, Final Repor t for Florida State Department of Transportation, Research Study BB275, October 1999. 40. Z. Lou, M.Gunaratne, J. J. Lu, and B. Dietrich, Application of Neural Network Model to Forecast Shortterm Pavement Crack Condition: Florida Case Study. Journal of Infrastructure Systems, ASCE 7(4), pp.166171, 2001. PAGE 110 ABOUT THE AUTHOR Jidong Yang received his Bachelors Degr ee in Civil Engineer ing in 1996 from Hebei Agricultural University, China. Before joining the University of South Florida, he was a graduate student in the Department of Civil Engineering in Tianjin University, China, where his research interest con centrated on the structure vibration and earthquakeresistant theory. Mr. Yang entered the Department of Ci vil and Environmental Engineering in University of South Florida (USF) as a rese arch assistant in January 2000. During his stay in USF, he extended his research ar ea to pavement condition performance modeling and pavement management system applicati on. He involved in a research project titled Application of Neural Network Models fo r Forecasting of Pavement Crack Index and Pavement Condition Rating, sponsored by Fl orida Department of Transportation. The research findings and results were summarized in a technical paper, which was presented on the 2003 Transportation Research Board (T RB) annual meeting and published in the Transportation Research Record. 