|
|
Also available in PDF
To view PDF files, you need the Acrobat® Reader®.
Feature: Testing Programs for Transportation Management Systems
Members Discuss Challenges |
The Transportation Management Center (TMC) Pooled Fund Study is a forum for participating members to identify and address issues that are common among transportation management centers. The TMC Pooled Fund Study members get together at annual meetings and via conference calls to share the insight of their experience, problem solving techniques, and lessons learned. This active discussion and information sharing has further expanded to a peer-to-peer discussion forum allowing members to ask questions and gather responses, documents and examples from others. This e-mail-based discussion forum has proven an effective and efficient means for information exchange and experience sharing.
“Many times I get a quick turn around request for information on what other states are doing or how we can address this challenge, as a member of the TMC Pooled Fund Study, I can send out an e-mail and get 10 to 15 responses from different states in less than a couple of hours on how they have addressed that same challenge and more importantly what has worked and what has not worked.” -- Mike Jenkinson, Caltrans
The list below highlights some topics that have been discussed by the members:
“The products and the peer network produced from the TMC Pooled Fund Study have been very helpful for Wisconsin's growth in traffic operation and management. There is much to learn from the National Experts that are a part of this group.” -- Doug Dembowski, Wisconsin DOT
A complete compilation of questions and responses on the TMC PFS peer discussion forum is available by logging on to the members-only area of the TMC PFS website at http://tmcpfs.ops.fhwa.dot.gov/members/login.cfm.
Transportation Management Systems (TMS) are composed of a complex, integrated blend of hardware, software, processes, and people performing a range of functions. These functions typically include data acquisition, command and control, data processing and analysis, and communications.
Testing is an important part of the deployment of a TMS. Testing describes a series of processes and procedures developed in the information technology community to verify and validate system performance. While some of the terms used in testing may be unfamiliar to transportation professionals, the core concepts and processes are not technically complex; rather, they represent sound practices in developing and maintaining any system.
 |
The purpose of testing is two-fold. First, testing verifies what was specified and what was delivered: it verifies that the product (system) meets the functional, performance, design, and implementation requirements identified in the procurement specifications. A good testing program requires well-written requirements for both the components and the overall system. Without testable requirements, there is no basis for a test program.
Second, testing manages risks for both the acquiring agency and the system's vendor/developer/ integrator. The test program that evolves from the overarching systems engineering process, if properly structured and administered, facilitates the process of managing the programmatic and technical risks and helps to assure the success of the project. The testing program is used to identify the point at which the work has been "completed" so that the contract can be closed, the vendor paid, and the system shifted into the warranty and maintenance phase of the project. Incremental testing is often tied to intermediate milestones and allows the agency to start using the system for the benefit of the public.
Many important decisions are made early in the system engineering process that affect the testing program. The attention to detail when writing and reviewing the requirements or developing the plans and budgets for testing will see the greatest pay-off (or problems) as the project nears completion. A good testing program is a tool for both the agency and the integrator/supplier; it typically identifies the end of the “development” phase of the project, establishes the criteria for project acceptance, and establishes the start of the warranty period.
Testing is the practice of making objective judgments regarding the extent to which the system (device) meets, exceeds or fails to meet stated objectives. The complexity and ultimate cost of the system test program is directly related to the test method(s) specified for verification of each requirement. As the test method becomes more rigorous, performing that testing becomes more time consuming, requires more resources and expertise, and is generally more expensive. However, the risk of accepting a requirement under a less rigorous testing method may be undesirable and may ultimately prove to be more expensive than a more rigorous test method. There are five basic verification methods:
 |
It is important to carefully consider the tradeoffs and consequences attendant to the specification of the test methods since they flow down with the requirements to the hardware and software specifications and finally to the procurement specification. From a verification standpoint, inspection is the least rigorous method, followed by certificate of compliance, analysis, demonstration, and then test (formal) as the most rigorous method. A vendor’s certificate of compliance may be evidence of a very rigorous development and test program, but that testing is typically not defined, approved, or witnessed by the system’s acquiring agency; therefore, there is some risk in accepting that certification. That risk is often outweighed, however, by the costs of performing equivalent testing by the acquiring agency.
The Testing Programs for Transportation Management Systems Handbook provides an introductory guide to transportation management system (TMS) testing. It begins by discussing testing within the system engineering life-cycle process and stages in Chapter 2. This discussion introduces the system engineering process and identifies the sources of requirements that are the basis for testing and ultimate system acceptance. Chapter 3 provides an overview of the TMS acquisition process starting with the development of a regional architecture for the TMS. It then discusses system procurement considerations and practices with emphasis on how the test program affects various phases of the system's life cycle, including post-acceptance operations and maintenance. The material on the testing role in the system engineering process and the project life cycle is necessary to set the stage for testing and to put it into context.
Chapter 4 addresses the basics of testing. It discusses testing methods, planning, test development, resources, and execution. This chapter folds the technical discipline into a compact presentation. It is not intended to make readers an expert but introduces readers to the terminology and many of the concepts that will be used by the system vendors. Chapter 5 focuses on planning a project test program. The broad-based technical discussion of testing fundamentals provides material that a TMS project manager must be familiar with to implement the project's testing program and fit it into the overall TMS deployment program. The success of the project will depend on the test program that evolves from this system engineering process and, when properly structured and administered, the test program allows the programmatic and technical risks to be managed. Understanding the role of testing, its terminology, and technical issues is critical to managing a test program for a TMS deployment project. This material also allows the project manager to discuss the program in detail with the technical experts that may be employed to assist with development and implementation.
Contract Types, Agency Risk Allocation and Test Program Burden |
Chapters 6 and 7 discuss hardware and software testing respectively and provide real-world examples. Chapter 8 addresses testing at the subsystem and system levels. Chapter 9 provides guidance and recommendations on such topics as the meaning of "shall" and "will" in requirements statements, how to write testable requirements, and pass/fail criteria. This chapter also includes helpful information and suggestions on test reporting, testing timeframes, testing organization independence, testing relevancy and challenges, failure modes and effects, testing myths, and estimating test costs. Chapters 6-9 are much more pragmatic discussions of testing, presenting lessons learned and providing practical guidance for planning, developing, and conducting testing.
Chapter 10 provides a list of available resources for further investigation. The handbook includes four appendices: Appendix A - Example Verification Cross Reference Matrix, Appendix B - Sample Test Procedure, Appendix C - Sample System Problem/Change Request Form, and Appendix D - Example Application of NTCIP Standards.
This handbook will benefit individuals that are responsible for or involved in the planning, design, implementation, operation, maintenance, and evaluation of the TMS for public agencies. Targeted end users of the handbook are first-level supervisors (managers and supervisors) and technical staff that may include transportation planners, traffic engineers and technicians, construction and maintenance engineers, and traffic management center (TMC) staff.
For a non-technical audience, an associated TMS Testing Primer identifies key aspects of testing, identifies the benefits of developing and using testing programs, and profiles successful practices of existing programs. The Handbook and primer are available at the FHWA Office of Operations website at: http://ops.fhwa.dot.gov/int_its_deployment/standards_imp/tptms.htm.
Ongoing TMC Pooled Fund Study projects are briefly described in the following paragraphs. A complete quarterly project progress report can be accessed on the TMC Pooled Fund Study Website: http://tmcpfs.ops.fhwa.dot.gov.
“Developing and Using Concept of Operations in Transportation Management Systems”
Purpose: Develop a document that describes the need for a concept of operations for a transportation management system and provides technical guidance and recommended practices for developing and using a concept of operations throughout the system’s life cycle.
“Transportation Management Center Business Planning and Plans Handbook”
Purpose: Produce a handbook that provides guidance and best practices on how to develop a TMC business plan. The handbook will also outline business-planning models that were successfully employed by transportation agencies to ensure the long-term sustainability of TMCs and associated ITS applications.
“TMC Performance Monitoring, Evaluation, and Reporting Handbook”
Purpose: Develop a handbook that explains the need for performance monitoring and provides guidance and recommended monitoring practices. The handbook will advise how to initiate, sustain, and use information generated from monitoring, evaluating, and reporting on TMC performance and describe roles, responsibilities, functions, and support services as they relate to traffic management.
“Regional, Statewide, and Multi-State TMC Concept of Operations and Requirements”
Purpose: Building off the existing Developing and Using Concept of Operations in Transportation Management Systems Handbook, this project will develop a document that will provide detailed guidance on how to develop and use concept of operations and system requirements as it applies to the life cycle of a regional, statewide, or multi-state TMC.
“Recovery and Mitigation of TMCs”
Purpose: Develop a technical document that synthesizes current practices and state of the practices, highlight technical issues, lessons learned, and recommended practices, and detail how to plan, develop and implement redundancy design and recover plans for TMCs and transportation management systems.
“Driver Use of Real-Time En-Route Travel Time Information”
Purpose: Assess impacts of en-route real-time travel time/delay/speed information on drivers; define the most effective way to provide en-route real-time travel time information; and develop preliminary guidance to practitioners for delivering en-route travel time information.
“Procuring, Managing, and Evaluating the Performance of Contracted TMC Services”
Purpose: Develop a technical document that will provide guidance and recommended practice to TMC owners and managers in making decisions related to outsourcing portions, or in entirety, of their TMC or transportation management system operation to a private contractor or contractors.
“Integration of TMC and Law Enforcement: Needs Assessment”
Purpose: Assess the current practices and identify issues, needs, and challenges that all involving agencies are facing in integrating TMCs and law enforcement. The results of this effort will lead to identification of a list of topics and issues to be addressed and a series of next steps to be considered in a further study that is intended to develop a product to provide necessary guidance to address agencies’ needs.
“TMC Clearinghouse Support Services, Phase 2”
Purpose: Enhance and improve the support services for the TMC clearinghouse website that will be available online in Spring/Summer 2006. The study will also evaluate consumer feedback and recommendations for enhancing and improving the features and contents of the clearinghouse.
“Methodologies to Measure and Quantify TMC Benefits, Phase 1”
Purpose: Gain a better understanding of and to quantify benefits in traffic operations due to the implementation of TMCs and the systems, infrastructure, and functions associated with their operations. Phase 1 of the project will synthesize methodologies for measuring, quantifying, and evaluating costs and benefits of TMCs, as well as assess the feasibility of developing a software tool for quantifying TMC benefits.
“Developing Travel Time Information”
Purpose: Synthesize the state of the practice, successful stories, and lessons learned as well as develop a technical document that provides guidance and recommended practices on the concepts, methods, techniques, and procedures for TMCs to collect, calculate, and predict travel time information.
“Requirements and Position Descriptions for TMC Support Staff”
Purpose: Build off information already compiled for operators in a previous effort and compile the needed information related to KSA’s for other tasks and services required to support TMCs.
“Techniques for Managing Service Patrol Operations”
Purpose: Identify and synthesize current best practices, state of the practices, and models and innovative techniques for managing service patrol operations.
“Best Practices for Road Condition Reporting Systems”
Purpose: Synthesize current best practices and state of the practices in planning, design, and operation of road condition reporting systems as well as in integrating such systems with other road weather information/management systems.
 |
Jim McGee
Highway Programs Administrator
Nebraska DOR
Mr. McGee is the Highway Programs Administrator of the Operations & Intelligent Transportation Systems for the Nebraska Department of Roads (NDOR). He has worked for the NDOR since 1993. He holds a Master of Public Administration degree and is also an adjunct instructor at the University of Nebraska at Omaha College of Public Affairs and Community Service School of Public Administration.
Mr. McGee has been involved with the Nebraska ITS Program since its inception in 1997 and has been responsible for ITS Planning and funding issues, ITS partnerships, CVISN, the Nebraska Department of Justice AMBER Plan Committee, the Nebraska Homeland Security Planning Team, the Omaha Urban Area Security Initiative, the Nebraska Motorist Assist Patrols, and Traffic Incident Management.
Mr. McGee has actively served on the TMC Pooled Fund Study since 2005. He is also one of the founding members of ITS Heartland, one of the most active ITS multi-state partnerships in the United States. Mr. McGee has also served on several Transportation Research Board panels.
Agencies may join the TMC Pooled Fund Study at anytime during the year by committing funds at a level agreed upon by existing participants in the study. The TMC pooled fund study was approved for 100 percent State Planning and Research Program funding. Any noncommercial agency or organization that is responsible for the management and operation of any portion of the surface transportation system is welcome to participate.
State transportation agencies interested in joining the TMC Pooled Fund Study can submit funding commitment online at the Transportation Pooled Fund Program web site at: http://www.pooledfund.org (see Solicitation No. 870; SPR-2(207)).
Other agencies should complete and submit the TMC Pooled Fund Study commitment form downloadable at the TMC Pooled Fund Study website at: http://tmcpfs.ops.fhwa.dot.gov.
January 13-17, 2008 |
TRB 87th Annual Meeting, Washington, DC |
March 27, 2008 |
TMC Pooled Fund Study Quarterly Conference Call |
March 30 - April 2, 2008 |
ITE Technical Conference and Exhibit, Miami, Florida |
April 7-9, 2008 |
ITS Heartland Annual Meeting, Springfield, Missouri |
June 15-19, 2008 |
2008 Freeway and Tollway Operations Conference, Fort Lauderdale, Florida |
July 15-16, 2008 |
TMC Pooled Fund Study Annual Meeting, Nashville, Tennessee |
August 17-20, 2008 |
ITE 2008 Annual Meeting and Exhibit, Anaheim, California |
September 3-5, 2008 |
National Rural ITS Conference, Anchorage, Alaska |
November 16-20, 2008 |
15th World Congress on ITS & ITS America's 2008 Annual Meeting & Exposition, New York, New York |
“Real-Time Traveler Information Services Business Models: State of the Practice Review” (FHWA-HOP-07-115, May 2007) – This state of the practice review documented a range of business models for real-time traveler information services, and provided ‘real world’ examples of how States and regions are developing partnerships and business plans within business model frameworks. Included with this review is a summary of current prevalent business models, which include public-sector funded, franchise operations, private sector funded and business-to-business models. It addressed issues such as roles and responsibilities within the models, pros and cons of the various approaches, and provided case study examples of traveler information programs throughout the country. Available at: http://www.ops.fhwa.dot.gov/publications/rtis_busmodels/index.htm.
“Unifying Incident Response” (in Public Roads, September/October 2007, FHWA-HRT-07-006) – This article describes the National Unified Goal (NUG), a policy initiative aimed at reducing congestion and increasing safety for users and incident responders developed by the NTIMC (National Traffic Incident Management Coalition) in 2006 in response to USDOT’s Congestion Initiative. The NUG outlines eighteen strategies that promote the development of a multidisciplinary, multi-jurisdictional Traffic Incident Management program with the goals of 1) responder safety, 2) safe, quick clearance, and 3) prompt, reliable, interoperable communications. These strategies include policies states can adopt such as “move over”, “driver removal” and “authority removal” laws that would require motorists to move to an opposite lane of a responder, operable vehicles in incidents to be driven off of traffic lanes and give responders authority to remove material including vehicles and cargo impeding traffic lanes. Available at: http://www.tfhrc.gov/pubrds/07sep/04.htm.
“Tabletop Exercise Instructions for Planned Events and Unplanned Incidents/Emergencies” (FHWA-HOP-08-005, November 2007) – This document serves as a guide for using tabletop exercises to test transportation management plans prior to a planned special event. It describes the need for and purpose of tabletop exercises, provides guidelines for conducting an exercise, identifies key stakeholders and lists potential exercise objectives. Though originally designed for planned special events, the information in the guide can be used for the management of unplanned incidents including traffic incidents and responses to emergencies. Available at: http://ops.fhwa.dot.gov/resources/news/news_detail.asp?ID=395.
“Presentations from I-95 Corridor Coalition's Travel Information Forums” (October 2007) – In a two-day conference held on October 24-25 in Baltimore, Maryland, the I-95 Corridor Coalition hosted concurrent forums called: “From Data to Travel Information – How to Get There” and “Bringing Together Airport and Ground Transportation Information Systems”. The forums were held for surface and air transportation users and providers interested in improving travel information services along the I-95 corridor. The forums provided an interactive way to exchange information on successful strategies in the assimilation of multi-modal travel data and dissemination of information to transportation users. A compilation of the presentations and discussion results can be found at: http://www.i95coalition.org/meeting_tis.html.
“Presentations from the 2007 National Rural ITS Conference” (October 2007) – This four-day event was hosted by ITS Michigan and Michigan DOT on October 7-10 in Traverse City, Michigan. Topics of presentations and roundtable discussions included road weather information, rural communications, rural transit, ITS partnerships and deployment, traveler information and data collection, commercial vehicle operations, border security, U.S. DOT’s 511 Coalition and Clarus Initiative, and VII for non-urban areas. A compilation of the presentations and discussion results can be found at:
http://www.nritsconference.org/Proceedings2007.html.
“2007 National Traffic Signal Report Card” (National Transportation Operations Coalition, October 2007) –The 2007 National Traffic Signal Report Card showed how local jurisdictions such as cities, counties and states can realize significant benefits, such as mitigating congestion and lessening fuel consumption, by making small changes in the way they manage and operate traffic signal systems. The report also highlighted several jurisdictions that have been successful in making changes since the 2005 report card was released. The results contained in the report card are based on the 2007 Traffic Signal Operation Self Assessment released by NTOC in the fall of 2006. A total of 417 agencies responded, representing 47 states. Available at: http://www.ite.org/reportcard/.
“Turbo Architecture 4.0” (October 2007) – Turbo Architecture Version 4.0 that fully supports Version 6.0 of the National ITS Architecture is now available for free download. Version 4.0 includes a Turbo Conversion facility that supports quick and easy conversion of existing Turbo databases, providing a convenient migration path for existing Turbo users. For more information and to download Turbo Architecture, visit http://www.iteris.com/itsarch/html/turbo/turbomain.htm.
Idaho Transportation Department Introduces Improved 511 Web Site – The Idaho Transportation Department (ITD) has improved their Travel Services website. The site now gives the option to choose between low-bandwidth for dial-up users and high-bandwidth for high speed internet users. The low-bandwidth site is similar in design to ITD’s previous 511 website, but features new capabilities such as the ability to print event lists for selected routes. The high-bandwidth site offers a larger, more interactive map and access to camera images. The website is available at: 511.idaho.gov
PennDOT Traveler Information Website and Hotline – The new travel information section on the Pennsylvania DOT’s (PennDOT) website is part of an effort to help prepare motorists this year for winter driving. The site features updated road conditions and closures, weather advisories, and access to PennDOT cameras. PennDOT has also established a hotline where users can call in and get updated road and weather condition information. The travel information section can be found on the PennDOT website at: www.dot.state.pa.us. The toll-free hotline is 1-888-783-6783 for in-state callers and 717-783-5186 for out-of-state callers.
Virginia DOT Opens Fifth and Final Traffic Management Center – On November 20, 2007, the Virginia DOT opened a new Traffic Management Center (TMC) in Salem. This completed the network of five TMCs that manage highway and traffic operations on Virginia highways and freeways. The Salem TMC will manage 34 counties in the southwest region of the state and will employ some of the latest technology including a 511 service and electronic message boards. It will also serve as the dispatch center for the Safety Service Patrol, a traffic incident response unit.
Missouri DOT’s New Traveler Information Map Gives Drivers an Edge on Weather-Related Road Conditions – In November 2007 the Missouri DOT (MoDOT) launched a new online travel information map. The map provides updated road and weather conditions giving users advanced warning of congested or hazardous situations. The map is available on MoDOT’s website: www.modot.org.
Tennessee DOT Launches Phase II of SmartWay Traveler Information System – The Tennessee DOT launched Phase II of the TDOT SmartWay traveler information system. Phase II improvements include 19 new closed circuit traffic cameras, eight dynamic message boards, 80 speed detection devices and the addition of a city-wide Highway Advisory Radio (HAR) system. The new cameras and message boards are located along routes I-440, I-65 and I-24. The HAR features flashing beacons on signs that activate when a traffic incident has caused a lane closure prompting motorists to tune into the station AM 1630. For access to the new cameras and road conditions, users can go to the SmartWay website at: www.tdot.state.tn.us/tdotsmartway/.
The TMC Pooled Fund Study will have a booth at the TRB Annual Meeting that is held on January 13-17, 2008 in Washington, DC. Please stop by our booth to learn the latest about the program, projects, activities, available products, and how to participate in this exciting program.
|
TMCUpdate is a quarterly newsletter produced by the Transportation Management Center (TMC) Pooled Fund Study. This quarterly publication highlights major TMC Pooled Fund Study activities and achievements and shares TMC related news and resources. Reproduction (in whole or in part) and broad distribution of this newsletter is strongly encouraged. The TMC Pooled Fund Study invites inquiries about articles and suggestions for TMC developments and advancements to be covered in future issues. For more information, please contact the Program Administrators, Raj Ghaman at Tel: 202-493-3270, E-mail: raj.ghaman@fhwa.dot.gov; or Tom Granda at Tel: 202-493-3365, E-mail: thomas.granda@fhwa.dot.gov; or the newsletter editor, Ming-Shiun Lee at Tel: 612-373-6335 or E-mail: ming_shiun_lee@urscorp.com. |
 |
