Why the Transportation Industry is Getting on Board with Big Data & Hadoop

Our daily commute may not feel like such a high tech experience, but whether you feel it or not – it is. Big data and Hadoop have revolutionized the transportation industry over the past several years. Whether in a car, a train, a plane or a delivery truck, we all use big data throughout our travels. Let’s go through a few specific use cases to spotlight transportation businesses that are using big data in a big way.


Trains and railways have an old-timey nostalgia attached to them; we typically associate train travel with the simpler times of our grandparents. However, the reality of the railway transportation system is far from slow and simple. Just like many other industries, railroad companies have integrated big data into many different aspects of their operations.

Certain elements of the railway system are predictable. The staff, cars, schedule, etc. are predetermined before a single car is moving. The real data generation magic begins once the trains start moving.

So, where is all this big data coming from? Here are just a few of the many data sources utilized today:

  • Maintenance logs
  • GPS units combined with weather data can be used to ensure train safety
  • Handheld field tablets
  • GPS units that record speed, distance between trains, arrival time and location
  • Visual and acoustic sensors in brakes, rails, switches and other hardware

All of these data sources provide rich analytics that can quickly influence both automated and human decision-making.

As an example of railway automation, one of the nation’s largest railroads just invested in a fully automated rescheduling system. This big data system manages the rescheduling of over 8,000 trains. No matter what unexpected scenario comes up, these 8,000 trains are now able to be on time across 23 states. Now that’s big business value.

Traffic Management

Everybody hates traffic, especially those in cities who are trying to attract a talented population to their productive infrastructure. This is why many cities are starting to use big data to curb the traffic monster.

In one New Jersey town, 22-foot tall sensor screens pick up millions of daily cellphone and GPS signals from commuters passing by. This data translates the following to road operators:

  • Car speeds
  • Sources of acceleration and deceleration
  • Weather conditions
  • Community events

The data is then cross referenced with any other data they have on road conditions from sensors and other sources. All of this data is woven together to form a live-data traffic map of over 2,600 miles of New Jersey roads.

This map enables road technicians and monitors to identify traffic problems quickly. For example, one day the digital map alerted officials of a point along I-80 that was inhibiting the afternoon commute. Within 30 minutes, technicians had a service vehicle on site and the overturned car removed. Prior to the new sensor data, commuters would have been waiting hours for the issue to be resolved. Between traffic cameras and a scattered network of roadside sensors, the ability for officials to efficiently detect and resolve roadside problems before the live-data traffic map was insufficient at best.

Delivery and Trucking Companies

Delivery and trucking companies are a critical piece to our modern society. In order to keep up with the high expectations of their customers, they too have implemented big data technologies.

One of the ways big data is saving trucking companies big money is with fuel consumption. In some cases, mathematical models are used to optimize shipping routes. By honing in on excessive driving routes, drivers can see a reduction of nearly one mile of driving every day. Now, to most of us a mile doesn’t seem like much. For a company like UPS, a reduction of one mile per day per driver would equal a savings of as much as $50 million a year in fuel.

Another way trucking companies use data to save money on fuel is by using predictive modeling to select fuel efficient trucks. One company was depending on this data to help them make the right choice in selecting a new fleet of 50 trucks – a $6 million decision.

The predictive model used to determine the actual fuel economy of the trucks analyzed much more than standard metrics. They combined data variables like driving behavior, fuel tank levels, load weight, road conditions and much more. The detail of the data provided executives with a clear picture of which trucks would provide the most fuel savings over time.

Road Repair

Local infrastructure and road repair is difficult to stay on top of. Many cities feel like they are always behind in terms of tackling serious repair needs. Even more frustrated are their commuting constituents, who are on the receiving end of vehicle damages due to road deterioration.

Boston is just one of many cities who have turned to big data to address this specific problem. As part of their solution, they ventured into the development of a big data app. Residents who want to improve the local infrastructure use this app, known as Street Bump, on their cell phones. The app uses signal data from a cell phone’s accelerometer to detect jolts. If other phones with the app report jolts from the same location, a pothole in need of repair has been identified. In the past, cities had to depend on actual road surveyors and engineers to identify roads that needed repair. This typically cost the city of Boston $200,000 every year. Comparatively, the Speed Bump app’s development cost a mere $80,000. In the end, Speed Bump is providing the city substantial cost savings in the evaluation of their roads with an added benefit of greater accuracy and efficiency.

An efficient transportation system is a key factor in any thriving economy. Big data has helped transportation companies stay on track through increased operational efficiency, improved customer experiences, reduced fuel costs/increased profits, and enhanced service offerings.

If you’re interested in learning more about how big data and Hadoop can help you, download The Executive’s Guide to Big Data and Apache Hadoop.


Streaming Data Architecture:

New Designs Using Apache Kafka and MapR Streams




Download for free