Big Data

Logistics and SCM are being influenced by big data technologies.

What is Big Data

Big data is an all-encompassing term for any collection of data sets so large and complex that it becomes difficult to process using on-hand data management tools or traditional data processing applications.

Industry analyst Doug Laney articulated the now-mainstream definition of big data as the three Vs:

• Organizations collect data from a variety of sources, including business transactions, social media and information from sensor or machine-to-machine data. In the past, storing it would’ve been a problem – but new technologies (such as Hadoop) have eased the burden.
• Data streams in at an unprecedented speed and must be dealt with in a timely manner. RFID tags, sensors and smart metering are driving the need to deal with torrents of data in near-real time.
• Data comes in all types of formats – from structured, numeric data in traditional databases to unstructured text documents, email, video, audio, stock ticker data and financial transactions.

Two more dimensions were added as

• In addition to the increasing velocities and varieties of data, data flows can be highly inconsistent with periodic peaks. Is something trending in social media? Daily, seasonal and event-triggered peak data loads can be challenging to manage. Even more so with unstructured data.
• Today’s data comes from multiple sources, which makes it difficult to link, match, cleanse and transform data across systems. However, it’s necessary to connect and correlate relationships, hierarchies and multiple data linkages or your data can quickly spiral out of

Big data can be classified as

• Social Networks (or human-sourced information): this information is the record of human experiences, previously recorded in books and works of art, and later in photographs, audio and video. Human-sourced information is now almost entirely digitized and stored everywhere from personal computers to social networks. Data are loosely structured and often ungoverned.
• Internet of Things (or machine-generated data): derived from the phenomenal growth in the number of sensors and machines used to measure and record the events and situations in the physical world. The output of these sensors is machine-generated data, and from simple sensor records to complex computer logs, it is well structured. As sensors proliferate and data volumes grow, it is becoming an increasingly important component of the information stored and processed by many businesses. Its well-structured nature is suitable for computer processing, but its size and speed is beyond traditional approaches.

Human Generated Data is emails, documents, photos and tweets. We are generating this data faster than ever. Just imagine the number of videos uploaded to You Tube and tweets swirling around. This data can be Big Data too.

Machine Generated Data is a new breed of data. This category consists of sensor data, and logs generated by ‘machines’ such as email logs, click stream logs, etc. Machine generated data is orders of magnitude larger than Human Generated Data. Before ‘Hadoop’ was in the scene, the machine generated data was mostly ignored and not captured. It is because dealing with the volume was NOT possible, or NOT cost effective.

By this definition, Big Data as a concept requires three distinct layers before application: more data, processing systems, and analytics. If Big Data only recently entered the supply chain management spotlight, then, it may be because the technology only recently reached the last layer to deliver insights.

Big Data Impact

Ways big data is revolutionizing international logistics as

• The scale, scope and depth of data supply chains are generating today is accelerating, providing ample data sets to drive contextual intelligence. Forward-thinking manufacturers are looking at big data as a catalyst for greater collaboration.
• Enabling more complex supplier networks that focus on knowledge sharing and collaboration as the value-add over just completing transactions. Big data is revolutionizing how supplier networks form, grow, proliferate into new markets and mature over time. Transactions aren’t the only goal, creating knowledge-sharing networks is, based on the insights gained from big data analytics.
• Big data and advanced analytics are being integrated into optimization tools, demand forecasting, integrated business planning and supplier collaboration & risk analytics at a quickening pace.
• Using geoanalytics based on big data to merge and optimize delivery networks. One of the examples provided is how the merger of two delivery networks was orchestrated and optimized using geoanalytics.
• Big data is having an impact on organizations’ reaction time to supply chain issues, increased supply chain efficiency of or greater, and greater integration across the supply chain.
• Embedding big data analytics in operations leads to improvement in order-to-cycle delivery times, and improvement in supply chain efficiency of 10% or greater.
• Greater contextual intelligence of how supply chain tactics, strategies and operations are influencing financial objectives. Supply chain visibility often refers to being able to see multiple supplier layers deep into a supply network. It’s been my experience that being able to track financial outcomes of supply chain decisions back to financial objectives is attainable, and with big data app integration to financial systems, very effective in industries with rapid inventory turns.
• Traceability and recalls are by nature data-intensive, making big data’s contribution potentially significant. Big data has the potential to provide improved traceability performance and reduce the thousands of hours lost just trying to access, integrate and manage product databases that provide data on where products are in the field needing to be recalled or retrofitted.
• Increasing supplier quality from supplier audit to inbound inspection and final assembly with big data. IBM has developed a quality early-warning system that detects and then defines a prioritization framework that isolates quality problem faster than more traditional methods, including Statistical Process Control (SPC). The early-warning system is deployed upstream of suppliers and extends out to products in the field.