Big Data

What is Big Data

It is an all-encompassing term for any collection of data sets so large and complex that it becomes difficult to process using on-hand data management tools or traditional data processing applications.

Industry analyst Doug Laney articulated the now-mainstream definition as the three Vs:

• Volume. Organizations collect data from a variety of sources, including business transactions, social media and information from sensor or machine-to-machine data. In the past, storing it would’ve been a problem – but new technologies (such as Hadoop) have eased the burden.
• Velocity. Data streams in at an unprecedented speed and must be dealt with in a timely manner. RFID tags, sensors and smart metering are driving the need to deal with torrents of data in near-real time.
• Variety. Data comes in all types of formats – from structured, numeric data in traditional databases to unstructured text documents, email, video, audio, stock ticker data and financial transactions.

Two more dimensions were added as

• Variability.  In addition to the increasing velocities and varieties of data, data flows can be highly inconsistent with periodic peaks. Is something trending in social media? Daily, seasonal and event-triggered peak data loads can be challenging to manage. Even more so with unstructured data.
• Complexity. Today’s data comes from multiple sources, which makes it difficult to link, match, cleanse and transform data across systems. However, it’s necessary to connect and correlate relationships, hierarchies and multiple data linkages or your data can quickly spiral out of

It can be classified as:

• Social Networks (or human-sourced information): this information is the record of human experiences, previously recorded in books and works of art, and later in photographs, audio and video. Human-sourced information is now almost entirely digitized and stored everywhere from personal computers to social networks. Data are loosely structured and often ungoverned.
• Internet of Things (or machine-generated data): derived from the phenomenal growth in the number of sensors and machines used to measure and record the events and situations in the physical world. The output of these sensors is machine-generated data, and from simple sensor records to complex computer logs, it is well structured. As sensors proliferate and data volumes grow, it is becoming an increasingly important component of the information stored and processed by many businesses. Its well-structured nature is suitable for computer processing, but its size and speed is beyond traditional approaches.

Human Generated Data is emails, documents, photos and tweets. We are generating this faster than ever. Just imagine the number of videos uploaded to You Tube and tweets swirling around. This data can be too.

Machine Generated Data is a new breed of it. This category consists of sensor data, and logs generated by ‘machines’ such as email logs, click stream logs, etc. Machine generated is orders of magnitude larger than Human Generated Data. Before ‘Hadoop’ was in the scene, the machine generated data was mostly ignored and not captured. It is because dealing with the volume was NOT possible, or NOT cost effective.

By this definition, it as a concept requires three distinct layers before application: processing systems, and analytics. If only recently entered the supply chain management spotlight, then, it may be because the technology only recently reached the last layer to deliver insights.

Big Data benefits to SCM

What are the real, tangible benefits that big data can bring to the supply chain? Let’s take a look:

• Traceability – Being able to pinpoint where products are in the supply chain is critical. Therefore, making sure items are traceable is key to a successful operation. By using barcode scanners and attaching radio frequency identification devices to certain products, supply chain managers can aggregate required to create this kind of traceability.
• The end-to-end tracking and traceability that sensors and barcodes provide enable store chains, food brands and goods supply networks to quickly identify points of origin and distribution if it’s discovered that food is contaminated
• Relationship management – It’s not news that the C-suite thinks that is worth the investment. According to SCM World’s report, 64 percent of supply chain executives think it an important and disruptive technology. One of the biggest reasons it is worth the investment is that it can help companies provide better service to customers and improve relationships the board. When every stop in the supply chain has access to the correct customer information, there’s a greater chance of fulfilling demand and fixing any problems that arise in distribution. Not to mention that in addition to learning what their customers want, companies can also use to better understand their vendors, as well.
• Forecasting – It Collective contributor Larry Alton noted that sets are especially useful when applied toward predicting what customers want – and when they want it. With accurate forecasting, which can take place when organizations look back at previous transactions throughout their supply chains, companies can improve profitability and better predict consumer demand, not to mention reduce supply chain waste.

Big Data Impact

Ways big data is revolutionizing supply chain management as

• The scale, scope and depth of supply chains are generating today is accelerating, providing ample sets to drive contextual intelligence. Forward-thinking manufacturers are looking at a catalyst for greater collaboration.
• Enabling more complex supplier networks that focus on knowledge sharing and collaboration as the value-add over just completing transactions. is revolutionizing how supplier networks form, grow, proliferate into new markets and mature over time. Transactions aren’t the only goal, creating knowledge-sharing networks is, based on the insights gained from analytics.
• It and advanced analytics are being integrated into optimization tools, demand forecasting, integrated business planning and supplier collaboration & risk analytics at a quickening pace.
• Using geoanalytics based to merge and optimize delivery networks. One of the examples provided is how the merger of two delivery networks was orchestrated and optimized using geoanalytics.
• It is having an impact on organizations’ reaction time to supply chain issues, increased supply chain efficiency of or greater, and greater integration across the supply chain.
• Embedding analytics in operations leads to improvement in order-to-cycle delivery times, and improvement in supply chain efficiency of 10% or greater.
• Greater contextual intelligence of how supply chain tactics, strategies and operations are influencing financial objectives. Supply chain visibility often refers to being able to see multiple supplier layers deep into a supply network. It’s been my experience that being able to track financial outcomes of supply chain decisions back to financial objectives is attainable, and with app integration to financial systems, very effective in industries with rapid inventory turns.
• Traceability and recalls are by nature data-intensive, making its contribution potentially significant. It has the potential to provide improved traceability performance and reduce the thousands of hours lost just trying to access, integrate and manage product databases that provide on where products are in the field needing to be recalled or retrofitted.
• Increasing supplier quality from supplier audit to inbound inspection and final assembly with it . IBM has developed a quality early-warning system that detects and then defines a prioritization framework that isolates quality problem faster than more traditional methods, including Statistical Process Control (SPC). The early-warning system is deployed upstream of suppliers and extends out to products in the field.