Type to search

A Primer on Big Data


A Primer on Big Data

Every project or task undertaken generates data. Historically, firms have selectively stored and used data that they may deem important for their future reference. Information technology has made it easier to collect and store this data or information. According to IBM, “Every day, we create 2.5 quintillion bytes of data, so much that 90 percent of the data in the world today has been created in the last two years alone.” Even this statement could be dated given the amount of data that is being generated daily by people, machines, systems and even systems of systems.

The concept of Big Data — large amounts of structured or unstructured data generated exponentially that cannot be consumed by traditional applications completely to produce actionable intelligence — is a reality and no more a hype. Collected data is analyzed computationally to reveal patterns, trends and associations.


Big data has started to play an important role in product design and in the design of materials, in addition to the energy, healthcare, automatic and aerospace industries. One might ask: If data has always been generated by systems, why has this become so large and complex? The answer is simple: Improvements in sensor technology, wireless connectivity and cloud and high-performance computing have led to large-scale creation and sharing of data. These improvements have jumped many fold due to reduction in size, price and integration of systems.

Unstructured Data

Enhancements in 3D CAD and PLM have facilitated the creation of structured data. On the other hand, the transactional data in ERP has been primarily structured. This large structured data in databases can be easily analyzed for future trends and current performance. The unstructured data that resides in files such as PDF files, various CAD formats, social discussions, Word files, etc., is larger than one can imagine and can contain much irrelevant information. Sifting through this data for useful information has already proven a challenge, but the sheer volume makes this task even harder.

Traditionally, and even today, some of the design, engineering and manufacturing departments avoid dealing with such unstructured data and put all their attention into deciphering information in the design systems. The real power of the analysis is in the ability to search historic files for solutions related to current tasks that can involve a part design, material, manufacturing process, quality or change order details. Big data analytics provides the ability to connect to multiple sources or repositories, irrespective of location, and provides actionable information.

Originators and Facilitators of Big Data

As discussed earlier, several technological advancements have encouraged the exponential growth of data. Cloud has primarily reduced the cost of computing alongside wireless connectivity, but there are various sources that have fueled the explosion of data. Social media now generates a huge volume of data around the planet, but such data is not very useful for engineers, designers and manufacturers. Technical discussions might not be as prolific as those on Facebook or Twitter, but the challenge to extract data for actionable intelligence from technical forums are challenging, and Big Data analytics software can easily solve that.

Smarter products that carry sensors provide large structured data sets over time. Smart factories generate data from machines on the shop floor with these sensors and talk to each other, generating yet more data. This structured data is analyzed by big data software for predictive maintenance and upkeep of the entire factory. Smarter products operating in the field are another source of data that is being used for various product improvements and decisions. For example, the Ford electric car measures various attributes related to performance of the car that enable the driver to be abreast of various attributes such as battery charge, gas mileage, etc. It also feeds back to Ford and its various vendors with remote application management software so they may act to improve the car’s design and performance — as well as the placement of charging stations.

Reducing Design Time

Big Data is improving design and product quality, reducing cost and reducing product downtime in the field and during manufacturing. Its applicability has yielded far better results in the operating field as the performance matrices assist in directly improving the design and manufacturing process. In the design department, designers can locate and reuse parts related to their current geometry and shapes, this process can even help in removing duplicates. The low hanging fruit for analytics remains metadata searches that can be performed effectively at any point of the design and engineering phase of a product.

Big Data enables designers to find related parts from an existing design, it also reduces the time required to search each document. Such unstructured data embedded in documents is discovered without drilling too deep into the source document. Big data tools can effectively work in a multi CAD or PLM environment to extract a plethora of data. It can include information related to part material and even to predict if a supplier’s inventory is low or out of stock at a certain time of the year before the design process can begin. Most design departments today have not discovered the full potential of Big Data in predictive analysis of design and suggestive features. In part, this may be due to the complexity of implementing analytics and or the un-availability of such data.

Big Data is being embraced quickly across industries and especially in the field of operations where the benefits can be measured quickly. For instance, Rolls Royce embraced Big Data by installing hundreds of sensors on its engines for collecting information. The data collected is used in design and manufacturing, and to monitor the engine’s operation and plan for its maintenance. The high volume (terabytes) of data generated is analyzed to check the design of the engine with various simulations.
Big Data is changing how products are designed, manufactured and maintained. It will play a big part in the Internet of Things (IoT), but the power is in the analysis and, to go one step further, predictive behavior can only be achieved with Big Data tools.

Most firms have just scratched the surface of what can be achieved with Big Data. The stage is set for Big Data to revolutionize the engineering field with coming innovations, such as self-driving cars or intelligent machines that will make split-second, human-like accurate decisions with changes in multiple physical conditions of its operations — all because of Big Data.

About the Author
Sanjeev Pal is an analyst and software architect with his firm, Neovion Group. He has more than 20 years of experience in the field of product development (CAD/CAM/CAE-PLM) and enterprise technologies. Previously, he worked as a research manager with IDC, in services and R&D at Dassault Systèmes and as a design professional at Timex watches.