Better Data Filter for Custom Logic, Numpy At

A step by step guide to use pandas in exploratory data analysis

If you want to apply a function at some specific ad-hoc positions in an array, then Numpy At is a good choice.

This method is very handy and instrumental in performing comparative analysis and detecting data abnormalities at an early stage.

Photo by Joshua Mayo on Unsplash

Let’s see how we can use it in troubleshooting mechanical problems. Implemented this for a client who performs Machine Health Monitoring Services in the automotive industry.

Domain Notes

For those who are new to the vibration analysis field, here is a quick overview.

What is machine health monitoring?

In general, from chocolate to an aircraft, anything that is produced in a factory needs several machines to operate in synergy. However, if a particular machine in the critical path goes down it affects the complete chain of production. As a result, it incurs a huge loss to the company. So, it is very important to keep monitoring the health of each machine.

As a matter of fact, many kinds of machines like grinding machines, compressors, or refiners keep producing some vibrations while they are running.

Machine health monitoring aims at predicting the faults of a machine before they occur. All in all, this stream of science periodically checks the machine conditions, analyses the data and predicts the failures.

What is vibration analysis?

When humans feel sick, they express their pain by voice to alert others and to communicate with a doctor. But how can machines do that? Of course, they too follow a similar way.

When there is a problem developing inside a machine, the machine expresses it by making different(abnormal) sounds/vibrations. Following that, a team of experts would analyze these vibrations to detect the nature and cause of the fault.

Vibration analysis is a growing field of data analytics. It has great potential in driving the early detection of machine failures and saving enormous failure costs.

What do the field engineers do?

Our field engineers love the machines like pets. They regularly visit the factories, read the machine condition send that data to our analytics team.

Basically, they use the devices like a vibrometer or an accelerometer and read the machine vibrations. In effect, these devices translate the vibrations into various mechanical attributes like velocity, acceleration, RPM of a machine.

What does the data analytics team do?

Well, the data engineers at the data analytics team enrich the attributes of the datasets. As soon as they receive the datasets from the field engineer, the data engineers transform the machine readings into more statistical measurements. And the enriched data can be easily interpreted by business analysts.

What do the business analysts do?

Here the business analysts are typically mechanical engineers. They understand well about the machine internals. From the statistics reported by the data analytics team, they determine the problem location. Subsequently, they prescribe the corrective action.

Business Scenario

Now let’s get into the specific requirement related to noise filtering. This is a scenario related to the welding machine and comparative analysis.

To detect the faults in a welding machine, we need to take continuous readings of the motor vibrations. But when this machine comes in contact with the external surface (the object being welded), some extra noise gets added to the system. The actual motor vibration readings get inflated by a certain percentage.

To address that, the field engineer uses two vibrometers. And takes the vibration readings from two different spots on the machine. One device is tuned to take the readings of the motor. The second device is tuned to take the readings of the external object. The second device tells at which timestamp did the external object came in contact with the welding machine.

The analytics module must filter out the noise from the motor readings. It has to modify the motor vibration readings exactly at the timestamps when the machine got in contact with the external object. Such readings have to be downgraded by 10 percent to filter out the approximate noise arising from the external object.

Let’s see how we can solve it by using the Numpy Accumulate method.


  1. Firstly, read the motor vibrations taken from the first device.
  1. Subsequently, read the object(noise) vibrations recorded by the second device. If the frequency is 0 that means the object has not contacted the machine. Any value above zero indicates there was a contact between the object and the machine that has emitted some noise.
  1. Derive the position in the above array, when the object has contacted the machine.
  1. Now, use the Numpy At method to modify the motor readings at these positions. Reduce the value by 10% as specified by the business analyst.

This method multiplies the values in the series df1[‘Frequency’], at the positions specified by noiseIndices, with a factor of 0.9.

  1. Just look at the modified dataframe.

Observe that the values are modified at the positions [1, 2, 6, 7, 10, 13, 14, 18].

  1. To see the difference in summary, plot a graph comparing the original frequencies with the modified frequencies.

That’s it, we have successfully filtered out the noise vibrations at some ad-hoc positions in the array.


We have read the vibration frequencies of the motor and the object (noise) into two data frames. Both have the same time indices. Once we have these readings ready, we have calculated the positions at which the object noise might have inflated the motor frequencies. Then we have applied Numy At method to filter the data frame exactly as per these positions and apply the multiplication function. That has filtered out the noise part from motor vibrations, by some fixed offset percentage.

What Next?

If you would like to share your business case on how you’ve used this method, please send the details to Our editorial team shall get back to you in adding it to this publication. Please contribute to the integration of knowledge towards building a new generation of fast learners. An example speaks a thousand paragraphs.

Also Read

An Excellent Method for Instant Data Transformation, DF GroupBy Transform

How is GroupBy Rank used in Pricing Intelligence?

Write a Comment