How a global medical device manufacturer keeps bad data out of its clinical AI models

A global medical device manufacturer builds AI models that run inside a patient and clinician app used by clinics around the world. The models help clinicians track how patients recover after surgery and spot those who may need extra attention. Behind them sits a small ML engineering team whose job is to make sure every model learns from data it can rely on.

For years, that team had no way to check its data before it reached a model. Then it adopted Bigeye. Today Bigeye catches data issues at least once a month, before they reach production, and saves the team at least 3–4 days of engineering work every month.

The Challenge

The AI team doesn’t produce its own data. It consumes raw data that another team collects and aggregates, and that data can arrive corrupt, incomplete or in a shape nobody expected. Upstream table structures change often, usually without notice.

“Before Bigeye, we didn’t have any data monitoring controls at all, so we would be hit by issues and outages very often,” says the company’s Lead ML Engineer.

The stakes are high. Models are retrained on new data as often as weekly, the fastest release cycle a heavily regulated product allows. A bad batch could mean an inaccurate result on a clinician’s screen.

“We are a heavily regulated industry and we have to be very confident in the things that we release.”

Why Bigeye

The team chose Bigeye for its easy onboarding, its range of data quality features and an engaged Bigeye team that stays close to the account.

Above all, Bigeye fit how the team works. It runs everything as infrastructure as code and was one of the first to use the Bigeye Python SDK. Every check is defined and deployed through the SDK and API, with no manual setup.

“We do everything as infrastructure as code. We deploy everything with scripts and barely touch the interfaces. It was very helpful that we had the SDK and the API.”

The 1,800-Year-Old Patient

One catch shows what changed. A Bigeye validity rule on the age field flagged a patient record showing an age of about 1,800 years. The team sent it back upstream to be fixed before it reached a model.

Patient wearables bring similar surprises. Smartwatches sometimes send readings that cannot be real, such as negative heart rates. Bigeye’s anomaly detection flags them before they reach production data or the data used to train AI models.

“Before it becomes a bug that a user sees on their app, or a clinician sees a red sign on the screen, we catch it because Bigeye alerted on it, and then we know to fix it.”

Early Warning on Upstream Changes

Schema changes are the most frequent risk. Bigeye detects them automatically and alerts the team before they break a pipeline, cause a missed SLA or quietly corrupt the features a model depends on.

“Schemas change upstream all the time, most of the time without anyone letting us know. Bigeye gives us an advance warning before those issues become severe.”

Bigeye also shows how the data changes over time. When the company signed new clients with a different patient mix, distribution monitoring showed the average patient age and recovery metrics shift. The team could confirm the change reflected real patients, not a data error, before retraining.

From Reactive to Proactive

Problems are now fixed upstream before they reach production, so pipelines and SLAs hold. Data used in research the team’s data scientists publish has already been checked. Every model release starts from validated data, which matters when each release has to stand up to regulatory scrutiny.

AI models are only as reliable as the data they learn from. For this team, Bigeye checks every batch of that data, as code, before it reaches a model or a clinician.