Product
-
March 25, 2024

Announcing Bigeye Lineage Plus

Massive, complex, hybrid enterprise data environments have never looked so good.

Kendall Lovett

We've always known data lineage would become an increasingly critical part of the data observability story. As we continued to attract large enterprise customers and prospects, the need for truly end-to-end enterprise lineage, and the current gap in the data observability market, only became more clear.

It makes sense that data observability vendors, born and bred in the cloud, would struggle to handle the size and complexity of enterprise data pipelines that weave through a broad mix of modern and decidedly not-so-modern technologies.

But if Bigeye was going to continue supporting the data observability initiatives of our Fortune 500 customers, we knew we needed to up our lineage game. So we did.

Bigeye Lineage Plus

Automated lineage for your entire hybrid stack. No, seriously.

Bigeye Lineage Plus delivers end-to-end enterprise lineage in unmatched detail. Now enterprise data teams can automatically map cross-source columnar lineage from BI tools, through modern and legacy data sources, and even across the boundary from cloud to on-prem.

☝️ Schedule a custom demo to see Bigeye Lineage Plus in action!☝️

Bigeye Lineage Plus is built for the enterprise.

  • End-to-end: Automated lineage mapping across BI dashboards, modern and legacy sources, ETL tools, and transactional databases.
  • High fidelity: Cross-source columnar mapping with unmatched detail into the interrelationships between each data source.
  • Cloud, on-prem, and hybrid: 50+ connectors that trace data lineage even as it moves from cloud to on-prem sources and back again.

Tracing lineage beyond the modern data stack

Data lineage has become a ubiquitous feature for many types of data operations tools. While mapping the lineage of intra-relationships within a single cloud data warehouse or lake is fairly straightforward, keeping track of the inter-relationships that occur once the data jumps to a different source—particularly a legacy source—is a whole nother story.

Mapping lineage across legacy or on-premises environments is complex and most tools, if they support it at all, require you to use complex, custom APIs or manual entry to try and capture the full picture.

After speaking with enterprise data leaders, we found three key limitations with other available data lineage offerings:

  1. Lineage can only be gathered for modern data stack sources
  2. Lineage can be gathered within a source but not between sources
  3. Lineage cannot capture the ETL jobs that are moving the data

Connecting all the enterprise dots

To tackle these challenges, we acquired a company with over 20 years of experience building lineage connectors for 70+ technologies still common in Fortune 500 data stacks today. We then combined our respective modern data stack and legacy data stack expertise to develop Bigeye Lineage Plus.

The result is the first completely end-to-end enterprise lineage technology that can finally address the aforementioned challenges with:

  1. 50+ connectors covering both modern data stack and traditional enterprise sources
  2. The ability to trace lineage even as data moves across sources
  3. The ability to capture ETL job information so no step in the pipeline is lost

Each of Bigeye's 50 Lineage Plus connectors includes column-level parsers to map transactional databases, cloud data warehouses, data lakes, ETL platforms, BI tools, and more. These parsers trace lineage at the column level even as it moves across sources throughout hybrid, cloud, and on-prem pipelines.

As a result, Bigeye customers can view a single lineage graph of their entire pipeline from original sources, to ETL job-level insights, and all the way through to analytics dashboards.

Bigeye Lineage Plus connectors are available today for many of the most popular data sources, including Tableau, Microsoft Power BI, Snowflake, Databricks, Google BigQuery, Amazon Redshift, Azure Synapse, IBM DB2, Oracle Database, MySQL, PostgreSQL, Microsoft SQL Server, SAP HANA, and Vertica.

A wide range of additional connectors will be made available throughout 2024, including Informatica PowerCenter, IBM Netezza, Teradata, SAS, Talend, SnapLogic, Apache Spark, IBM DataStage, MicroStrategy, QlikView, SAP Business Objects, Tibco Spotfire, and others. 

☝️ See the full list of available and coming soon connectors on our integrations page☝️

Ensuring enterprise data is reliable by default

Our customers use Bigeye Lineage Plus to get a complete picture of the health of their data pipelines and find and fix issues before they impact critical business operations. When combined with Bigeye Dependency Driven Monitoring, data teams can trace every upstream dependency for an analytics dashboard or data product across modern and legacy sources and deploy targeted data observability on each critical column.

When a data issue is detected, Bigeye Lineage Plus-powered root cause and impact analysis capabilities are available to help teams quickly understand the downstream impact and identify the root cause for fast prioritization and resolution.

The complete picture

With Bigeye Lineage Plus, data teams in the world’s largest enterprises can get a complete view of their entire pipeline, use data observability to find and fix issues at the source, and deliver reliable analytics the business can trust.

☝️ Register for our webinar on April 10th to learn more!☝️

share this episode
Resource
Monthly cost ($)
Number of resources
Time (months)
Total cost ($)
Software/Data engineer
$15,000
3
12
$540,000
Data analyst
$12,000
2
6
$144,000
Business analyst
$10,000
1
3
$30,000
Data/product manager
$20,000
2
6
$240,000
Total cost
$954,000
Role
Goals
Common needs
Data engineers
Overall data flow. Data is fresh and operating at full volume. Jobs are always running, so data outages don't impact downstream systems.
Freshness + volume
Monitoring
Schema change detection
Lineage monitoring
Data scientists
Specific datasets in great detail. Looking for outliers, duplication, and other—sometimes subtle—issues that could affect their analysis or machine learning models.
Freshness monitoringCompleteness monitoringDuplicate detectionOutlier detectionDistribution shift detectionDimensional slicing and dicing
Analytics engineers
Rapidly testing the changes they’re making within the data model. Move fast and not break things—without spending hours writing tons of pipeline tests.
Lineage monitoringETL blue/green testing
Business intelligence analysts
The business impact of data. Understand where they should spend their time digging in, and when they have a red herring caused by a data pipeline problem.
Integration with analytics toolsAnomaly detectionCustom business metricsDimensional slicing and dicing
Other stakeholders
Data reliability. Customers and stakeholders don’t want data issues to bog them down, delay deadlines, or provide inaccurate information.
Integration with analytics toolsReporting and insights

Join the Bigeye Newsletter

1x per month. Get the latest in data observability right in your inbox.