Agrology
A data-driven product that needed new data infrastructure.
Impact
2
ERP sources unified
Albor + Finnegans into single pipeline
100% serverless
Infrastructure
Lambda + S3 + Glue + Redshift
5
Pipeline stages
Landing → Vault → Raw → Stage → DW
Where they were
Agrology is an agro-tech platform that gives producers and financial institutions visibility into the agricultural cycle, from planting to the port. They had data from two of the most important ERPs to feed BI dashboards for producers, so they can make informed decisions. Although they had a solid knowledge base, the data infrastructure wasn’t keeping up: monolithic stored procedures, local and manual ingestion processes, cost inefficiencies, and an outdated data model for the project’s current needs.
What we found
Data ingestion was manual and fragile. They didn’t yet have a modern data warehouse, but a transactional database, without staging layers and without separation of responsibilities. The standardization of values entered by producers (who entered data inconsistently across ERPs) was done by hand. The team had tried Athena but abandoned it because the tooling felt unfamiliar to them. The real gap: lots of digital ambition, zero data architecture.
What we built
A serverless datalake on AWS, from scratch, as a source of truth at company level. S3 as the storage backbone with four clearly defined layers: Landing, Vault, Raw and Stage. Lambda functions handling ingestion and orchestration, for event-driven automation. Glue Jobs were used for transformation: renaming columns to Agrology’s standard, normalizing producer-entered data, and loading into Redshift. DataMarts segmented by business unit so each user sees only what they need. All infrastructure deployed with AWS SAM, with infrastructure as code, reproducible and versionable. Every Lambda and every Glue Job notifies Slack in real time.
What they can do now
The BI team queries a real warehouse instead of the transactional system. New data sources “plug into” the same pipeline pattern, standardizing processes. DataMarts give teams their own segmented views without stepping on each other. The architecture is documented and the team can extend it. When Agrology needed to evolve the platform further, the base was solid enough to build on.