case study · batch orchestration

Moving a bank's batch processing off Oracle BPM: what made a 67% faster, zero-disruption cutover possible

Brandon Eilers · Lead Software Engineer

For eight years I was technical lead on a mission-critical application in JPMorgan Chase's Wholesale Credit Risk organization. Its batch processing, the overnight and scheduled work the rest of the application depended on, ran on Oracle BPM: a licensed platform that was heading toward end of life. Moving off it was a cost decision and a modernization mandate at the same time. This is how we did it, and what I'd tell anyone facing the same kind of migration.

Pick the platform the organization can support

I evaluated and prototyped several orchestration platforms before committing. Apache Airflow won partly on its merits (code-defined workflows, a rich operator model, good visibility into runs) and partly for a reason that's easy to overlook: other lines of business in the firm already ran it. That meant existing know-how, support, and a paved path for infrastructure. What nobody in the firm had done yet was use Airflow to replace a true BPM-driven workflow, so the prototypes had to prove that our process logic would translate cleanly.

Design for reuse, not for a one-to-one port

The tempting approach is to port each legacy process into its own workflow and move on. Instead, the design centered on reuse. The final system was 72 DAGs and more than 1,100 tasks, built on 11 custom operators that captured the patterns the processes shared. Behind the operators sat a Spring Boot service with 20+ REST endpoints, so the workflow layer orchestrated work while the business logic stayed in a service the team already knew how to test, deploy, and support.

That split paid off twice: new workflows became mostly configuration of existing operators, and fixes landed in one place instead of dozens.

start + guardsskip if a run is in progress business calendardate + time-zone lookup branchnormal run or rerun trigger fileswait for inbound files FEED AFEED BBATCH + OUTBOUND split file 1processfile 2process join split file 1processfile 2process join batch stepbatch operator → Spring Boot API outboundREST calloutboundREST call validatePython check before loading load downstreamcontext lookup + final load 72 DAGs · 1,100+ tasks · 11 custom operators control flowfile-processing operatorbatch / service operatorREST operatorPython task
An illustrative workflow showing the pattern most of the 72 DAGs followed: guard checks and a calendar lookup, a run-or-rerun branch, parallel groups built from the same reusable operators, and a single convergence point. Simplified and generic; not a production DAG.

The cutover plan matters as much as the code

A batch platform for a credit application can't have a "we'll fix it Monday" weekend. The cutover was coordinated with testing and with our Site Reliability Engineering partners so that every workflow was validated before the old platform was switched off. The result was a zero-disruption cutover, and Oracle BPM was fully decommissioned afterward, along with its licensing and dedicated infrastructure.

Measure, then right-size

Batch runtime dropped 67% compared with the platform it replaced. After go-live, Grafana telemetry showed that the Airflow platform's Kubernetes memory reservation was far larger than real usage, so we right-sized it, cutting reserved memory 60% (from 120Gi to 48Gi) and lowering infrastructure cost for the team and the broader organization.

What I'd tell someone starting a migration like this

Questions about this project? Email me, connect on LinkedIn, or ask the AI assistant on brandoneilers.com.