Senior Big Data Engineer Senior Big Data Engineer …

BNY Mellon
à New York, NY, États-Unis
CDI, Plein-temps
Soyez parmi les premiers à postuler
BNY Mellon
à New York, NY, États-Unis
CDI, Plein-temps
Soyez parmi les premiers à postuler
Senior Big Data Engineer

Clearing, Markets & Issuer Services Technology (CMIST) is responsible for applicationdevelopment and support for critical business systems including Repo Edge(collateral management), Enterprise Payment Hub (multi-currency paymentprocessing), and Broker Dealer Clearance (securities clearing), along withapproximately 350 other applications used by the following high-prioritybusiness services and their clients.

Clearance andCollateral Technology (CCT) withinCMIST builds clearance and collateral management platforms toservice BNY Mellon's broker-dealer clients and is the sole provider for GovernmentSecurities Clearing Services. Considering the core nature of the business andthe dominant market share, high-performance and resiliency are key pillars ofthe technology architecture. The group is currently focused on RepoModernization, which will pave the way to the Future of Collateral for thebusiness - providing strengthened platform resiliency and new tradecapabilities within a unified system. All CCT applications are built forreal-time operations aided by a data warehouse for reporting and businessinsights.

This is a 100% hands-on role, and doesn't involve any teammanagement responsibilities
  • Develop, deploy, and maintain scalable/optimal data pipeline and the supporting data platforms and infrastructure
  • Standardize all data pipeline processes, develop reusable modules (auditing, logging, traceability), integrate with DevOps pipeline
  • Recommend improvements to existing data integration architecture and implementation of standard frameworks that automate manual processes, improve overall scalability, increase reliability, and improve quality
  • Define the right data models working with ML engineers, data scientists, and reporting/analytics developers - optimizing data storage, improving data accessibility, and making them re-usable

Consults with internal business groups to provide highly advanced application software development services or technical support. Consults to IT and business managers on issues, questions and problems related to his/her area of expertise. Accountable for resolving highly complex problems that have impact on the corporation's ability to conduct business. Uses in-depth knowledge of various technologies to establish and influence implementation of broad-based technology strategy for large IT Divisions. Stays abreast of emerging technologies and identifies potential use of new and existing technology within lines of business by participating in industry-wide conferences and research and by using in-depth knowledge of various business areas. Educates business leaders and prepares presentations for the business on IT strategies. Advocates for innovative, creative technology solutions. Contributes to the achievement of Application Development objectives.

  • BS degree (Computer Science, Math, Physics, Engineering). MS / PhD preferred.
  • 12+ years of experience in software development required
  • 5+ years overall hands-on experience in building and maintaining production grade data pipelines, developed in Python or Java, that support critical business functions, reporting, and analytics
  • 3+ years overall hands-on experience in Python
  • Very strong SQL skills with multiple years of experience in developing and maintaining complex SQLs across different standard and big data technologies (preferred)
  • Extensive experience with big data platforms, technologies, and tools and technologies (Cloudera Hadoop, Spark, Hive, Impala, Dremio, Google BigQuery, ECS, etc.) - understanding their architecture, infrastructure requirements, troubleshooting tricks, integration with other platforms, and automation of processes
  • Should have experience in handling large volume/size (billion / > 100 TB), wide data sets (> 200) and velocity (
  • Hands-on experience around compute and storage in any one of the public cloud platforms such as Google Cloud (preferred), AWS, or Azure is highly preferred.
  • Experience in the securities or financial services industry is a plus.

BNY Mellon is an Equal Employment Opportunity/Affirmative Action Employer.
Minorities/Females/Individuals With Disabilities/Protected Veterans.

Our ambition is to build the best global team - one that is representative and inclusive of the diverse talent, clients and communities we work with and serve - and to empower our team to do their best work. We support wellbeing and a balanced life, and offer a range of family-friendly, inclusive employment policies and employee forums.

Primary Location: United States-New York-New York
Internal Jobcode: 45220
Job: Information Technology
Organization: Clearing Markets ISS Svcs Tech-HR16624
Requisition Number: 2008627