- Polyglot software engineer with more than 8 years experience
- Flexible in Java, Scala, Node.js, Python (and Ruby for Chef)
- Machine learning and data pipeline projects using Apache Spark, AWS EMR, Azure HDInsights and Elasticsearch.
- Language and platform agnostic approach for problem-solving with a focus on keeping it simple.
- Fan of automation and DevOps tools like Terraform, Ansible, Chef and always does CI/CD
- Worked on cloud platforms like AWS and Azure
- Experience in Banking, Social Media, Advertising, Travel, and Government sector domains.
- Experience working in Agile (Scrum, Kanban) methodology with test-driven development (TDD) and pair programming.
- Expertise in designing and developing RESTful services.
- Full stack web application development, covering front end, middleware, and backend, including system architecture and database designing
- Contributed to open source projects like Dropwizard, Play framework etc on GitHub.
- Master’s degree in E-Business and Information Systems from Newcastle University, UK.
- Bachelor’s in Computer Science and Engineering from the University of Calicut, India.
- Continuous learner
- Object Oriented programming
- Functional Programming
- Team Player
- Understanding of design patterns
- Distributed/Client-Server systems
- Critical analysis & problem solving skills
- Good communication skills
- Data Engineering
- Proactive self-learner & motivated by achievements
June 2017 - Till Date Valtech, London, UK
I joined Valtech as Senior Technical consultant. At Valtech I got training on presentation skills, which I used to present a talk on Anomaly Detection in time series data.
1. Website customer behaviour prediction
January 2018 – Current Project, London, UK
Role: Data Engineer/DevOps
- Research on AWS Sagemaker
- Prepare JSON data from Bson using PyMongo library
- Design and implement data pipeline using EMR
- Design and implement machine learning pipeline using EMR
- Streaming data with Kinesis
- Streaming KMeans Clustering algorithm using Apache Spark Streaming context
- Persistance in S3 and DynamoDB
- Visualising streaming clusters with LightningServer and Dash
- Infrastructure automation with Terraform and Ansible
Technologies Used: AWS, Spark, Hive, Python, Jupyter, EMR, Terraform, Zeppelin, AWS Glue, AWS Athena, S3, DynamoDB
2. Big-data pipeline and analytics
October 2017 – December 2017, Norway, London
Role: Data Engineer/DevOps
Data project to analyse large volumes of data on Hadoop echo system on Microsoft Azure platform. This project has a data pipeline part, some data scrapping and data analysis part. Data came from various motors and sensors.
- Get vessel and weather information from web using Microdata and API’s in Python
- Data pipeline built on Azure HDInsights platform
- Apply transformations to Hive tables using PySpark based on algorithms by data scientists
- Created a Workload report of HDInsights using Azure Runbooks, using Yarn and Livy APIs
- Build a data analytics platform on Azure using HDInsights, Spark and Jupyter
- Setup Ansible playbooks for configuring JupyterHub and users
- Explored Apache Ranger using Hortonworks Sandbox on Ubuntu
Technologies Used: Jupyter, Apache Spark, HDInsights, Azure, Ansible, Plotly, Apache Livy, Horton-works, Python, Azure Storage accounts
3. HMRC - Senior Scala Consultant
June 2017 – October 2017, CSHR, London, UK
Role: Software Engineer
Project Type: Website, Customer responsive platforms
This is a UK government project delivered following the standards set by Government Digital Services. The UK government uses Fast Track and Fast Stream to recruit talents for the Civil Service.
- Re-design the two services, with new GOV.UK services, so that candidates can apply and attend online tests
- Back office system for candidate support
- This is a responsive website service that is easy to use on any device.
This project involved creating API, candidate frontend and admin frontend for UK Government Civil Service HR Faststream candidate application.
Part of this project is open source and is available at https://github.com/hmrc/fset-faststream, https://github.com/hmrc/fset-faststream-frontend
Technologies Used: Scala (Play framework), HTML5, CSS3, JS, Progressive enhancement, Respon-sive web design
August 2015 — Jan 2017 Expedia Affiliate Network, London, UK
Joined Expedia on contract basis for building the next generation reporting tool for internal users and then extended the contract on to anomaly detection platform. Responsibilities included software development in various languages, design & architecture discussions, DevOps, deployment pipeline, and interviewing candidates.
1. Reporting Platform
The project involved building a reporting platform using micro-services architecture in a Kanban environment. RESTful API’s were built using Node.js and front end was built with React.
- Developed user identity management service API and implemented authorisation algorithm using JSON Web Tokens(JWT) and oAuth.
Designed and implement various REST API endpoints in Node.js and Restify for getting the data. HATEOAS (Hypermedia As The Engine Of Application State) pattern was used for linking between resources.
- Wrote a framework Chef Cookbook (Library cookbook) for deploying Node.js based services.
Wrote Chef cookbooks (Application Cookbook) for various services, including rspec tests and integration tests with kitchen vagrant. Acted as pseudo DevOps member within the team.
- Implemented API Gateway using nginx and Chef.
- Implemented Service Discovery using Hashicorp Consul. This radically improved the time required for the service to be available in the gateway after deployment.
- Implemented next iteration of API Gateway with nginx, Consul and chef.
- Deployment in AWS Cloud. Setup nodes using Scalr. Fix various deployment issues.
- Did a brown bag session on Hashicorp Consul and service discovery, which was well-received.
Worked closely with Operations team to deploy services to Production environment.
Technologies Used: Node.js, Chef, cookbooks, rspec, serverspec, chefspec, AWS Cloud, Teradata, Node.js, Ruby, React, redux, rubocop,Jenkinss, Scalr, bamboo, stash, gitlab, DevOps, restify, Bamboo, upstart.
2. Anomaly Detection platform (Machine Learning)
Big data analytics project for detecting anomalies in various time series data using machine learning techniques. Worked closely with Data scientists to develop models and the platform.
- Worked closely with Data scientists to create models in Scala to run on Apache Spark. Original models were written in Python and R.
- Used Spark to parallelise forecasting based on splits of data. Wrote Scala based tool for triggering AWS Simple Notification Service (SNS).
- Automated Airflow DAG creation using Python based on configuration files in AWS S3.
- Wrote AWS Lambda functions in Python to trigger data pipeline.
- Setup Elastic Search cluster on AWS cloud. Wrote a script to automate the process.
- Optimised data ingestion to Elasticsearch from Hive on AWS Elastic Map Reduce (EMR), brought down the time taken from 3.5 hours to 20 minutes.
- Used Spark to move data from Teradata to S3 in Parquet format.
Technologies Used: Apache Spark, Hive, AWS Cloud, AWS Lambdas, Python, AWS SNS, AWS Elastic Map Reduce, Scala, Python, Airflow, Bamboo
February 2015 — June 2015 Kainos, Burnley, UK
In DWP location on a project related to Government Digital Service (GDS) and Gov.UK Verify using micro-services architecture as a Tech Lead in an Agile (Scrum) environment.
Actively upgraded existing source code to use Java 8 features. Helped new team members practice TDD. This project uses Elastic search to find possible matches from a matching data set. Wrote scripts to ease development process. Brought in consistency and standardisation to source code.
Responsibilities include technical discussions with architects, sprint planning, sprint elaboration, delivering sprint vision, knowledge sharing among team members, pair programming and code review.
Reported and fixed issue #953 in DropWizard project on GitHub during the stint. https://github.com/dropwizard/dropwizard/pull/1001
November 2014 — January 2015 Corelogic, Essex Road, LONDON, UK
The project was to build new module into existing social care product developed by Corelogic. Worked on Java (Spring/ibatis), Backbone/Marionette, Microsoft SQL Server and Oracle stack. Suggested positive changes to standards and practices.
Introduced JSR-303 server side validation. Designed RESTful API’s and structured controllers. Responsibilities included attending sprint meetings, code review and merging changes from other developers.
Technologies Used: Java, Spring, ibatis, Backbone.js, Marionette, Handlebars, Underscore.js, MSSQL, Oracle, Weblogic server, Gitlab, JIRA, TestNG, Mockito.
October 2014 — Till Date Downtown Consulting, Harrow, UK
Available to hire for contract jobs immediately. Happy to relocate to anywhere in UK.
March 2013 — September 2014 Collective: The Audience Engine, Bangalore, India
Collective is a leader in multi-screen advertising and dynamic optimized creatives and programmatic advertising that connects brands to audience with personalized ads. Joined the team at Collective to build and evolve the best dynamic creative optimized ad platform in an Agile environment.
- In-house BigData Analytics product, which produces reports from the ad delivery/engagement logs generated by the ad server.
- Designed and developed secure REST API for reporting dashboard using Play Framework and Redis for caching.
- Play framework and NodeJS/ExpressJS based secure RESTful APIs
- Rich UI using AngularJS and RequireJS
- RESTful Reporting API (Dashboard)
- Setup and lead Play (Scala) project from scratch (Proposed the use of this framework).
- Slick (Scala) for database access.
- Implemented a state machine based Authentication header parser and developed algorithm.
- Single sign-on integration with Authentication API
- Redis for caching
- Jackson for JSON generation.
- JAXB annotations for converting authentication message to object.
- Spring Batch for batch processing and pre populating the Cache.
- Vagrant boxes for database and caching during development.
- Pair Programming
- Test Driven Development
- In-memory H2 database with YAML data files for unit testing.
- Knowledge transfer to team members.
- Build with Jenkins, deployment with Chef
- Reporting UI (Dashboard)
- Setup and lead Node/Express project from scratch.
- Visualization using HighCharts and D3JS
- Build with Jenkins, deployment with Chef
- Proof Of Concept
- POC for the very ambitious dashboard project
- Created using Play (Java/Scala)
- UI with Angular JS
- Data visualization with D3JS and NVD3
- Tableau Reporting
- Yeoman based project setup.
- UAT in Heroku
- Build with Grunt and Jenkins, deployment with Chef
- In-house Analytics product
- Maintained this already matured product.
- Modernised UI using Bootstrap, Struts, jspx and tagx
- Added new reports
- Scheduling with Quartz
- IBatis for SQL queries
- Python scripts for chores.
- Build and deployment with in-house tool
November 2012 — January 2013 University of Newcastle upon Tyne, UK
This was a paid project given by the University as part of Newcastle Work Experience program to build an academic assignment feedback generation application which, makes it possible to prepare constructive and comprehensive feedback in an efficient and effective manner.
- Built using rapid application development tool Spring Roo, Jasper Reports and Dojo
- User authentication and authorisation features using Spring Security.
- Java Mail API to send registration email.
- UAT deployment in Heroku and PostgreSQL
- Production deployment in Tomcat server and MySQL database located at remote University Server using ssh key authentication.
March 2012 — September 2012 Happiest, UK
Happiest is a start-up in Newcastle upon Tyne, that focused on Rewarding customers based on engagements. To be a part of an exciting start-up in the UK was really thrilling. Most exciting part was the thought of building something that is going to delight people. The internship turned into an offer for doing Master’s dissertation.
Data Engineer (June - September)
- Dissertation project on Social Network Analysis and Centrality metrics
- This project was considered the initial phase of bigger data analysis project.
- Created a friend recommendation system using Neo4J graph database.
- Analysis of social network data using R, IGraph
- Happiest’s network data was analysed and centrality points were given to each node in the network.
- Graph data visualization using R and Gephi.
API Developer (March - June)
- Developed various API end points in PHP
- Database programming in PostgreSQL
- Responsible for GitHub commits and merges.
- Scrum methodology
- Participate in daily stand-up sessions and discussions.
August 2008— August 2011 JMR Infotech, India
JMRInfotech is a technology consulting company Banking and Financial Services Industry domain in India, where I served for 3 years including offshore support and client site development projects.
Software Engineer/Technical Consultant
- Was deployed in client site and acted as single point of contact.
- Lead the team in design and development and also gave training to team members and knowledge transfer to new members.
- Designed and developed Java based web framework for development.
- Training in Flexcube (OFSS), PL/SQL and Oracle Forms.
- Offshore support for Flexcube.
- Responsible for code maintenance and version control.
- Responsible for UAT deployment.
- User and session management module - Created the complete design document for the project.
- Customised account reports from Flexcube database - Lead developer in Java, JQuery, Hibernate, Jasper Reports, production deployment in Websphere server
- Bank cheque truncation automation project - Lead developer in Java Swing interface between Flexcube and cheque truncation automation vendor. The Swing application generates XML files as specified by the client in a given location.
- Credit card tab for Internet Banking - Developed in Java, production deployment in Websphere server. The credit card tab worked by consuming Webservice (SOAP) provided by Credit card vendor.
- Bulk Signature Uploader - Upload scanned signature images from a given location. Single-handedly developed. Used XML for representing data and XSLT to translate to html.
- Data double entry - Web application to verify same data entered by two employees. Single-handedly developed. Used XML for representing data and XSLT to translate to html.
- Company Website - Built on DotNetNuke. Team member.
- Search Engine Optimization - Webmaster
|APIs||JEE (JSP, Servlets, Filters), JPA, JAXB, JUnit, Jasper Reports, Log4J|
|Frameworks||Play Framework, Spring Core, Spring MVC, Spring Roo, Spring Security, Hibernate, Spring Data Neo4J, Spring JUnit|
|Methodologies||Agile (Scrum), OOP, Design Patterns|
|Middleware||Apache Tomcat, Websphere, nginx|
|Operating Systems||Mac OSX, Linux, Windows|
|Database||Oracle, PostgreSQL, MySQL, Neo4J(NoSQL graph database)|
|Tools||IntelliJ IDEA, Spring Tool Suite, Eclipse, Toad, PL/SQL Developer, pgAdmin, Sequel Pro|
|Source control||git, svn, perforce|
|Acquainted with||C, C++, R, PHP, Ruby|
|Unofficial||Amazon Web Services, MongoDB, Travis CI|
September 2011 — September 2012 M.Sc. E-Business Information Systems (Merit)
Taught postgraduate course with modules from both computing school and business school, which gave great exposure and outlook on both technology and business aspects. Skills for project management and business planning were enhanced by the consultancy project and business plan group work.
- Dissertation on Social Network Analysis and centrality metrics.
- Prepared a complete Business Plan and prototype as part of group work.
- Individual assignment and research on BigData and its uses in EBusiness, based on Gartner’s hype cycle.
- Developed a simple web server in Java.
- Modified a ticket booking application using Java, Hibernate and Memcache in IntelliJ.
- Developed an Android and PHP application for Newcastle University Alumni Association, which uses Facebook SDK and Twitter API.
- Class representative - Participated in meetings and voiced issues faced by classmates.
- Actively participated in events happening in the school like Start-up Weekend, Thinking Digital Meeting.
- CSC8004 Network and Web Technologies
- CSC3103 Internet Technology and Electronic Commerce
- NBS8207 E‐Business
June 2004 — June 2008 B.Tech Computer Science & Engineering (Merit)
Bachelors degree course that covered in-depth concepts of computer science, mathematics, algorithms and software development. Programming lab in C, Java and Lisp was included in the course to enrich the understanding of subject.
- IEEE and CSI (Computer Society of India) associations
- Secretary of Computer Science Society
- Organised events for Computing Society
- Won programming competitions in C
- Mini project on face detection and pattern recognition
- Main project was a robot that rolled on wheels with voice recognition and response, face detection, obstacle detection and path learning capabilities.
Automate Email Reply
- Use Gmail API to automate reply of emails from recruiters, with the latest copy of my CV as attachment
- Written in Python
- Features include greeting with First name extraction, apology note incase I’m replying the next day, dynamic greeting based on time of day (in-progress)
- Small Reactjs application, with Material Design
- CI/CD with bitbucket and Wercker
- Ruby on Rails app that lists cool places in Bangalore to hangout.
- RESTful API based.
- People can rate and comment on places.
- Places can put up events and other marketing stuff in their pages.
- Work in progress
- Webapp to store and split expenses between group of people
- Using Play framework
- REST API based
- Work in progress
- Using AngularJS.
- Stores score in browser
- Next step: Use Cordova to port this app into mobile.
- Wall Climbing
- Sky Diving
- Blogging - I write about technology in my blog at https://deepumohan.com/tech/