Devops solution for AI-based financial Recommendation System
Due to the complexity and tough security requirements of AI-based financial recommendation and analytic system Itexus DevOps team made great efforts to setup processes automation and migrate the system between different cloud platforms.
Industry
FinTech
Solution
Machine Learning, NLP, Enterprise
Engagement model
T&M
Effort and Duration
5+ years
Technologies
Python, Django ReactJS, Flask, TensorFlow, Celery, Redis, Azure, AWS, IBM Cloud, Google Cloud, Jenkins.
About the Project
The client is a FinTech startup developing an AI-powered financial analysis and recommendation system for the stock market and wealth management domain. The product is used by tier 1 banks and asset management companies.
The system processes various data feeds such as CRM data, news, documentation archives, client portfolios, and transactions and stock market data and recommends news and documents relevant to the client and his portfolio and helps prioritize the clients according to the stock market events.
DevOps Solution Highlights
Due to the enterprise nature of the project, its complexity, 5+ years duration, security requirements, cloud independence during its course we had to cope with a very wide range of DevOps tasks on all major cloud platforms.
Process Automation
The complexity of the system, the necessity to support and update multiple versions for different clients required full automation of the development and deployment processes with strict version control and release management. During the development process our team successfully completed the following DevOps tasks:
- Setup Continuous Integration and Continuous Delivery (CI/CD) pipeline for Python applications with Jenkins supplemented with automated Unit and Service level tests
- Setup automated code quality analysis check as part of the CI process using Veracode
- Moved all infrastructure to Docker containers
- Setup Database cluster manager
- Setup site-to-site VPN access for the distributed development team
- Implement an automated monitoring system based on Sentry. The development team received automated alerts of any issues (peak memory/cpu usage, data processing delays, errors, service unavailability etc) happening with the production instance.
- Codified the Infrastructure with Terraform (Infrastructure as a code approach)
As a result of the automation, the team could release new software versions much faster, of much better quality, without errors caused by human factors and could significantly reduce the time and money spent on integration/deployment and testing activities.
Migrations Between Cloud Providers
Major cloud providers have startup support programs providing their cloud infrastructure for free for the first 1 or 2 years. Our client could obtain such credits from AWS, Google Cloud, IBM Cloud and Microsoft Azure and he didn’t have to pay a dime for his development infrastructure for 5 years. However this required the development team to keep the system cloud-agnostic and once in 1-2 years migrate up to 30 servers and data (Dev, QA and Production environments) from AWS to Google Cloud, from Google Cloud to IBM Cloud and from IBM Cloud to Microsoft Azure.
We used Docker, Ansible, Cloud Formation to perform this migration.
On-Premises Installation for Enterprise Clients
Enterprise clients required that the system is installed in their own cloud and installation and support is handed over to their own DevOps team. Access to the production environment for our team was restricted.
We created an Installation guide, Ansible Playbook, and Cloud Formation Template allowing the clients to automatically create all infrastructure and install the system and subsequent updates without involving our engineers. We also conducted a few training sessions with screen sharing to train the customer’s DevOps team to use the installation scripts.
All highlights of the project and the development process are described here.
Related Projects
All ProjectsHealthy app for tracking calories and improving eating habits
Healthy app for tracking calories and improving eating habits
- Healthcare
This mobile app designed to be the all in one solution for improving the users’ eating habits. It is an essential tool for losing weight, building muscles and developing an effective diet plan. This healthy app is a simple way for users to reach their healthy eating goals. It includes the possibility of using calorie tracker, macros calculator, carb & protein counter. In addition to that, many healthy recipes available within the app give users some additional insights into how to lose weight.
Smart Contract Development for Cryptocurrency Investment Platform
Smart Contract Development for Cryptocurrency Investment Platform
- Fintech
The added functionality helps mitigate investment risks by encoding complex financial arrangements on the Ethereum blockchain. The delivered solution uses custom tokens for renting, making payments, and managing cryptocurrency portfolios.
Document management system
Document management system
- Enterprise
A user-friendly dashboard which can always be accessed from anywhere using a web browser. It is a Document Management System in which the importance of version control, classification and the accessibility of information such as drawings, certificates and procedures, is evident. Moreover, the system facilitates extensive collaboration between parties because everyone has access to the same central document management system from their own work station. It can also be linked easily to once existing document management system.
Educational Mobile App for CanadaQBank MCCEE QBank
Educational Mobile App for CanadaQBank MCCEE QBank
- Healthcare
- Education
Educational mobile app for the medical students. Allows to access synchronized CanadaQBank MCCEE question bank data across all platforms, makes possible for users to create a test with their customized level of difficulty/subjects, view test history, continue their next test/assess history from another device etc.
HR platform for candidate and recruiter matchmaking
HR platform for candidate and recruiter matchmaking
- Enterprise
This set of web and the mobile-based applications was designed to match candidates with suitable job vacancies. The matching process performed with the help of specified algorithms that can be altered by recruiters at any time.
Platform for improved event organization and management
Platform for improved event organization and management
- E-commerce
The B2B Ticketing System is a unique innovative platform designed to solve the common problems of the French event ticketing market. It provides event organizers such as theatres, operas, cinemas and retailers with an opportunity to provide final consumers with sufficient quality of service for buying tickets online.
Digital insurance marketplace with recommender engine
Digital insurance marketplace with recommender engine
- Fintech
Online marketplace that connects insurance brokers, insurance companies and the end customers. The definite feature of this marketplace is automation of information exchange and key business processes such as customer acquisition, underwriting and claims management. Featuring a recommender engine that is assessing client’s information and suggesting the best fitting insurance plan.
Energy Resources Consumption Management System for Enterprises
Energy Resources Consumption Management System for Enterprises
- IOT
- Enterprise
The system enables to monitor the flow rate and cost of consumed resources (heating, electricity, gas, water) in enterprises, buildings and geographically distributed objects (a group of buildings, region, city), efficiently select suppliers and rates depending on actual and planned consumption. It also allows regulating daily consumption both in manual and automatic modes with the aim of minimizing costs.
Contact Form
We keep all your project information safe and secure. Drop us a line and we"ll get back to you shortly.