Report on Integrated BigData Infrastructure I
Last updated on:
FINSEC Big Data Infrastructure aims to provide a complete stack comprising an infrastructure management system to support the analysis and prediction algorithms of the FINSEC platform, managing the historical and real data of the security, the knowledge base and the description of the assets, being completely scalable, adaptable to runtime and high performance.
In this way, FINSEC Big Data Infrastructure will respond to the needs of big data operations and data intensive applications. The system will base all infrastructure management decisions on data analysis, monitoring data from implementations and the logic derived from data operations that govern and influence storage, compute and network resources, as well as their interdependencies. In addition to the infrastructure management system, "Data as a Service" will be offered to data providers, decision makers, private and public organizations. Approaches to data cleansing, data layout and efficient storage, combined with seamless data analysis, will be holistically implemented across multiple data stores and locations.
In order to provide the information required for better management of the infrastructure, FINSEC Big Data Infrastructure will provide a set of basic services, such as the CRUD DB interface, which facilitates the analysis and sizing of data-driven applications in terms of service forecasting required data, their interdependencies with the micro-services application and the underlying resources required. In addition, a data toolkit will allow data scientists to assimilate their data analysis functions and specify their preferences and constraints, which will be exploited by the infrastructure management system for resources and data management. Finally, a process modelling framework will be provided to enable function-based modelling of processes, which will be mapped to an automated analysis of the process of a concrete technical level.
This deliverable describes the key functionalities of the general architecture, the interactions between the main building blocks and their components, while providing a first version of the interiors of these components regarding the research approaches to be implemented during the project.