Enhanced Platform for Artificial Intelligence – Data Engineering for AI

Many companies aim to gain a competitive advantage through data-driven decisions and digital business models. Significant challenges in achieving these goals, according to a survey by the Enterprise Strategy Group, include the increasing number of systems, a multitude of different data sources and applications, the ever-growing volume of data, and the integration of artificial intelligence (AI).

The implementation of new business and optimisation potentials therefore requires a flexible and powerful platform that addresses these requirements in a reliable way. The relevance for practice can be supported by statistics (cf. Figure 1 and 2). This clearly shows that modern technologies are still not in widespread use. Also, many valuable data sources like sensors, networks, and marketplaces are currently underutilized.

To enable such a platform for agile development, handling large data pools, and dynamic scalability, modern technology approaches like native cloud capability, streaming, and data lineage are essential. In addition to technical aspects, a high degree of user-friendliness and seamless integration into development processes are also crucial factors for success. Especially with the increasing interdisciplinary teams in AI-based projects, such a platform needs to support all involved employees with the necessary data and analysis – from the business analyst and data engineer to the data scientist. With this background, a platform model will be presented here that has already proven itself in numerous solutions: the Enhanced Platform for Artificial Intelligence. This platform can be divided into four different layers, as visualised in Figure 3.

…