A background pattern of shaded interlocking triangles

An Infrastructure Architecture system: Low-Level Designs for Automation, Orchestration, Database and Front-Ends

IT Infrastructure Architect

This article is the third in a series on building an Infrastructure Architecture system. The previous two articles discussed the conceptual case for building such a system and explored functions and relationships in a High-Level design.

This article, the first of many Low-Level designs, will present an option for building the components required to realise and operate the system.

Realisation Options

In accordance with System Engineering principles, this realisation is simply one of many options. This option is based upon my experiences of building such systems where the technologies were appropriate for the operating environments.

However, you might choose to realise the High-Level functions and relationships using different technologies better suited to your environment, and that would be entirely appropriate.

Automated Orchestration

Automated Orchestration is the first function to be built as it is required to drive actions in all other parts of the system.

Server

A virtualised Windows Server will be used of any modern supported version:

  • Add a D:\ drive of 32GB to contain artefacts related to the operation of the Automated Orchestration, the configuration of which will be discussed below.
  • The C:\ drive of at least 100GB will be reserved for the operating system and select application binaries
  • Where possible, PowerShell will be used for scripts to access data sources. Failing this, we may consider installing other scripting capabilities including Python and AutoIt.
  • Where PowerShell depends upon Modules for access to particular systems then the Modules will be installed upon this machine on the C:\ drive in an all-user context.
  • Orchestration will be driven by an Orchestration Script, details of which will be discussed below. The Orchestration Script will orchestrate (call in order) the functions of the system.
  • Scheduling of the Orchestration Script, and other scheduling required by the system, will be driven by the Windows Task Scheduler.

File System Structure

Create the following directory structure on D:\ to enable the operations of the system:

D:\InfraArch
D:\InfraArch\Scripts
D:\InfraArch\Logs
D:\InfraArch\Backups
D:\InfraArch\Input
D:\InfraArch\Output

Where:

  • The Logs folder will contain logged output from the operation of the scripts that will assist with troubleshooting and operation of the system.
  • The Scripts folder will contain the scripts and shared modules that make up the system.
  • The Input folder will contain input files that will be consumed by scripts, for example, should the same script need to be targeted at multiple systems or the same system in different modes.
  • The Output folder will contain scripted outputs. This is where the extracted data will be written.
  • The Backups folder will contain previous versions of scripts, input files and exported data that you wish to retain.

Database

The database system will be Microsoft SQL server, served from a shared database server:

  • A database will be created to hold infrastructure architecture information called InfraArch.
  • The database will contain tables to hold extracted data. In general, one table will be created for each data extract. The process for designing the tables will be discussed later in the series.
  • The tables need not be related in the database as relationships can be defined arbitrarily in views and reports.
  • Appropriate AD-integrated roles and security will be set up within the database to secure use; for example, separate groups to delegate admin, read and modify permissions.
  • SSIS (SQL Server Integration Services) will also be available on the server, and be used to load scripted data exports into the database tables in due course.

Front-End

The front hand will be a Microsoft Access 365 database file that can be run on any windows client operating system:

  • The file will be called infra arch front end
  • The file will contain links to the tables in SQL Server. Data will not be stored in access tables in the front end.
  • Relationships between tables will not be formerly defined in the front end unless good reason exists to do so. Instead relationships will be defined arbitrarily using the Microsoft Access query function in order to build ad-hoc views.
  • Calculated fields and string processing will be used to automatically create scripting based upon field values.
  • Automation of functionality will be implemented through use of forms and VBA scripting.
  • Where scripted exports do not exist for certain data types then facility will be provided to manually load data into the database using import functions and copy paste from a spreadsheet. This will enable integration of data from arbitrary sources.

Automated Reporting

Automated reporting will be provided by SQL Server reporting services:

  • sQL Server reporting services will be installed on a SQL Server.
  • the developer of the system will have access to necessary Tooling required to build SQL Server reports. This will be Microsoft Visual Studio configured with ssrs capability.
  • Views required to inform reporting will be created and stored within the database.
  • Reports will consume the views. This ensures the business logic that informs views is retained within the database rather than being defined arbitrarily within reports to promote reuse of business logic in a consistent manner.
  • Reports will be scheduled to run in an automated fashion and distributed to relevant stakeholders without human intervention.

Orchestration Script

Create your first PowerShell script at: D:\InfraArch\Scripts\InfraArchOrchestrator.ps1 to call your source data extractions.

For now, this script will be empty, and we will add to it in later articles in this series.

Schedule the for an appropriate time for data extracts to run in your environment; perhaps every weekday morning at 03:00. Disable the schedule for now and enable it once at least one activity is added.

Next Steps

The next article in this series will explore a Low-Level design for the data pipeline.

You should also read: