Dedicated Servers
  • Instant
  • Custom
  • Single CPU servers
  • Dual CPU servers
  • Servers with 4th Gen CPUs
  • AMD Ryzen ve Intel Core i9'lu Sunucular
  • Storage Servers
  • Servers with 10Gbps ports
  • Hosting virtualization nodes
  • GPU
  • Sale
  • VPS
    GPU
  • Dedicated GPU server
  • VM with GPU
  • Tesla A100 80GB & H100 Servers
  • Sale
    Apps
    Cloud
  • VMware and RedHat's oVirt Сlusters
  • Proxmox VE
  • Colocation
  • Colocation in the Netherlands
  • Remote smart hands
  • Services
  • Intelligent DDoS protection
  • Network equipment
  • IPv4 and IPv6 address
  • Managed servers
  • SLA packages for technical support
  • Monitoring
  • Software
  • VLAN
  • Announcing your IP or AS (BYOIP)
  • USB flash/key/flash drive
  • Traffic
  • Hardware delivery for EU data centers
  • About
  • Careers at HOSTKEY
  • Server Control Panel & API
  • Data Centers
  • Network
  • Speed test
  • Hot deals
  • Sales contact
  • Reseller program
  • Affiliate Program
  • Grants for winners
  • Grants for scientific projects and startups
  • News
  • Our blog
  • Payment terms and methods
  • Legal
  • Abuse
  • Looking Glass
  • The KYC Verification
  • Hot Deals

    05.12.2022

    WindowsPE Live-CD in Jenkins/Foreman Infrastructure

    server one
    HOSTKEY
    Rent dedicated and virtual servers with instant deployment in reliable TIER III class data centers in the Netherlands and the USA. Free protection against DDoS attacks included, and your server will be ready for work in as little as 15 minutes. 24/7 Customer Support.

    Author: Hostkey DevOps Team Leader Egor Garadzha

    We have already covered the setup of the Centos/RockyLinux based LiveCD build kitchen for Linux that we use at Hostkey. It's time to talk about building a WindowsPE distribution in a Linux infrastructure, automating this process using Jenkins, and deploying MS Windows-based systems through this helper.

    Traditionally, PXE-deploy Windows is implemented as part of a common Microsoft infrastructure, including DHCP / DNS / TFTP and other services. In our infrastructure, the host server for deploying the OS is Foreman, and it seemed beside the purpose to deploy the deployment environment separately for Microsoft from the very beginning.

    Many years ago, we chose the path of downloading a WindowsPE wim image via iPXE. The image was manually assembled from Microsoft infrastructure. One person was watching the kitchen. As the company grew, and along with it the complexity of it tasks, we decided to automate the assembly and image management processes as much as possible. At the same time, we faced the task of supporting UEFI installations, which turned out to be not as easy as one might assume.

    Automation of Image preparation

    The first need was to get rid of the Microsoft infrastructure for imaging. To this end, we decided to use the old wimtools project. In this case, its support for RedHat systems is at an advanced enough level, and so we got a ready-made SPEC.

    This set of console utilities allows you to work with wim images and modify them. To prepare the WindowsPE image, we need:

    • Vanilla Windows distribution (for example, Windows 10);
    • The mkwinpeimg utility from wimtools;
    • A starting cmd script for WindowsPE.

    During the build process, you can overlay and add an arbitrary directory to the set of scripts and software you need. We added to WinPE the latest version of PowerShell from Github Microsoft, a set of scripts for installing the OS, and a number of utilities that we also needed to interact with Foreman.

    Build command example:

    mkwinpeimg --iso --windows-dir=/mnt/windowsrepo --overlay=./PEAddons --start-script=./startpe.cmd windowspe-${BUILD_NUMBER}.iso

    The path /mnt/windowsrepo is for the loop-mounted ISO image of the vanilla version of Windows 10.

    After long hours of work on the image, we have formed the following assembly kitchen, more or less:

    • The Jenkins task for building wimtools-rpm (we discussed earlier how you can organize basic work with packages in the infrastructure);

    • The Jenkins task to build and deliver the ISO image to the WindowsPE central repository;

    • The Git repository, containing the main set of scripts and small binaries for working inside the image. The same startpe.cmd and the PEAddons directory, inside which we have created a directory for PowerShell, a bin for small utilities like Go tool written by ourselves for working with “syslog” and “scripts” - for additional scripts, and so on.

    • We have also placed the repository for the windows_driver_store device drivers into Git for versioning, and any problems with large binaries are solved using LFS.

    • Our internal mirror with vanilla distributions, where the donor ISO is drawn from to create the PE image. The same repository had already been used while installing the OS.

    Everything except the ISO mirror is stored and versioned in Git. Needless to say, it is impossible to understate the importance and convenience of this if more than one person works on the infrastructure.

    If you have everything you need in Git, and a set of Linux build utilities, the task can be easily automated in Jenkins:

    #!/usr/bin/env bash
    
    ssh-agent bash -c "ssh-add $GITLAB_KEY; git clone -b $short_branch [email protected]:windows_driver_store.git PEAddons/drivers" 
    wget -nv https://github.com/PowerShell/PowerShell/releases/download/v$PWSHVER/PowerShell-$PWSHVER-win-x64.zip
            
    unzip -qqo PowerShell-$PWSHVER-win-x64.zip -d PEAddons/pwsh/ || true
    rm -rf PowerShell-$PWSHVER-win-x64.zip PEAddons/drivers/.git
    sed "1aecho Starting windowspe-${BUILD_NUMBER}" -i ./startpe.cmd
    
    mkwinpeimg --iso --windows-dir=/mnt/windowsrepo --overlay=./PEAddons --start-script=./startpe.cmd windowspe-${BUILD_NUMBER}.iso

    Scripts and linking with Foreman

    An important task when performing the deployment of a system like this is the organization of the image itself, in which control over its behavior will be transferred to an external service. This kind of solution will eliminate the need to rebuild the image for each individual installation.

    According to the WindowsPE standard, the start script should be startpe.cmd. This is, in fact, a regular bat file with all the limitations of the standard Microsoft shell. We use it to start PowerShell scripts.

    The first and main script is the so-called getforeman.ps1 script, which is launched by startpe.cmd (code below: PowerShell).

    The main problem when organizing a Windows deployment by drawing from its ISO image via iPXE is that you cannot throw any parameters into the image; that is, the image inside must receive enough information to then fetch the Foreman scripts. The scripts themselves are generated during the deployment process and are available via links such as the following:

    http://foreman.example.com/unattended/provision?token=xxxxxxxx

    So, we need a host token. It can be obtained from Foreman itself if you know the host name, which is provided, including to via dhcp host itself.

    To get its name in the CLI, we use the dhcptest.exe utility:

    $dhcphostname = ( & $env:SystemDrive\bin\dhcptest.exe --mac $netinfo.macaddress --query --print-only 12[str] --quiet )

    Next, we get the host token by making a request to Foreman through a user with view permissions:

    $creds = "$($user):$($foremanpass.$location)"
    $uri = $foremanuri.$location
    
    $encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($creds))
    
    $basicAuthValue = "Basic $encodedCreds"
    
    $Headers = @{
      Authorization = $basicAuthValue
    }
    
    $foremanhostinfo = @{ $location=$(Invoke-RestMethod -Uri "$uri/api/hosts/$dhcphostname" -Headers $Headers -SkipCertificateCheck) }
    $token = $foremanhostinfo.$location.token

    Using the token, we can request all the necessary provision scripts directly from Foreman, and the task is complete.

    Further tasks are already connected with the organization of the installation procedure on Foreman. This is also quite an extensive topic, especially considering UEFI deployment, which we hope to dedicate a separate article to.

    An important question in such a system remains the issue of security. Foreman’s address, the restricted user for interacting with it, and other parameters have to be built directly into the script for accessing information and, if any of these change, the image has to be rebuilt. If the issue of convenience is resolved by assembly, then security is a subtler matter. Indeed, it was security issues that made us look towards more complex installation options that exclude WindowsPE in the first place. However, in non-public infrastructures which require Windows but don't require a large Microsoft infrastructure to deploy, performing your systems deployment through WindowsPE can be a good option.

    Rent dedicated and virtual servers with instant deployment in reliable TIER III class data centers in the Netherlands and the USA. Free protection against DDoS attacks included, and your server will be ready for work in as little as 15 minutes. 24/7 Customer Support.

    Other articles

    23.05.2024

    The Capabilities of Large Language Models: Hacking or Helping?

    Exploring the Capabilities of LLM Agents: A Study on Website Hacking

    01.05.2024

    Checklist: 10 Points to Choose a Hosting Service That Won't Let You Down

    How to choose the best hosting for your projects? We have prepared a checklist of 10 questions you should ask the hosting service that will help you make a decision.

    17.04.2024

    How to choose the right server with suitable CPU/GPU for your AI?

    Let's talk about the most important components that influence the choice of server for artificial intelligence.

    04.04.2024

    VPS, Website Hosting or Website Builder? Where to host a website for business?

    We have compared website hosting options, including VPS, shared hosting, and website builders.

    15.03.2024

    How AI is fighting the monopoly in sports advertising with GPUs and servers

    AI and AR technologies allow sports advertising to be customized to different audiences in real time using cloud-based GPU solutions.

    HOSTKEY Dedicated servers and cloud solutions Pre-configured and custom dedicated servers. AMD, Intel, GPU cards, Free DDoS protection amd 1Gbps unmetered port 30
    4.3 67 67
    Upload