Home Lab – scripted installation – Part I

Posted on by admin

I was fascinated by VCF, VLC, and other scripts, including those provided by William Lam. I decided to create my own script using PowerShell, which was a lot of fun and helped me develop a deeper understanding.

I am hoping that this is the first part of many articles to come. I would like to create a script that would set up the environment for vRA with just one click, without the need to download any additional elements. I want everything to be included in the script.

The first part is to prepare a prereqVM that will work as PXE, TFTP, DHCP, NTP, HTTP and DNS. And then create 4 esxi hosts with PXE

variables we need to specify

####################################################
#	variables
####################################################

#MyVMware Credentials
$myUsername = ""
$myPassword = ""

#Host
$localPath = "C:\HomeLabFull"

#ESXI
$esxiHost = "192.168.100.100"
$esxiU = "root"
$esxiP = "VMware1!"

$esxiDatastore = "OS-Data"
$networkName = "VM Network"
$vmPortGroup = "HomeLab"


#jumper
$jumpName = "prereqVM"
$prereqVmIp = "192.168.100.180"
$prereqVmNet = "24"
$prereqVmGw = "192.168.100.4"
$prereqVmDns = "8.8.8.8"

$prereqVmU = "root"
$prereqVmP = "changeme"
$prereqVmPassword = ConvertTo-SecureString $prereqVmP -AsPlainText -Force
$credentialsPrereqVM = New-Object System.Management.Automation.PSCredential($prereqVmU, $prereqVmPassword)

#Global
$globalIPOctet = "192.168.1"

#esxi_nested
$ssdstore_size_GB = "60"
$osstore_size_GB = "20"
$vSphereISO = "VMware-VMvisor-Installer-7.0b-16324942.x86_64.iso"

I’m using a standard logger

# Specify log file path
$logPath = "$localPath\logs\script.log"

# Create log file if it doesn't exist
if (-not (Test-Path $logPath)) {
    New-Item -ItemType File -Path $logPath -Force | Out-Null
}

function Write-Log {
    param (
        [Parameter(Mandatory=$true)]
        [string]$Message,
        [ValidateSet('INFO', 'WARNING', 'ERROR')]
        [string]$Level = 'INFO'
    )
    $FormattedDate = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    $LogMessage = "[ $FormattedDate ] [$Level] $Message"
    Add-Content -Path $logPath -Value $LogMessage
    Write-Host $LogMessage
}

as prerequisites, I use 3 modules that are automatically installed, so it is very important that we run the entire script with administrator privileges

# Check if NuGet package provider is installed
if ((Get-PackageProvider -Name NuGet -ErrorAction SilentlyContinue) -eq $null) {
    try {
        Install-PackageProvider -Name NuGet -Force -Confirm:$false
        Write-Log -Level INFO -Message "NuGet package provider installed successfully"
    } catch {
        Write-Log -Message "Error installing NuGet package provider: $_" -Level ERROR
    }
} else {
    Write-Log -Message "NuGet package provider is already installed" -Level INFO
}

# Check if Posh-SSH module is installed
if ((Get-Module -Name Posh-SSH -ListAvailable -ErrorAction SilentlyContinue) -eq $null) {
    try {
        Install-Module -Name Posh-SSH -Scope AllUsers -Force -Confirm:$false 
        Write-Log -Level INFO -Message "Posh-SSH module installed successfully"
    } catch {
        Write-Log -Message "Error installing Posh-SSH module: $_" -Level ERROR
    }
} else {
    Write-Log -Message "Posh-SSH module is already installed" -Level INFO
}

# Check if VMware PowerCLI module is installed
if ((Get-Module -Name VMware.PowerCLI -ListAvailable -ErrorAction SilentlyContinue) -eq $null) {
    try {
        Install-Module -Name VMware.PowerCLI -Scope AllUsers -Force -Confirm:$false 
        Write-Log -Level INFO -Message "VMware PowerCLI module installed successfully"
    } catch {
        Write-Log -Message "Error installing VMware PowerCLI module: $_" -Level ERROR
    }
} else {
    Write-Log -Message "VMware PowerCLI module is already installed" -Level INFO
}

Two tools that I need at the beginning but I think also in later parts are ovf tool and VCC (VMware Customer Connect CLI)

# Download VMware Customer Connect CLI
$vccSDK = "https://github.com/vmware-labs/vmware-customer-connect-cli/releases/download/v1.1.4/vcc-windows-v1.1.4.exe"

# Check if File Downloader Exist 
if (Test-Path "$localPath\vcc.exe") {
    Write-Log -Message "File exists!"
} else {
    Write-Log -Message "File does not exist. Downloading a file." -Level INFO
    try {
        Invoke-WebRequest $vccSDK -OutFile "$localPath\vcc.exe"
        Write-Log -Message "Download successful." -Level INFO
    } catch {
        Write-Log -Message "Download failed." -Level ERROR
        Write-Log -Message $_.Exception.Message -Level ERROR
    }
}
# Download OVF Tool
$ovfSDK = "https://github.com/rgl/ovftool-binaries/raw/main/archive/VMware-ovftool-4.5.0-20459872-win.x86_64.zip"
$ovfDestinationPath = "C:\Program Files\VMware\VMware OVF Tool\"

# Check if OVF Tool is already downloaded and extracted
if (Test-Path "$ovfDestinationPath\ovftool.exe") {
    Write-Log "OVF Tool is already downloaded and extracted to '$ovfDestinationPath'."
} else {
    Write-Log "OVF Tool is not downloaded or extracted."
    Write-Log "Downloading OVF Tool."
    
    if (Test-Path "$localPath\VMware-ovftool-4.5.0-20459872-win.x86_64.zip") {
        Write-Log "OVF Tool archive file exists!"
    } else {
        Write-Log "OVF Tool archive file does not exist."
        Write-Log "Downloading OVF Tool archive file."
        Invoke-WebRequest $ovfSDK -OutFile "$localPath\VMware-ovftool-4.5.0-20459872-win.x86_64.zip"
        Write-Log "OVF Tool archive file downloaded."
    }

    if (Test-Path "$ovfDestinationPath\ovftool\ovftool.exe") {
        Write-Log "OVF Tool is already extracted to '$ovfDestinationPath'."
    } else {
        Write-Log "Extracting OVF Tool to '$ovfDestinationPath'."
        Expand-Archive "$localPath\VMware-ovftool-4.5.0-20459872-win.x86_64.zip" -DestinationPath $ovfDestinationPath 
        Write-Log "OVF Tool extracted to '$ovfDestinationPath'."
    }
}

Once we have everything, we proceed to create prereqMV. I won’t copy all the code here because it will be available on GIT

The script first checks if the PhotonOS file already exists in the specified location, and if not, downloads it from the specified URL.

Next, the script creates a cloud-config file that specifies the configuration for the PhotonOS VM. This includes the hostname, fully qualified domain name, timezone, network settings, and commands to run on startup. The cloud-config file is then compressed and encoded as base64 for use as guestinfo data in the PhotonOS VM.

The script then uses the VMware OVF Tool to deploy the PhotonOS VM to the specified ESXi host and datastore. The cloud-config data is injected into the VM using the guestinfo.userdata and guestinfo.userdata.encoding options. The script waits for cloud-init to complete and the VM to be fully deployed and configured before disconnecting from the vSphere server.

This is very importand stuff as on ESXI we are not able to use all ovf properties and I don’t wanna use vcenter as not all of you have in your llabs

$cloud_config = @"
#cloud-config
hostname: $jumpName
fqdn: $jumpName
timezone: Europe/Berlin
write_files:
  - path: /etc/systemd/network/10-eth0-static-en.network
    permissions: 0644
    content: |
      [Match]
      Name=eth0

      [Network]
      Address=$prereqVmIp/$prereqVmNet
      Gateway=$prereqVmGw
      DNS=$prereqVmDns
runcmd: 
  - systemctl restart systemd-networkd
  - tdnf update -y
bootcmd:
  - /bin/sed -E -i 's/^root:([^:]+):.*$/root:\1:18947:0:99999:0:::/' /etc/shadow
"@


$bytes = [System.Text.Encoding]::UTF8.GetBytes($cloud_config)
$compressedBytes = [System.IO.MemoryStream]::new()
$gzipStream = [System.IO.Compression.GzipStream]::new($compressedBytes, [System.IO.Compression.CompressionMode]::Compress)
$gzipStream.Write($bytes, 0, $bytes.Length)
$gzipStream.Dispose()
$userDataBase64 = [System.Convert]::ToBase64String($compressedBytes.ToArray())

C:\'Program Files'\VMware\'VMware OVF Tool'\ovftool\ovftool.exe  --noSSLVerify --acceptAllEulas --X:injectOvfEnv --allowExtraConfig --network=”$networkName” `-ds="$esxiDatastore" -n="$jumpName" "$localPath\$filename" vi://"$esxiU":"$esxiP"@$esxiHost

Connect-VIServer $esxiHost -User $esxiU -Password $esxiP

$vm = Get-VM -Name $jumpName

$vm.ExtensionData.Config.ExtraConfig += New-Object VMware.Vim.OptionValue -Property @{Key="guestinfo.userdata";Value=$userDataBase64}
$vm.ExtensionData.Config.ExtraConfig += New-Object VMware.Vim.OptionValue -Property @{Key="guestinfo.userdata.encoding";Value="gzip+base64"}
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$spec.ExtraConfig = $vm.ExtensionData.Config.ExtraConfig
$vm.ExtensionData.ReconfigVM($spec)

As our machine already exists, we still need to separate our lab, which is why I use protgroup

The script first connects to the vSphere server using the Connect-VIServer command and retrieves all virtual switches associated with the specified ESXi host.

Next, the script checks if the specified port group already exists on the virtual switch using the Get-VirtualPortGroup command. If the port group does not exist, the New-VirtualPortGroup command is used to create a new port group with the specified name and associate it with the virtual switch.

The script then waits for the port group to become accessible using a loop that checks for the port group’s existence at regular intervals. Once the port group is accessible, the script assigns it to the specified VM using the New-NetworkAdapter command and waits for the assignment to complete before logging a success message.

####################################################
#               Create PortGroup on ESXI
####################################################
Connect-VIServer $esxiHost -User $esxiU -Password $esxiP

# Get all virtual switches
$virtualSwitches = Get-VirtualSwitch -VMHost $esxiHost -Standard
$vm = Get-VM -Name $jumpName

# Loop through the virtual switches
foreach ($vswitch in $virtualSwitches) 
{
    # Check if the virtual switch has at least one port group and one NIC
    if ($vswitch.ExtensionData.Portgroup.Count -gt 0 -and $vswitch.Nic.Count -gt 0) 
    {
        if (-not (Get-VirtualPortGroup -Name "$vmPortGroup" -ErrorAction SilentlyContinue)) {
            New-VirtualPortGroup -Name "$vmPortGroup" -VirtualSwitch $vswitch -Confirm:$false
        } else {
            Write-Log "The port group $vmPortGroup already exists."
        }
        
        # Wait for the port group to become accessible
        $portGroup = Get-VirtualPortGroup -Name "$vmPortGroup" -VirtualSwitch $vswitch -ErrorAction SilentlyContinue
        $timeout = 30  # seconds
        if ($portGroup -eq $null) 
        {
            do{
                $portGroup = Get-VirtualPortGroup -Name "$vmPortGroup" -VirtualSwitch $vswitch -ErrorAction SilentlyContinue
                if ($portGroup -eq $null) 
                {
                    Start-Sleep -Seconds 1
                }
            } until (($portGroup -ne $null) -or ((Get-Date) - $startTime).TotalSeconds -ge $timeout)
        }
        
        if ($portGroup -eq $null) 
        {
            $errorMessage = "Failed to create port group '$portGroup' on virtual switch '$($vswitch.Name)' within $timeout seconds."
            Write-Log -Level ERROR -Message $errorMessage
            Write-Host $errorMessage
        } else 
        {
            $successMessage = "Port group '$portGroup' was created successfully on virtual switch '$($vswitch.Name)'."
            Write-Log -Level INFO -Message $successMessage
            Write-Host $successMessage

            # Assign the port group to the VM

            New-NetworkAdapter -VM $vm -NetworkName "$vmPortGroup" -StartConnected
            $adapter = Get-NetworkAdapter -VM $vm -Name "$vmPortGroup" -ErrorAction SilentlyContinue
            $timeout = 30  # seconds
            if ($adapter -eq $null) 
            {
            do{
                    $adapter = Get-NetworkAdapter -VM $vm -Name "$vmPortGroup" -ErrorAction SilentlyContinue
                    if ($adapter -eq $null) 
                    {
                        Start-Sleep -Seconds 1
                    }
                } until (($adapter -ne $null) -or ((Get-Date) - $startTime).TotalSeconds -ge $timeout)
            }
            Write-Log "Port group '$vmPortGroup' was assigned successfully to VM '$vmName'."
        }
    }
}
# Disconnect from the vSphere server
Disconnect-VIServer -Server $esxiHost -Confirm:$false

as we have portgrupe, we set everything on the machine

  • Static IP
  • Disabling the Firewall
  • Install modules (dhcp-server atftp syslinux wget tar ntp)
  • Configure DHCP
$dhcpConf = @"
subnet $globalIPOctet.0 netmask 255.255.255.0 {
  range $globalIPOctet.100 $globalIPOctet.200;
  option routers $globalIPOctet.2;
  option domain-name-servers 8.8.8.8, 8.8.4.4;
  filename "pxelinux.0";
}
"@
$dhcpConf = $dhcpConf -replace "`r`n", "`n"
$output = Invoke-SSHCommand -SessionId $session.SessionId -Command "echo '$dhcpConf' > /etc/dhcp/dhcpd.conf"
  • Configure TFTP
$tftpdConf = @"
[Unit]
Description=Advanced TFTP server
After=network.target

[Service]
Type=simple
ExecStart=/usr/sbin/atftpd --user tftp --group tftp --daemon --no-fork --bind-address $globalIPOctet.2 /var/lib/tftpboot
Restart=always

[Install]
WantedBy=multi-user.target
"@
$tftpdConf = $tftpdConf -replace "`r`n", "`n"
$output = Invoke-SSHCommand -SessionId $session.SessionId -Command "echo '$tftpdConf' > /usr/lib/systemd/system/atftpd.service"

Then we need to prepare PXE, here is the thing that I like the least because to install ESXi with PXE we need to have syslinux in the appropriate version, which unfortunately is not so easy to achieve now, so you need a package of files that are available on my GIT so that everyone can do it download. Here you can find the link (https://github.com/vWorldLukasz/vmware/raw/main/vmware.tar)

We also download the Esxi iso and put it on our prereq VM

$downloadvSphere = "$localPath\vcc.exe download -p vmware_vsphere -s esxi -v 7.* -f $vSphereISO --accepteula --user ""$myUsername"" --pass ""$myPassword"" --output ""$localPath\"""

change the files with sed

sed -i 's/\///g' /var/lib/tftpboot/images/esx/boot.cfg
sed -i 's/prefix=/prefix=\/images\/esx\//' /var/lib/tftpboot/images/esx/boot.cfg

and create a default file for PXE

$esxBoot = @"
default menu.c32
prompt 0
timeout 300

menu title PXE Boot Menu
menu color border 0 #00000000 #00000000 none

label ESXI-Install
    menu label ESXI
    COM32 pxechn.c32
    kernel images/esx/mboot.c32
    APPEND -c images/esx/boot.cfg ks=http://$globalIPOctet.2/ks.cfg
    IPAPPEND 2
LABEL hddboot
    LOCALBOOT 0x80
    MENU LABEL ^Boot from local disk
"@
$esxBoot = $esxBoot -replace "`r`n", "`n"




# Log the start of the task
Write-Log -Message "Starting to setup PXE environment" -Level INFO

# Set default boot options and copy necessary files
$output = Invoke-SSHCommand -SessionId $session.SessionId -Command "echo '$esxBoot' > /var/lib/tftpboot/pxelinux.cfg/default"

We install HTTPD to place the kickstart file on it

It uses unbound for the DNS service

$dnsFile = @"

server:
    interface: $globalIPOctet.2
    port: 53
    do-ip4: yes
    do-udp: yes
    access-control: $globalIPOctet.0/24 allow
    verbosity: 1

local-zone: "vworld.domain.lab." static
local-data: "hl-esxi1.vworld.domain.lab A $globalIPOctet.10"
local-data: "hl-esxi2.vworld.domain.lab A $globalIPOctet.11"
local-data: "hl-esxi3.vworld.domain.lab A $globalIPOctet.12"
local-data: "hl-esxi4.vworld.domain.lab A $globalIPOctet.13"
local-data-ptr: "$globalIPOctet.10 hl-esxi1.vworld.domain.lab"
local-data-ptr: "$globalIPOctet.10 hl-esxi2.vworld.domain.lab"
local-data-ptr: "$globalIPOctet.10 hl-esxi3.vworld.domain.lab"
local-data-ptr: "$globalIPOctet.10 hl-esxi4.vworld.domain.lab"

forward-zone:
   name: "."
   forward-addr: 8.8.4.4
   forward-addr: 8.8.8.8
"@

$dnsFile = $dnsFile -replace "`r`n", "`n"

$output = Invoke-SSHCommand -SessionId $session.SessionId -Command "echo '$dnsFile' > /etc/unbound/unbound.conf"

and configures NTP, I have Polish addresses set, but I think that I will add some editing functionality

$ntpFile = @"

server 0.pl.pool.ntp.org
server 1.pl.pool.ntp.org
server 2.pl.pool.ntp.org
server 3.pl.pool.ntp.org
"@

$ntpFile = $ntpFile -replace "`r`n", "`n"

$output = Invoke-SSHCommand -SessionId $session.SessionId -Command "echo '$ntpFile' >> /etc/ntp.conf"

the last element of the whole script is the loop installation of 4 ESXi hosts that will be included in the cluster in the future

####################################################
#               ESXI - nested hosts - SETUP
####################################################

# Connect to the ESXi host
# Test connection to ESXi host
if (Test-Connection $esxiHost -Count 1 -Quiet) {
    Write-Log "Successfully tested connection to $esxiHost." -Level INFO
} else {
    Write-Log "Failed to connect to $esxiHost. Ensure that it is reachable and try again." -Level ERROR
    return
}

try {
    # Connect to the ESXi host
    Connect-VIServer $esxiHost -User $esxiU -Password $esxiP -ErrorAction Stop

    # Log success
    Write-Log "Successfully connected to $esxiHost." -Level INFO
} catch {
    # Log error
    Write-Log "Failed to connect to $esxiHost. $($Error[0].Exception.Message)" -Level ERROR
    return
}
$vmDatastore = Get-Datastore $esxiDatastore

$session = New-SSHSession -ComputerName $prereqVmIp -Credential $credentialsPrereqVM  -AcceptKey -Force
if ($session) {
    Write-Log "SSH session successfully established."
} else {
    Write-Log "Failed to establish SSH session." -Level ERROR
}

# Define the range of numbers to iterate over
$start = 10
$end = 13
for ($i = $start; $i -le $end; $i++) {

##################################################################################################################################
$kickStart=@"
vmaccepteula
install --firstdisk --overwritevmfs --novmfsondisk

network --bootproto=static --device=vmnic0 --ip=$globalIPOctet.$i --netmask=255.255.255.0 --gateway=$globalIPOctet.1 --hostname=hl-esxi$i.vworld.domain.lab --nameserver=$globalIPOctet.2
rootpw VMware1!


reboot

%firstboot --interpreter=busybox

# enable VHV (Virtual Hardware Virtualization to run nested
grep -i "vhv.enable" /etc/vmware/config || echo "vhv.enable = \"TRUE\"" >> /etc/vmware/config
 

# Enable SSH
vim-cmd hostsvc/enable_ssh
vim-cmd hostsvc/start_ssh

#disable ipv6
esxcli network ip set --ipv6-enabled=false

# Enable ESXi Shell
vim-cmd hostsvc/enable_esx_shell
vim-cmd hostsvc/start_esx_shell

# Suppress Shell warning
esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1

# NTP
esxcli system ntp set -s $globalIPOctet.2
esxcli system ntp set -e 1

# enter maintenance mode
esxcli system maintenanceMode set -e true

# Needed for configuration changes that could not be performed in esxcli
esxcli system shutdown reboot -d 60 -r "rebooting after host configurations"

"@

$kickStart = $kickStart -replace "`r`n", "`n"

$output = Invoke-SSHCommand -SessionId $session.SessionId -Command "echo '$kickStart' > /etc/httpd/html/ks.cfg"
# Log the output of the command
if ($output.Output) {
    Write-Log -Message $output.Output -Level INFO
}
if ($output.Error) {
    Write-Log -Message $output.Error -Level ERROR
}

# Clear the $output variable
$output = $null

$output = Invoke-SSHCommand -SessionId $session.SessionId -Command "chmod -R 755 /etc/httpd/html/"
# Log the output of the command
if ($output.Output) {
    Write-Log -Message "Successfully set permissions for HTTP server directory." -Level INFO
}
if ($output.Error) {
    Write-Log -Message "Failed to set permissions for HTTP server directory. Error: $($output.Error)" -Level ERROR
}

# Clear the $output variable
$output = $null
##################################################################################################################################


# Define virtual machine parameters
$vmName = "hl-esxi$i"

$vmGuestOS = "vmkernel7Guest"
$vmCpuCount = 2
$vmMemoryGB = 48

# Create new virtual machine
Write-Log "Creating virtual machine '$vmName'..."
New-VM -Name $vmName -Datastore $vmDatastore -DiskStorageFormat Thin -DiskGB $osstore_size_GB -MemoryGB $vmMemoryGB  -NumCpu $vmCpuCount -CD -GuestId $vmGuestOS  -Confirm:$false



$vm = Get-VM -Name $vmName
Remove-NetworkAdapter -NetworkAdapter (Get-NetworkAdapter -VM $vm) -Confirm:$false

Start-Sleep -Seconds 30

# Create first SSD on HBA #3
$New_Disk1 = New-HardDisk -vm $vm -CapacityGB $ssdstore_size_GB -StorageFormat Thin -datastore $vmDatastore


# Add one more SSD on HBA #3
$New_Disk2 = New-HardDisk -vm $vm -CapacityGB $ssdstore_size_GB -StorageFormat Thin -datastore $vmDatastore


$ExtraOptions = @{
	"disk.EnableUUID"="true";
    "scsi0:2.virtualSSD" = "1";
    "scsi0:3.virtualSSD" = "1";
}

$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec

Foreach ($Option in $ExtraOptions.GetEnumerator()) {
    $OptionValue = New-Object VMware.Vim.optionvalue
    $OptionValue.Key = $Option.Key
    $OptionValue.Value = $Option.Value
    $vmConfigSpec.extraconfig += $OptionValue
}

$vmview=get-vm $vmName | get-view
$vmview.ReconfigVM_Task($vmConfigSpec)


# Disable EFI Secure Boot
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$bootOptions = New-Object VMware.Vim.VirtualMachineBootOptions
$bootOptions.EfiSecureBootEnabled = $false
$spec.BootOptions = $bootOptions
$vm.ExtensionData.ReconfigVM($spec)


# Change firmware type to BIOS
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$spec.Firmware = [VMware.Vim.GuestOsDescriptorFirmwareType]::bios
$vm.ExtensionData.ReconfigVM($spec)

New-NetworkAdapter -VM $vm -NetworkName "$vmPortGroup" -StartConnected:$true -Type "vmxnet3"
Start-Sleep -Seconds 30
Start-VM -VM $vm

do{

    Write-Host "Waiting for deployment"
    $vm = Get-VM -Name $vmName
    Start-Sleep -Seconds 20

}until($vm.Guest.HostName -match $vmName)

}
Disconnect-VIServer -Confirm:$false

and that would be it for the first part, in the second I hope to be able to deploy vcenter and router (I’m thinking about pfsense or opnsense) but everything will depend on how difficult it will be to automate.

Share with:


Leave a Reply

Your email address will not be published. Required fields are marked *