Get whitelist.txt from Private Azure Container using URL protected with SAS token

[plyagent00]

Hello,

We use the approach to get the 3rd party software whitelist directly from Azure Public Blob Storage, we want to protect the URL with an SAS token and make our Container private.
Unfortunately, the protected SAS URL doesn't work with WAU and we receive such error: Couldn't reach/find/compare/copy from

Is It possible to use It?

[AndrewDemski-ad-gmail-com]

Hi,
Are you positive that you've set everything on the Azure side correctly?

I just performed a quick test and it works with SAS token and URLs

[AndrewDemski-ad-gmail-com]

I decided to propose an expansion of a function responsible for providing the whitelist of selected IDs.

Link to PR: #586

[plyagent00]

Hi Andrew!

Thanks for your efforts here.
Regarding the issue itself, I think I forgot to mention one important thing.
We are deploying WAU with the use of Intune Apps and from there, we use the following installation command:

"%systemroot%\sysnative\WindowsPowerShell\v1.0\powershell.exe" -noprofile -executionpolicy remotesigned -file "Winget-AutoUpdate-Install.ps1" -Silent -UpdatesInterval BiWeekly -InstallUserContext -StartMenuShortcut -DoNotUpdate -DisableWAUAutoUpdate -UseWhiteList -NotificationLevel SuccessOnly -noClean -stage stage3 -ListPath https://blablabla.blob.core.windows.net/winget-autoupdate-stage3

-ListPath works only with public URL, when we change our container type to private and use the SAS protected URL, we receive the "Couldn't reach/find/compare/copy from" error.

[plyagent00]

Bump ^

[AndrewDemski-ad-gmail-com]

I am not entirely convinced that it should be deployed with ListPath parameter.
there is an AzureBlobURL

Winget-AutoUpdate/Sources/WAU/Winget-AutoUpdate-Install.ps1

Lines 37 to 38 in 3b8d1f5

	.PARAMETER AzureBlobURL
	Set the Azure Storage Blob URL including the SAS token. The token requires at a minimum 'Read' and 'List' permissions. It is recommended to set this at the container level

But I got lost where it is used later and how.

From my perspective the best approach would be defining the full path incl Blob SAS URL+token and putting it in the policy node in registry via separate mechanism.

In my example above i had that long uri pasted to registry and it worked just fine.

[plyagent00]

.PARAMETER ListPath
Get Black/White List from Path (URL/UNC/GPO/Local)

If -ListPath supports URLs, in theory, SAS protected URL to my Private container should work, but It doesn't.
It works only for public ones which are not protected.
I've tried to modify registry keys manually and the result is the same.

[KnifMelti]

Maybe the ? in your path is the culprit?
You can't hav the file name in the PATH (can't see that masked section).

[plyagent00]

This link directs only to container, not to .txt file used for whitelist items.
Schema about SAS URL from MS reference:

[KnifMelti]

I made the function for standard URL:s, but I'm not skilled enough in PS to handle a ? in the path

[KnifMelti]

It's up to the community, someone must code it!

[AndrewDemski-ad-gmail-com]

Okay, so..
What if we split the whole SAS URL into two parts?

• 1st: uri of resource in the container
• 2nd: the token

Then it would be much easier to operate on more than one list?
I am open to suggestions because that would open a path to a new feature which in my mind would be really useful

[KnifMelti]

https://github.com/Romanitho/Winget-AutoUpdate/blob/3b8d1f5ff26cf5bc543a7b1364689fe7bf545442/Sources/WAU/Winget-AutoUpdate/functions/Test-ListPath.ps1

[AndrewDemski-ad-gmail-com]

Eficient, would that be more powershell ..ish if we replace webclient

$wc = New-Object System.Net.WebClient
$wc.OpenRead("$ExternalList").Close() | Out-Null
$dateExternal = ([DateTime]$wc.ResponseHeaders['Last-Modified']).ToString("yyyy-MM-dd HH:mm:ss")

with a little bit of TLS/proxy awareness

[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12;
[System.Net.WebRequest]::DefaultWebProxy = [System.Net.WebRequest]::GetSystemWebProxy();
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials;

and a head request?

$response = Invoke-WebRequest -UseBasicParsing -Uri $someuri -Method Head
$dateExternal = ([DateTime]$response.Headers.'Last-Modified').ToString("yyyy-MM-dd HH:mm:ss")

Some users might be behind company proxies after all.

I wont code more tonight. Maybe tomorrow, with fresh head

[KnifMelti]

This seems to work if $ListPath contains the character ?:

$Listpath = "https://storagesample.blob.core.windows.net/sample-container?v=2023-11-31&sr=b&sig=39Up9jzHkxhUIhFEjEh9594DIxe6cIRCgOVOICGSP%3A377&sp=rcw"
$ListType = "included_apps.txt"

# Test if $ListPath contains the character "?"  (testing for SAS token)
if ($Listpath -match "\?") {
    # Split the URL into two strings at the "?" substring
    $splitPath = $ListPath.Split("`?")

    # Assign the first string (up to "?") to the variable $resourceURI
    $resourceURI = $splitPath[0]

    # Assign the second string (after "?" to the end) to the variable $sasToken
    $sasToken = $splitPath[1]
    
    # Join the parts and add "/$ListType?" in between the parts
    $ExternalList = -join ($resourceURI, "/$ListType`?", $sasToken)
}

Write-Host $resourceURI
Write-Host $sasToken
Write-Host $ExternalList

[KnifMelti]

@AndrewDemski-ad-gmail-com tested your changes too, but here they didn't work.
Tested my change on a regular URL ListPath and it works (skipping it).

@plyagent00 could you replace the file Test-ListPath.ps1 under functions and try this instead (make a copy of the original first and rename this included to Test-ListPath.ps1) with your WAU_ListPath in registry set to the correct resourceURI/sasToken
Test-ListPath.ps1.txt

[plyagent00]

It works dear friends!
The new version of function downloaded the included_apps.txt file with success from SAS protected URL!

[plyagent00]

I spot one more thing which maybe, could be improved, but It's related to the included_apps.txt file itself.
When I locally modified the included_apps.txt file and removed all the package IDs from there.
The data mismatch was not detected and the local file was not overwritten with data contained in the file placed in Azure.
If some user decides, that He doesn't want to use WAU, He will just simply remove all IDs from the local file and WAU will have nothing to do.
In my honest opinion, the file should be overwritten every WAU run to have the most valid app list.
Is it something that we could improve as well to protect ourselves from such users?

[KnifMelti]

A standard user doesn't have write access to that file, only administrators.
This is an enterprise solution building on standard users using the clients.

[plyagent00]

Aaah! My bad, I forget about It.
Thank you very much for your support, I really appreciate It :)
WAU is great!

[KnifMelti]

It works dear friends!
The new version of function downloaded the included_apps.txt file with success from SAS protected URL!

Now as PR #606 - but:

Can't figure out how to install WAU with the parameter -ListPath like:
-ListPath https://storagesample.blob.core.windows.net/sample-container?v=2023-11-31&sr=b&sig=39Up9jzHkxhUIhFEjEh9594DIxe6cIRCgOVOICGSP%3A377&sp=rcw

? and & are the culprits - my knowledge of PS has come to an end (help wanted)!

[KnifMelti]

Escaping characters at installation time for the parameter -ListPath, now explained in the README.md