Redirect output to file in PowerShell
I share a snippet of code that I execute in my shell, which also works without issues on Windows through Cygwin, git-bash, and so on:
APP_ID=...
adb exec-out run-as "$APP_ID" tar -c --exclude "$APP_ID/subfolder" -C data/data "$APP_ID" > out.tarThe fact that I’m invoking adb is not particularly relevant.
What is relevant, is that the output of adb is binary; it is not text.
Everything worked fine, until someone tried to execute it from PowerShell.
The command can be easily adapted, and it seems to work as expected:
$APP_ID=...
adb exec-out run-as "$APP_ID" tar -c --exclude "$APP_ID/subfolder" -C data/data "$APP_ID" > out.tarUntil you try to open out.tar. Unless you changed some default settings of PowerShell, out.tar will be 4 times as big, and no archive utility will be able to open it.
Searching around, you might find following suggestion: change the encoding of Out-File to utf8, as using > is equivalent to using Out-File.
After trying it out, I noticed that ascii is even better in case of binary files, as the difference between PowerShell and POSIX shells is even smaller.
Thus I temporarely ended with the following commands:
$APP_ID=...
$PSDefaultParameterValues['Out-File:Encoding'] = 'ascii'
C:/path-to-adb/adb.exe exec-out run-as "$APP_ID" tar -c --exclude "$APP_ID/subfolder" -C data/data "$APP_ID" > out.tar
# or
$APP_ID=...
C:/path-to-adb/adb.exe exec-out run-as "$APP_ID" tar -c --exclude "$APP_ID/subfolder" -C data/data "$APP_ID" | Out-File -Encoding ascii out.tarBut it is still not good enough. After inspecting again the output of out.tar file, I noticed that all \n where converted to \r\n!
There is an option for avoiding this conversion in Out-File: -NoNewLine.
$APP_ID=...
$PSDefaultParameterValues['Out-File:Encoding'] = 'ascii'
$PSDefaultParameterValues['Out-File:NoNewLine'] = $true
C:/path-to-adb/adb.exe exec-out run-as "$APP_ID" tar -c --exclude "$APP_ID/subfolder" -C data/data "$APP_ID" > out.tar
# or
$APP_ID=...
C:/path-to-adb/adb.exe exec-out run-as "$APP_ID" tar -c --exclude "$APP_ID/subfolder" -C data/data "$APP_ID" | Out-File -Encoding ascii -NoNewLine out.tarAfter comparing the output, I could still find some differences!
Truth is, that saving the output of a command to a file is only half of the story.
PowerShell, trying to be helpful, parses the output of commands and converts them to UTF-16 before doing anything else.
Thus before the content is handled by Out-File, it has been already modified.
The only reliable solution I could find was to use Start-Process with -RedirectStandardOutput:
$APP_ID=...
Start-Process -NoNewWindow -Wait -FilePath C:/path-to-adb/adb.exe -ArgumentList "exec-out","run-as","$APP_ID","tar","-c","--exclude","$APP_ID/subfolder","-C","data/data","$APP_ID" -RedirectStandardOutput out.tarThere is a fix for PowerShell Core, the version that is not bundled with Windows.
The TLD;DR is that you should not redirect the binary output of a command to a file with the version of PowerShell that is bundled in Windows.
I also decided to change the default values of Out-File, normally I do not want utf-16 files with \r\n as newlines!
Thus I’ve added in my $PROFILE.AllUsersAllHosts the following lines
$PSDefaultParameterValues['Out-File:Encoding'] = 'ascii'
$PSDefaultParameterValues['Out-File:NoNewLine'] = $trueOne could also change the Encoding for all commands that accept that parameter:
$PSDefaultParameterValues['*:Encoding'] = 'ascii'I am currently no brave enough to set it; my fear is that it might break something else. After all, I use PowerShell rarely, and I’m confident that changing only the default of Out-File covers most of my needs.
If you have questions, comments, or found typos, the notes are not clear, or there are some errors; then just contact me.